jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
7,483 Commits 1 Branch 0 Tags
29cdaddd8606a76e2eb6d6bbf4fdcf06fe8037ad
Commit Graph
1230 Commits
Author SHA1 Message Date
evazion 29cdaddd86 PostSetPresenters::Post#related_posts: clean up metatag parsing (#2894). 
* Fix `#related_tags` to use `Tag.has_metatag?`.
* Fix Tag::SUBQUERY_METATAGS and Tag::METATAGS to be arrays instead of regexes.
 2018-09-20 19:23:47 -05:00
evazion 6fe883c316 posts index: clean up limit:<n> / order:random parsing (#2894) 
* Move the limit:<n> / order:random metatag parsing from the controller
  to the post set.

* Introduce `Tag.has_metatag?` and use it to parse these metatags
  instead of using a regex (#2894).
 2018-09-20 19:23:47 -05:00
evazion 958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page. 
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
 2018-09-19 23:32:21 -05:00
evazion f4e08ef30d Downloads::File: fix SSRF inside is_cloudflare? (#2498). 
Fixes the banned IP check not being applied when sending the HEAD
request for is_cloudflare?.

Also fixes the `#size` method not using the uncached url (which meant
the bookmarklet could report the wrong filesize on artstation uploads).
 2018-09-19 20:11:53 -05:00
Albert Yi 9e1e73ec4c migrate blacklists during mass updates (#2251) 2018-09-19 16:25:06 -07:00
Albert Yi ed5b2ad443 throw error on counts/posts endpoint during a timeout (#2170) 2018-09-19 13:34:50 -07:00
evazion 2f17082e73 Downloads::File: fix SSRF when following redirects (#2498). 
Fixes the banned IP check not being applied when following redirects:

  http://danbooru.donmai.us/uploads/new?url=http://httpbin.org/redirect-to%3Furl=http://127.0.0.1/test.jpg
 2018-09-18 16:00:03 -05:00
evazion 99221e4028 Downloads::File: fix SSRF attack when fetching remote size (#2498). 
Fixes the banned IP check not being applied when fetching the remote
file size. This allowed one to trick Danbooru into sending HEAD requests
to private IPs:

  http://danbooru.donmai.us/uploads/new?url=http://127.0.0.1/test.jpg
 2018-09-18 12:16:27 -05:00
evazion 9cdfbba6c2 Fix #3910: Corrupted images during upload. 
Use a fresh tempfile for each download attempt instead of reusing the same
file (and having to rewind/truncate it after each failed attempt).
 2018-09-18 10:01:44 -05:00
evazion d3c135ec72 Downloads::File#http_get_streaming: clean up retry logic. 
Replace handrolled retry logic with retriable gem (already pulled in by another gem).
 2018-09-18 09:44:15 -05:00
evazion 96e89cecfb tests: move twitter canonical url test. 2018-09-17 23:27:53 -05:00
Albert Yi 79f53c25ea skip tests that throw net::opentimeout 2018-09-17 17:00:46 -07:00
Albert Yi aea2d77c3a Merge pull request #3907 from evazion/fix-3900 
#3900: Allow to search for urls associated with artists using wildcards
 2018-09-17 12:03:15 -07:00
evazion 4a99cb098f moebooru: use the image url as the canonical url. 2018-09-16 21:00:11 -05:00
evazion d9ce953752 Fix #3906: Moebooru strategy raises NotImplementedError. 2018-09-16 21:00:11 -05:00
evazion cae78fa8ee moebooru: move tests from unit/downloads to unit/sources. 2018-09-16 21:00:11 -05:00
evazion f135a7c064 twitter: normalize canonical urls. 
Normalize http://mobile.twitter.com to http://twitter.com in canonical urls.
 2018-09-16 15:03:47 -05:00
evazion bd47641601 twitter: don't fail when api key isn't configured. 2018-09-16 15:03:47 -05:00
evazion 325120ee51 twitter: fix parsing of the artist name from the url. 
Fixes URLs like https://twitter.com/intent/user?user_id=123 being
incorrectly normalized to http://twitter.com/intent/ in artist entries.

Also fixes the artist name to be taken from the url when it can't be
obtained from the api (when the tweet is deleted).
 2018-09-16 15:03:23 -05:00
evazion c9b3c8d217 artist urls: add tests for controller + artist url search. 2018-09-15 19:58:55 -05:00
evazion 761f2649af artists: add more invalid name tests (#3901). 2018-09-14 13:32:45 -05:00
Albert Yi 6dd64da9ba fix invalid content type leaving upload stuck in preprocessing status (#3896) 2018-09-13 13:50:18 -07:00
Albert Yi f487b2a2c6 Merge pull request #3889 from evazion/fix-replace-artist-finder 
Cleanup artist finder
 2018-09-12 11:44:42 -07:00
evazion fbd5f6b7f2 pixiv: fix preview_urls for ugoiras (#3891). 2018-09-12 00:43:10 -05:00
evazion 37fc215d75 pixiv: fix preview_urls to use correct url (#3891). 2018-09-11 23:55:46 -05:00
evazion 583f8457f0 artists: clean up artist finding logic. 
Rename Artist#find_all_by_url to url_matches and drop previous
url_matches method, along with find_artists and search_for_profile.

Previously find_artists tried to lookup the url, referer url, and profile
url in turn until an artist match was found. This was wasteful, because
the source strategy already knows which url to lookup (usually the profile
url). If that url doesn't find a match, then the artist doesn't exist.
 2018-09-11 20:14:46 -05:00
evazion 10ca4dd3ad artists: replace artist finder with fetch source data. 
* On posts, automatically trigger "Fetch source data" when clicking the
  Edit tab, instead of triggering the artist finder button. This way we
  find both the artist and the translated tags in one ajax call.

* Remove the "Artist" finder button next to the source field. This isn't
  necessary given that "Fetch source data" finds the artist itself.

* Remove the /artists/finder.json API endpoint. This is no longer used
  after removing the "Artist" finder button.
 2018-09-11 20:09:25 -05:00
Albert Yi a5df178bcc Merge pull request #3886 from r888888888/source-api-caching 
cache api clients
 2018-09-11 17:34:25 -07:00
Albert Yi 5f06952ed1 fix upload service test 2018-09-11 15:54:06 -07:00
Albert Yi 4972c998f8 rely on preview urls if available for gallery 2018-09-11 15:06:12 -07:00
Albert Yi 266c7c0d5b cache api clients 2018-09-11 14:19:17 -07:00
Albert Yi e0e85a3139 skip additional tests failures that are caused by networking issues 2018-09-11 13:43:34 -07:00
Albert Yi f16c3a3f40 fix nijie specs 2018-09-11 13:27:00 -07:00
evazion 9a980367f6 twitter: normalize artist commentaries to nfkc (#3719) 
Fixes hashtags not being interpreted when the author uses a fullwidth
number sign (＃, U+FF03).

ref: https://github.com/r888888888/danbooru/issues/3719#issuecomment-419535610
 2018-09-10 21:45:50 -05:00
evazion c9300cc54e sta.sh: add tests + docs. 2018-09-10 19:26:53 -05:00
evazion 0d7b7e6e85 twitter tests: fix dead tweet. 2018-09-09 14:10:43 -05:00
evazion b924c2bb9c nijie: fix artist url normalization. 2018-09-09 13:17:52 -05:00
evazion a67edb8783 deviantart: fix artist finder for artist names with underscores. 
Fix the artist finder for urls like this:

  https://orig00.deviantart.net/4274/f/2010/230/8/a/pkmn_king_and_queen_by_mikoto_chan.jpg

that don't contain a deviantart id but do contain the artist name.
 2018-09-07 12:23:48 -05:00
evazion 610391205f deviantart: fix artist finder for profile urls missing the 'www'. 
Fix the artist finder to work when the profile url in the artist entry
is missing the 'www'. Example:

  https://deviantart.com/noizave
  https://www.deviantart.com/noizave
 2018-09-07 11:36:48 -05:00
evazion aee1906761 Fix #3738: Artist URL search should be case-insensitive for domains. 2018-09-05 19:14:24 -05:00
evazion c2cf6a5441 Fix #3871: Autocomplete can return duplicate tags. 2018-09-05 18:56:30 -05:00
Albert Yi 69b7c9efd0 Merge pull request #3838 from evazion/fix-3837 
Fix #3837: Remove expired bans.
 2018-09-05 12:03:46 -07:00
Albert Yi a41d006380 fix pool ordering js (#3869), fix forum topic tests 2018-09-05 12:01:59 -07:00
Albert Yi 3025db66d7 switch fuzzy autocomplete to spellcorrect intent 2018-09-05 09:41:25 -07:00
Albert Yi 314341773c fix tests 2018-09-04 18:23:49 -07:00
Albert Yi 30a5d745a0 additional spec fixes 2018-09-04 14:01:35 -07:00
Albert Yi a5943de418 Merge pull request #3868 from evazion/fix-3864 
Fix #3864: DeviantArt fetch source data failure
 2018-09-04 13:42:01 -07:00
Albert Yi 8ec96f42f7 fix specs 2018-09-04 13:38:09 -07:00
Albert Yi 0529ec9ad0 Merge pull request #3862 from evazion/fix-3857 
Refactor searching text attributes
 2018-09-04 13:25:40 -07:00
Albert Yi 4a56f8d160 fixes #3856 for pixiv fanbox urls 2018-09-04 12:53:58 -07:00