jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
11,646 Commits 1 Branch 0 Tags
f7794de0b7791f0008d239675e4aaf90fabe2e9a
Commit Graph

2555 Commits

Author SHA1 Message Date
evazion
f6c928e660 tests: fix upload service tests. 
Fix regression in a45bd52e9. Delay jobs so that that the dead upload
pruner doesn't immediately prune new uploads.
 2018-10-04 21:52:18 -05:00
evazion
bb5f291112 artists: don't create new version when nothing changed. 
Fix an issue where saving an artist entry without changing anything
would create a new artist version.
 2018-10-04 20:01:38 -05:00
evazion
03cc3dfa50 artists: fix editing invalid urls in artist entries (fix #3720, #3927, #3781) 
Convert to an autosave association on urls. This ensures that when we
save the artist we only validate the added urls, not bad urls that we're
trying to remove, and that url validation errors are propagated up to
the artist object.

This also fixes invalid urls being saved in the artist history despite
validation failing (#3720).
 2018-10-04 19:49:16 -05:00
evazion
bd3fb7d70e Post#normalized_source: fix for yande.re urls. 
Fix regex for yande.re urls like this:

    https://files.yande.re/image/b66909b940e8d77accab7c9b25aa4dc3/yande.re%20377828.png
 2018-10-01 20:03:21 -05:00
evazion
0c5452cdc3 Fix #3937: Blank lines in tagbox if certain taggroups are absent 2018-10-01 19:47:39 -05:00
evazion
215591403e uploads: fix 'rating:safe' not assigning the rating (#3929). 2018-10-01 11:09:00 -05:00
evazion
2ae7ec42df Post#fast_count: raise min cache lifetime to 3 minutes (#3925). 2018-10-01 10:38:15 -05:00
evazion
4425150298 Post#fast_count: fix cache expiry not being set (#3925). 2018-10-01 10:38:15 -05:00
evazion
151fa00933 tests: fix tests for /comments/new, /post_flags/new. 2018-10-01 10:38:15 -05:00
evazion
2cc4e35cc9 Fix #3930: Can't remove children from a parent post through child: metatag. 
Add `child:none` and `-child:123` edit metatags. Allow using ranges with
these metatags (e.g. `-child:1..10`, `child:1,3,5`).
 2018-09-29 20:42:38 -05:00
evazion
984cd0432c Fix #3928: fix case sensitivity in metatags. 2018-09-27 23:42:30 -05:00
evazion
d1e9f9c3ce Fix broken test artist controller tests, tag alias test. 2018-09-26 00:42:47 -05:00
evazion
f5c9616f62 tests: fix broken tests. 2018-09-24 17:34:08 -05:00
evazion
d9063a9f2a nijie: support preview urls (#3919). 2018-09-24 17:08:37 -05:00
evazion
5525bbe1ca nijie: normalize all thumbnail urls (#3919). 2018-09-23 20:08:14 -05:00
evazion
d294514dc0 nijie: don't crash on invalid urls or deleted works (#3919). 2018-09-23 20:08:14 -05:00
evazion
b6228505aa nijie: fix page_url method. 
The id in a bare image url is the member id, not the illust id.
 2018-09-23 20:08:13 -05:00
evazion
3414d27c78 post replacements: fix self-replacements; don't swallow upload errors. 2018-09-23 10:51:16 -05:00
evazion
39fe5273a9 Fix #3917: Uploads: validate file before resizing/distributing. 2018-09-23 10:51:16 -05:00
evazion
39f9e01b13 nicoseiga: fix canonical_url to use the image url. 2018-09-22 11:07:18 -05:00
evazion
09a8198979 /artists: add wildcard, regex search to url field (#3900) 
Allow searching the URL field by regex or by wildcard.

If the query looks like `/twitter/` do a regex search, otherwise if it
looks like `http://www.twitter.com/*` do a wildcard search, otherwise if
it looks like an url do an artist finder search, lastly if it looks like
`twitter` do a `*twitter*` search.
 2018-09-21 21:19:01 -05:00
evazion
a4608daf38 /artists: add more search options for other names, group name. 
Add these search params:

* /artists?search[<field>]=
* /artists?search[<field>_eq]=
* /artists?search[<field>_not_eq]=
* /artists?search[<field>_like]=
* /artists?search[<field>_not_like]=
* /artists?search[<field>_ilike]=
* /artists?search[<field>_not_ilike]=
* /artists?search[<field>_regex]=
* /artists?search[<field>_not_regex]=

where `<field>` can be `name`, `group_name`, or `other_names`.

Remove these search params:

* /artists?search[name_matches]=
* /artists?search[other_names_match]=
* /artists?search[group_name_matches]=

`/artists?search[<field>_like]=` effectively does the same thing that
these searches did.
 2018-09-21 20:55:14 -05:00
evazion
f917b83d6f /artists: drop deprecated search syntax, add regex search for names. 
Drop support for the following pseudo-metatags in the Name field in the
artists search form:

* name:<name>
* other:<other name>
* group:<group name>
* status:banned
* status:active
* http://www.example.com

Instead, make the Name field do a wildcard search against the artist
name, group name, or other names. If the query looks like `/regex/`,
then do a regex search against any of these names.

/artists?search[name] now does a literal exact match and
/artists?search{any_name_matches] does the above wildcard/regex search.
 2018-09-21 20:51:53 -05:00
evazion
237ab9b782 dmail filters: fix filters being case sensitive. 2018-09-20 19:51:59 -05:00
evazion
03abbd0683 Fix #2894: Use [[:space:]] instead of \s in regexes. 2018-09-20 19:24:38 -05:00
evazion
29cdaddd86 PostSetPresenters::Post#related_posts: clean up metatag parsing (#2894). 
* Fix `#related_tags` to use `Tag.has_metatag?`.
* Fix Tag::SUBQUERY_METATAGS and Tag::METATAGS to be arrays instead of regexes.
 2018-09-20 19:23:47 -05:00
evazion
6fe883c316 posts index: clean up limit:<n> / order:random parsing (#2894) 
* Move the limit:<n> / order:random metatag parsing from the controller
  to the post set.

* Introduce `Tag.has_metatag?` and use it to parse these metatags
  instead of using a regex (#2894).
 2018-09-20 19:23:47 -05:00
evazion
958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page. 
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
 2018-09-19 23:32:21 -05:00
evazion
f4e08ef30d Downloads::File: fix SSRF inside is_cloudflare? (#2498). 
Fixes the banned IP check not being applied when sending the HEAD
request for is_cloudflare?.

Also fixes the `#size` method not using the uncached url (which meant
the bookmarklet could report the wrong filesize on artstation uploads).
 2018-09-19 20:11:53 -05:00
Albert Yi
9e1e73ec4c migrate blacklists during mass updates (#2251) 2018-09-19 16:25:06 -07:00
Albert Yi
ed5b2ad443 throw error on counts/posts endpoint during a timeout (#2170) 2018-09-19 13:34:50 -07:00
evazion
2f17082e73 Downloads::File: fix SSRF when following redirects (#2498). 
Fixes the banned IP check not being applied when following redirects:

  http://danbooru.donmai.us/uploads/new?url=http://httpbin.org/redirect-to%3Furl=http://127.0.0.1/test.jpg
 2018-09-18 16:00:03 -05:00
evazion
99221e4028 Downloads::File: fix SSRF attack when fetching remote size (#2498). 
Fixes the banned IP check not being applied when fetching the remote
file size. This allowed one to trick Danbooru into sending HEAD requests
to private IPs:

  http://danbooru.donmai.us/uploads/new?url=http://127.0.0.1/test.jpg
 2018-09-18 12:16:27 -05:00
evazion
9cdfbba6c2 Fix #3910: Corrupted images during upload. 
Use a fresh tempfile for each download attempt instead of reusing the same
file (and having to rewind/truncate it after each failed attempt).
 2018-09-18 10:01:44 -05:00
evazion
d3c135ec72 Downloads::File#http_get_streaming: clean up retry logic. 
Replace handrolled retry logic with retriable gem (already pulled in by another gem).
 2018-09-18 09:44:15 -05:00
evazion
96e89cecfb tests: move twitter canonical url test. 2018-09-17 23:27:53 -05:00
Albert Yi
79f53c25ea skip tests that throw net::opentimeout 2018-09-17 17:00:46 -07:00
Albert Yi
aea2d77c3a Merge pull request #3907 from evazion/fix-3900 
#3900: Allow to search for urls associated with artists using wildcards
 2018-09-17 12:03:15 -07:00
evazion
4a99cb098f moebooru: use the image url as the canonical url. 2018-09-16 21:00:11 -05:00
evazion
d9ce953752 Fix #3906: Moebooru strategy raises NotImplementedError. 2018-09-16 21:00:11 -05:00
evazion
cae78fa8ee moebooru: move tests from unit/downloads to unit/sources. 2018-09-16 21:00:11 -05:00
evazion
f135a7c064 twitter: normalize canonical urls. 
Normalize http://mobile.twitter.com to http://twitter.com in canonical urls.
 2018-09-16 15:03:47 -05:00
evazion
bd47641601 twitter: don't fail when api key isn't configured. 2018-09-16 15:03:47 -05:00
evazion
325120ee51 twitter: fix parsing of the artist name from the url. 
Fixes URLs like https://twitter.com/intent/user?user_id=123 being
incorrectly normalized to http://twitter.com/intent/ in artist entries.

Also fixes the artist name to be taken from the url when it can't be
obtained from the api (when the tweet is deleted).
 2018-09-16 15:03:23 -05:00
evazion
c9b3c8d217 artist urls: add tests for controller + artist url search. 2018-09-15 19:58:55 -05:00
evazion
761f2649af artists: add more invalid name tests (#3901). 2018-09-14 13:32:45 -05:00
Albert Yi
6dd64da9ba fix invalid content type leaving upload stuck in preprocessing status (#3896) 2018-09-13 13:50:18 -07:00
Albert Yi
f487b2a2c6 Merge pull request #3889 from evazion/fix-replace-artist-finder 
Cleanup artist finder
 2018-09-12 11:44:42 -07:00
evazion
fbd5f6b7f2 pixiv: fix preview_urls for ugoiras (#3891). 2018-09-12 00:43:10 -05:00
evazion
37fc215d75 pixiv: fix preview_urls to use correct url (#3891). 2018-09-11 23:55:46 -05:00