jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
7,567 Commits 1 Branch 0 Tags
6e54c37b04db683b86de078674012f3312eca1a5
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
evazion 6e54c37b04 maintenance: add rake tasks for maintenance jobs. 2018-10-09 20:11:24 -05:00
evazion 5201954413 maintenance: consolidate maintenance tasks in logical/maintenance.rb. 2018-10-09 20:11:23 -05:00
evazion 53a51310a3 tumblr: add canonical url tests (#3385). 2018-10-09 12:55:48 -05:00
evazion 16b1b72da5 tumblr: fix video urls not being recognized. 2018-10-09 12:44:59 -05:00
evazion 184a5ebf3e tumblr: fix _640 images not being recognized (#3944). 
Fixes _640 images not being matched by the IMAGE regex and therefore not
being rewritten to the largest size.
 2018-10-09 12:44:59 -05:00
evazion d874c68419 tumblr: fix image_urls when api data is unavailable. 2018-10-09 12:44:59 -05:00
evazion b0d7d90103 tumblr: extract info from url when api data is unavailable. 
Derive the artist name / profile url / page url from the source URLs when
the API response is unavailable because the Tumblr post was deleted.

This fixes the artist finder to work on bad_tumblr_id posts.
 2018-10-09 12:44:59 -05:00
evazion 0c31a5d6a9 tumblr: don't fail when api data is unavailable (#3948). 
The api data is unavailable when the work is deleted (bad_tumblr_id), or
when the source is a direct image url with no page referer.
 2018-10-09 12:44:59 -05:00
evazion 4c55c809b0 tumblr: don't fail when api key isn't configured. 2018-10-09 12:44:59 -05:00
evazion 8aed1e19b9 tumblr: remove obsolete cdn rewrite rule. 
This cdn is no longer used by tumblr.
 2018-10-09 12:44:59 -05:00
evazion 5d77f1db08 daily maintenance: remove dead tag pruner code. 2018-10-06 14:27:53 -05:00
evazion c8d538f618 moebooru: delegate to substrategy based on post source (#3911). 
If the yande.re or konachan.com post has a source from a supported site,
for example Pixiv or Twitter, then delegate the artist and commentary
lookup to that substrategy.

Only do this for sources from recognized sites, not the null strategy.
 2018-10-06 14:27:49 -05:00
evazion e5a4193dd4 moebooru: support batch bookmarklet previews (#3911). 2018-10-06 00:58:22 -05:00
evazion fdb6e4ecee moebooru: rewrite konachan urls for Post#normalized_source (#3911). 2018-10-06 00:58:22 -05:00
evazion 864349dc7b moebooru: fetch tags (#3911). 2018-10-06 00:58:22 -05:00
evazion 7eee77d520 moebooru: proxy images to avoid hotlink protection (#3911). 
Fix image previews being broken on the upload page due to yande.re not
allowing hotlinking of images.
 2018-10-06 00:58:22 -05:00
evazion 9301bf2f11 tests: fix post expungement test. 
Delay jobs so the uploaded file isn't deleted immediately (see f6c928e660).
 2018-10-06 00:58:22 -05:00
evazion 1e710e1ae0 tests: fix typo in artist controller test. 2018-10-04 21:55:17 -05:00
evazion f6c928e660 tests: fix upload service tests. 
Fix regression in a45bd52e9. Delay jobs so that that the dead upload
pruner doesn't immediately prune new uploads.
 2018-10-04 21:52:18 -05:00
evazion bb5f291112 artists: don't create new version when nothing changed. 
Fix an issue where saving an artist entry without changing anything
would create a new artist version.
 2018-10-04 20:01:38 -05:00
evazion 03cc3dfa50 artists: fix editing invalid urls in artist entries (fix #3720, #3927, #3781) 
Convert to an autosave association on urls. This ensures that when we
save the artist we only validate the added urls, not bad urls that we're
trying to remove, and that url validation errors are propagated up to
the artist object.

This also fixes invalid urls being saved in the artist history despite
validation failing (#3720).
 2018-10-04 19:49:16 -05:00
evazion c78dece411 tag corrections: remove distributed cache logic (#3943). 
Remove logic for displaying the state of the tag category cache on both
servers.
 2018-10-04 13:46:10 -05:00
evazion 4ab97a01bd Fix #3943: Eliminate legacy code for distributed memcached servers. 2018-10-04 13:22:49 -05:00
evazion 060112293b /users: add more search params. 2018-10-04 13:03:18 -05:00
evazion 0966a290e4 Fix #3941: Add ability to search users by inviter. 2018-10-04 12:44:59 -05:00
evazion 4fd949ca8d news updates: fix news updates not staying hidden. 
ref: https://danbooru.donmai.us/forum_posts/151011

Bug: `Cookie.get("news-ticker") === key` always returned false because
the cookie was a string but the key was an integer.

Regressed in f72b32b27b after switching from `==` to `===`.
 2018-10-02 13:28:41 -05:00
evazion d727e0e34c Fix #3939: Quoting a post with apostrophes replaces them with '. 2018-10-01 20:24:00 -05:00
evazion bd3fb7d70e Post#normalized_source: fix for yande.re urls. 
Fix regex for yande.re urls like this:

    https://files.yande.re/image/b66909b940e8d77accab7c9b25aa4dc3/yande.re%20377828.png
 2018-10-01 20:03:21 -05:00
evazion 0c5452cdc3 Fix #3937: Blank lines in tagbox if certain taggroups are absent 2018-10-01 19:47:39 -05:00
evazion eeadd1ef11 artists/tags subnav: prefer direct tag count over Post#fast_count. 2018-10-01 17:51:48 -05:00
evazion 35eaf28822 Post#fast_count: lower timeout to 1 second. 2018-10-01 12:55:57 -05:00
evazion ccb57e802c Fix #3934: Post#fast_count has very slow worst case behavior. 2018-10-01 12:55:57 -05:00
evazion 215591403e uploads: fix 'rating:safe' not assigning the rating (#3929). 2018-10-01 11:09:00 -05:00
evazion 2ae7ec42df Post#fast_count: raise min cache lifetime to 3 minutes (#3925). 2018-10-01 10:38:15 -05:00
evazion 4425150298 Post#fast_count: fix cache expiry not being set (#3925). 2018-10-01 10:38:15 -05:00
evazion 151fa00933 tests: fix tests for /comments/new, /post_flags/new. 2018-10-01 10:38:15 -05:00
evazion 88a177e1d5 TagSetPresenter: refactor humanized_essential_tag_string. 
Move Post#humanized_essential_tag_string to TagSetPresenter#humanized_essential_tag_string.

This allows humanized_essential_tag_string to reuse the same set of tags
already fetched by the tag set presenter for the sidebar.

This avoids fetching the tag categories from memcache again (via
Post#typed_tags) when we're already fetched the tags once before.

This also means it's no longer necessary to cache humanized_essential_tag_string
itself in memcache, since it can be generated as quickly as the sidebar taglist.
 2018-09-30 21:52:24 -05:00
evazion 739bb1270c TagSetPresenter: refactor tag string for post edit form. 
Move PostPresenter#categorized_tag_groups to TagSetPresenter#split_tag_list_text.

This allows split_tag_list_text to reuse the same set of tags already
fetched by the tag set presenter for the sidebar.

This avoids a memcache call to get the tag categories when rendering the
tag string for the post edit form.
 2018-09-30 21:52:24 -05:00
evazion b1f2096d72 TagSetPresenter: refactor *_tag_list_html to avoid memcache calls. 
Refactor the tag set presenter to get both the tag categories and the
tag counts in the same call to the database, instead of getting the
counts from the db and the categories from memcache.
 2018-09-30 21:52:24 -05:00
evazion 99632d5e8a TagSetPresenter: refactor to pass options explicitly. 
Refactor tag_list_html, split_tag_list_html, and inline_tag_list_html to
take the `show_extra_links` and `current_query` options explicitly,
rather than implicitly relying on CurrentUser or taking `params[:tags]`
from the template.
 2018-09-30 21:52:24 -05:00
evazion 2cc4e35cc9 Fix #3930: Can't remove children from a parent post through child: metatag. 
Add `child:none` and `-child:123` edit metatags. Allow using ranges with
these metatags (e.g. `-child:1..10`, `child:1,3,5`).
 2018-09-29 20:42:38 -05:00
evazion 929fd29c33 Fix post archive test failures. 
Fix test failures due to Post#versions not being defined when post
archives wasn't enabled because the SQS url wasn't configured.
 2018-09-29 20:42:38 -05:00
evazion dfb7cf6994 comments: add standalone new comment form. 
Add standalone /comments/new page to allow commenting if javascript is
disabled.
 2018-09-29 20:42:38 -05:00
evazion 9ac59aeadd comments.js: clean up comment quoting. 
Replace Comment.quote with a remote new.js.erb call.
 2018-09-29 20:42:38 -05:00
evazion 75232c0cd4 comments.js: clean up unnecessary code. 
* Remove Dtext import (no longer used).
* Remove create.js.erb template (not used).
* Don't hide `#hidden-comments-notice-for-...` (didn't exist).
 2018-09-29 20:42:38 -05:00
evazion 984cd0432c Fix #3928: fix case sensitivity in metatags. 2018-09-27 23:42:30 -05:00
evazion 654d40875f posts/show: only render detailed rejection dialog for approvers. 2018-09-26 20:46:22 -05:00
evazion a5438aaf37 Fix dtext preview button to use event delegation. 
Fixes the dtext preview button and dtext expandable to work with
dynamically loaded dtext forms.
 2018-09-26 20:01:49 -05:00
evazion 6aecbf4a7e posts/show: skip rendering dialogs for anon users. 
Anonymous users can't use any of the artist commentary / add to pool /
add to favgroup dialogs, so rendering them is unnecessary.
 2018-09-26 20:01:49 -05:00
evazion f886f2fdb3 posts/show: load flag/appeal/replacement dialogs via ajax (#3922). 2018-09-26 20:01:14 -05:00
evazion d1e9f9c3ce Fix broken test artist controller tests, tag alias test. 2018-09-26 00:42:47 -05:00
evazion f5c9616f62 tests: fix broken tests. 2018-09-24 17:34:08 -05:00
evazion d9063a9f2a nijie: support preview urls (#3919). 2018-09-24 17:08:37 -05:00
evazion 99babe19f3 views/static: removed unused templates. 2018-09-23 20:18:34 -05:00
evazion d188e5be33 wiki_page.rb: remove dead methods. 2018-09-23 20:13:51 -05:00
evazion 52704ef6cd nijie: take artist id from url when possible (#3919) 
Fixes the artist finder to work on Nijie posts that have been deleted.
 2018-09-23 20:08:14 -05:00
evazion 5525bbe1ca nijie: normalize all thumbnail urls (#3919). 2018-09-23 20:08:14 -05:00
evazion d294514dc0 nijie: don't crash on invalid urls or deleted works (#3919). 2018-09-23 20:08:14 -05:00
evazion b6228505aa nijie: fix page_url method. 
The id in a bare image url is the member id, not the illust id.
 2018-09-23 20:08:13 -05:00
evazion b4cf423058 Clean backtraces in uploads/show, static/error.json. 2018-09-23 20:08:13 -05:00
evazion 3414d27c78 post replacements: fix self-replacements; don't swallow upload errors. 2018-09-23 10:51:16 -05:00
evazion 39fe5273a9 Fix #3917: Uploads: validate file before resizing/distributing. 2018-09-23 10:51:16 -05:00
evazion 39f9e01b13 nicoseiga: fix canonical_url to use the image url. 2018-09-22 11:07:18 -05:00
evazion 09a8198979 /artists: add wildcard, regex search to url field (#3900) 
Allow searching the URL field by regex or by wildcard.

If the query looks like `/twitter/` do a regex search, otherwise if it
looks like `http://www.twitter.com/*` do a wildcard search, otherwise if
it looks like an url do an artist finder search, lastly if it looks like
`twitter` do a `*twitter*` search.
 2018-09-21 21:19:01 -05:00
evazion a4608daf38 /artists: add more search options for other names, group name. 
Add these search params:

* /artists?search[<field>]=
* /artists?search[<field>_eq]=
* /artists?search[<field>_not_eq]=
* /artists?search[<field>_like]=
* /artists?search[<field>_not_like]=
* /artists?search[<field>_ilike]=
* /artists?search[<field>_not_ilike]=
* /artists?search[<field>_regex]=
* /artists?search[<field>_not_regex]=

where `<field>` can be `name`, `group_name`, or `other_names`.

Remove these search params:

* /artists?search[name_matches]=
* /artists?search[other_names_match]=
* /artists?search[group_name_matches]=

`/artists?search[<field>_like]=` effectively does the same thing that
these searches did.
 2018-09-21 20:55:14 -05:00
evazion f917b83d6f /artists: drop deprecated search syntax, add regex search for names. 
Drop support for the following pseudo-metatags in the Name field in the
artists search form:

* name:<name>
* other:<other name>
* group:<group name>
* status:banned
* status:active
* http://www.example.com

Instead, make the Name field do a wildcard search against the artist
name, group name, or other names. If the query looks like `/regex/`,
then do a regex search against any of these names.

/artists?search[name] now does a literal exact match and
/artists?search{any_name_matches] does the above wildcard/regex search.
 2018-09-21 20:51:53 -05:00
evazion 237ab9b782 dmail filters: fix filters being case sensitive. 2018-09-20 19:51:59 -05:00
evazion 03abbd0683 Fix #2894: Use [[:space:]] instead of \s in regexes. 2018-09-20 19:24:38 -05:00
evazion 29cdaddd86 PostSetPresenters::Post#related_posts: clean up metatag parsing (#2894). 
* Fix `#related_tags` to use `Tag.has_metatag?`.
* Fix Tag::SUBQUERY_METATAGS and Tag::METATAGS to be arrays instead of regexes.
 2018-09-20 19:23:47 -05:00
evazion 6fe883c316 posts index: clean up limit:<n> / order:random parsing (#2894) 
* Move the limit:<n> / order:random metatag parsing from the controller
  to the post set.

* Introduce `Tag.has_metatag?` and use it to parse these metatags
  instead of using a regex (#2894).
 2018-09-20 19:23:47 -05:00
evazion 235271706b Fix #3913: Remove legacy /post/create.xml endpoint. 2018-09-20 12:25:16 -05:00
evazion 958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page. 
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
 2018-09-19 23:32:21 -05:00
evazion f4e08ef30d Downloads::File: fix SSRF inside is_cloudflare? (#2498). 
Fixes the banned IP check not being applied when sending the HEAD
request for is_cloudflare?.

Also fixes the `#size` method not using the uncached url (which meant
the bookmarklet could report the wrong filesize on artstation uploads).
 2018-09-19 20:11:53 -05:00
evazion 2f17082e73 Downloads::File: fix SSRF when following redirects (#2498). 
Fixes the banned IP check not being applied when following redirects:

  http://danbooru.donmai.us/uploads/new?url=http://httpbin.org/redirect-to%3Furl=http://127.0.0.1/test.jpg
 2018-09-18 16:00:03 -05:00
evazion 99221e4028 Downloads::File: fix SSRF attack when fetching remote size (#2498). 
Fixes the banned IP check not being applied when fetching the remote
file size. This allowed one to trick Danbooru into sending HEAD requests
to private IPs:

  http://danbooru.donmai.us/uploads/new?url=http://127.0.0.1/test.jpg
 2018-09-18 12:16:27 -05:00
evazion 488b1fd994 Downloads::File: memoize strategy. 
Also remove unused options param from constructor
 2018-09-18 10:22:15 -05:00
evazion 9cdfbba6c2 Fix #3910: Corrupted images during upload. 
Use a fresh tempfile for each download attempt instead of reusing the same
file (and having to rewind/truncate it after each failed attempt).
 2018-09-18 10:01:44 -05:00
evazion d3c135ec72 Downloads::File#http_get_streaming: clean up retry logic. 
Replace handrolled retry logic with retriable gem (already pulled in by another gem).
 2018-09-18 09:44:15 -05:00
evazion 96e89cecfb tests: move twitter canonical url test. 2018-09-17 23:27:53 -05:00
evazion 5f328c842a /artist_urls: fix timestamp formatting. 2018-09-17 21:01:42 -05:00
evazion 823f78af49 autocomplete.js: add data attrs on autocomplete results (#3902). 2018-09-17 18:18:33 -05:00
evazion 2d5f6b8a35 Fix #3902: Add source of tag autocomplete to the results. 2018-09-17 18:07:48 -05:00
evazion 4a99cb098f moebooru: use the image url as the canonical url. 2018-09-16 21:00:11 -05:00
evazion d9ce953752 Fix #3906: Moebooru strategy raises NotImplementedError. 2018-09-16 21:00:11 -05:00
evazion cae78fa8ee moebooru: move tests from unit/downloads to unit/sources. 2018-09-16 21:00:11 -05:00
evazion f135a7c064 twitter: normalize canonical urls. 
Normalize http://mobile.twitter.com to http://twitter.com in canonical urls.
 2018-09-16 15:03:47 -05:00
evazion bd47641601 twitter: don't fail when api key isn't configured. 2018-09-16 15:03:47 -05:00
evazion 325120ee51 twitter: fix parsing of the artist name from the url. 
Fixes URLs like https://twitter.com/intent/user?user_id=123 being
incorrectly normalized to http://twitter.com/intent/ in artist entries.

Also fixes the artist name to be taken from the url when it can't be
obtained from the api (when the tweet is deleted).
 2018-09-16 15:03:23 -05:00
evazion 4a84ec9e26 artist urls: add trigram indexes on urls. 2018-09-15 19:58:55 -05:00
evazion c9b3c8d217 artist urls: add tests for controller + artist url search. 2018-09-15 19:58:55 -05:00
evazion 3afc0b3a78 artist urls: add more url search params for /artist_urls. 
Adds these search params:

* /artist_urls?search[url]=...
* /artist_urls?search[url_eq]=...
* /artist_urls?search[url_not_eq]=...
* /artist_urls?search[url_like]=...
* /artist_urls?search[url_ilike]=...
* /artist_urls?search[url_not_like]=...
* /artist_urls?search[url_not_ilike]=...
* /artist_urls?search[url_regex]=...
* /artist_urls?search[url_not_regex]=...

and likewise for normalized_url.
 2018-09-15 19:58:54 -05:00
evazion c06af060f9 artist urls: add artist, url_matches search params to /artist_urls. 2018-09-15 19:58:31 -05:00
evazion 1fce794b99 artist urls: add /artist_urls index page. 2018-09-15 19:58:05 -05:00
evazion 761f2649af artists: add more invalid name tests (#3901). 2018-09-14 13:32:45 -05:00
evazion bbb233b93f css: remove unused or redundant rules. 2018-09-14 13:31:59 -05:00
evazion 0aecc9d34c css: fix 020_base.scss getting imported twice. 
This import caused all in the rules in 020_base.scss to be included
twice in the final stylesheet.
 2018-09-14 10:31:30 -05:00
evazion a5ef97a608 css: remove obsolete/unused mixins. 2018-09-14 00:34:34 -05:00
evazion 42f97bf3e5 Fix external link icons (#3890). 2018-09-14 00:21:08 -05:00
evazion 26674ec88a font awesome: switch from js+svg to css+webfont version (fix #3890). 2018-09-14 00:18:30 -05:00
evazion de78c5fd68 /recommended_posts: enable blacklists. 2018-09-13 23:06:24 -05:00