Commit Graph
100 Commits
Author SHA1 Message Date
evazion 29b6121a07 pools: refactor #neighbors + fix broken #neighbors tests. 2018-11-04 19:40:57 -06:00
evazion 628341f7f0 Fix #3969: Translated tags should ignore artist tags. 2018-11-04 17:07:35 -06:00
evazion 19b92acabc Fix #3970: Make "Copy all notes" not builder exclusive. 2018-11-04 16:42:55 -06:00
evazion 0152cbe39e moebooru: fix delegation to nil strategy (fixup for 5cf6a43). 2018-11-04 15:24:22 -06:00
evazion c7e6c2a44a posts: fix pixiv id parsing (fixup for 5cf6a43) 2018-11-04 14:22:29 -06:00
evazion 5cf6a43918 sources: fix sources sometimes choosing wrong strategy (fix #3968)
Fix sources choosing the wrong strategy when the referer belongs to a
different site (for example, when uploading a twitter post with a pixiv
referer).

* Fix `match?` to only consider the main url, not the referer.

* Change `match?` to match against a list of domains given by the `domains` method.

* Change `match?` to an instance method.
2018-11-04 13:00:17 -06:00
evazion f5012464ab Fix #3965: Extraneous API attributes.
Remove the updater_id/updater_ip_addr virtual attributes from
pools/notes. Juss pass them in as params to create_version instead.
2018-10-30 15:41:17 -05:00
evazion 39374a70d3 posts/show: rename tags params to q in various places.
Fixup for 011a6f1f7.
2018-10-30 11:32:07 -05:00
evazion 77a576d5ed related tags: strip http:// from artist urls. 2018-10-25 23:38:24 -05:00
evazion 705217d337 Fix new dtext shortlinks redirecting to nonexistent pages.
Fix the following dtext shortlinks redirecting to nonexistent pages:

* flag #1
* appeal #1
* mod action #1
* bur #1
2018-10-25 23:00:43 -05:00
evazion a2186c19dc Fix positioning of tag edit dialog box.
Bug: if you open the tag edit dialog with shift+e, close it, then open
it again, it won't be vertically centered when it opens the second time.

Fix: hide the related tags section *before* opening the dialog, not after.
Hiding it afterwards shifts the layout, which uncenters the dialog.
2018-10-25 22:39:29 -05:00
evazion 6148cb39a2 Fix nested blank search params not being stripped from searches.
From https://danbooru.donmai.us/forum_topics/9127?page=258#forum_post_151308:

    When you do a user search (from https://danbooru.donmai.us/users/search)
    the results only include those with someone in the inviter field.

The bug was that nested blank search params (/users?search[inviter][name_matches]=)
didn't get stripped from the search.
2018-10-17 16:11:44 -05:00
evazion 6d293dfde2 related tags: fix style issues. 2018-10-16 13:09:22 -05:00
evazion b166c8116a related tags: fix unnecessary .erb extensions, remove unused ids. 2018-10-15 18:42:04 -05:00
evazion 33ca028927 related tags: fix initialization logic for recent/frequent/translated/artist columns.
Initialize the recent tags, frequent tags, translated tags, and artist
tags columns only once, when the related tags section is first shown.

The related tags section is shown automatically when the 'Edit' tab is
opened, or by default on the uploads page.

This fixes the /related_tags.js and /source.js calls being triggered
every time the tag edit dialog box or the 'Edit' tab was opened.
2018-10-15 18:42:04 -05:00
evazion 6223f89a99 related tags: refactor /related_tags.js to avoid updating unchanged columns.
Refactor the /related_tags.js call so that when the "Related tags"
button is clicked, it only replaces the columns that actually changed,
not the entire related tags section.

This avoids rebuilding the Recent / Frequent / Translated Tags / Artist
columns every time the "Related tags" button is clicked.
2018-10-15 18:41:32 -05:00
evazion d536f75505 related tags: eliminate recent_tags cookie (#3955). 2018-10-12 21:20:06 -05:00
evazion ae52b487e8 related tags: eliminate favorite_tags cookie (#3955). 2018-10-12 21:18:15 -05:00
evazion 4e2944ee06 related tags: build html server-side instead of client-side. 2018-10-12 21:18:15 -05:00
evazion 5a05961bde related tags: factor partials out from post, upload edit forms. 2018-10-12 21:18:15 -05:00
evazion e329764276 Fix #3952: Adding animated_gif tag to a ugoira post breaks the player. 2018-10-10 23:50:36 -05:00
evazion 60ea46e1e7 Add sample newrelic config. 2018-10-09 20:11:44 -05:00
evazion 5e57cccd18 maintenance: log errors to rails log and newrelic. 2018-10-09 20:11:42 -05:00
evazion 6e54c37b04 maintenance: add rake tasks for maintenance jobs. 2018-10-09 20:11:24 -05:00
evazion 5201954413 maintenance: consolidate maintenance tasks in logical/maintenance.rb. 2018-10-09 20:11:23 -05:00
evazion 53a51310a3 tumblr: add canonical url tests (#3385). 2018-10-09 12:55:48 -05:00
evazion 16b1b72da5 tumblr: fix video urls not being recognized. 2018-10-09 12:44:59 -05:00
evazion 184a5ebf3e tumblr: fix _640 images not being recognized (#3944).
Fixes _640 images not being matched by the IMAGE regex and therefore not
being rewritten to the largest size.
2018-10-09 12:44:59 -05:00
evazion d874c68419 tumblr: fix image_urls when api data is unavailable. 2018-10-09 12:44:59 -05:00
evazion b0d7d90103 tumblr: extract info from url when api data is unavailable.
Derive the artist name / profile url / page url from the source URLs when
the API response is unavailable because the Tumblr post was deleted.

This fixes the artist finder to work on bad_tumblr_id posts.
2018-10-09 12:44:59 -05:00
evazion 0c31a5d6a9 tumblr: don't fail when api data is unavailable (#3948).
The api data is unavailable when the work is deleted (bad_tumblr_id), or
when the source is a direct image url with no page referer.
2018-10-09 12:44:59 -05:00
evazion 4c55c809b0 tumblr: don't fail when api key isn't configured. 2018-10-09 12:44:59 -05:00
evazion 8aed1e19b9 tumblr: remove obsolete cdn rewrite rule.
This cdn is no longer used by tumblr.
2018-10-09 12:44:59 -05:00
evazion 5d77f1db08 daily maintenance: remove dead tag pruner code. 2018-10-06 14:27:53 -05:00
evazion c8d538f618 moebooru: delegate to substrategy based on post source (#3911).
If the yande.re or konachan.com post has a source from a supported site,
for example Pixiv or Twitter, then delegate the artist and commentary
lookup to that substrategy.

Only do this for sources from recognized sites, not the null strategy.
2018-10-06 14:27:49 -05:00
evazion e5a4193dd4 moebooru: support batch bookmarklet previews (#3911). 2018-10-06 00:58:22 -05:00
evazion fdb6e4ecee moebooru: rewrite konachan urls for Post#normalized_source (#3911). 2018-10-06 00:58:22 -05:00
evazion 864349dc7b moebooru: fetch tags (#3911). 2018-10-06 00:58:22 -05:00
evazion 7eee77d520 moebooru: proxy images to avoid hotlink protection (#3911).
Fix image previews being broken on the upload page due to yande.re not
allowing hotlinking of images.
2018-10-06 00:58:22 -05:00
evazion 9301bf2f11 tests: fix post expungement test.
Delay jobs so the uploaded file isn't deleted immediately (see f6c928e660).
2018-10-06 00:58:22 -05:00
evazion 1e710e1ae0 tests: fix typo in artist controller test. 2018-10-04 21:55:17 -05:00
evazion f6c928e660 tests: fix upload service tests.
Fix regression in a45bd52e9. Delay jobs so that that the dead upload
pruner doesn't immediately prune new uploads.
2018-10-04 21:52:18 -05:00
evazion bb5f291112 artists: don't create new version when nothing changed.
Fix an issue where saving an artist entry without changing anything
would create a new artist version.
2018-10-04 20:01:38 -05:00
evazion 03cc3dfa50 artists: fix editing invalid urls in artist entries (fix #3720, #3927, #3781)
Convert to an autosave association on urls. This ensures that when we
save the artist we only validate the added urls, not bad urls that we're
trying to remove, and that url validation errors are propagated up to
the artist object.

This also fixes invalid urls being saved in the artist history despite
validation failing (#3720).
2018-10-04 19:49:16 -05:00
evazion c78dece411 tag corrections: remove distributed cache logic (#3943).
Remove logic for displaying the state of the tag category cache on both
servers.
2018-10-04 13:46:10 -05:00
evazion 4ab97a01bd Fix #3943: Eliminate legacy code for distributed memcached servers. 2018-10-04 13:22:49 -05:00
evazion 060112293b /users: add more search params. 2018-10-04 13:03:18 -05:00
evazion 0966a290e4 Fix #3941: Add ability to search users by inviter. 2018-10-04 12:44:59 -05:00
evazion 4fd949ca8d news updates: fix news updates not staying hidden.
ref: https://danbooru.donmai.us/forum_posts/151011

Bug: `Cookie.get("news-ticker") === key` always returned false because
the cookie was a string but the key was an integer.

Regressed in f72b32b27b after switching from `==` to `===`.
2018-10-02 13:28:41 -05:00
evazion d727e0e34c Fix #3939: Quoting a post with apostrophes replaces them with '. 2018-10-01 20:24:00 -05:00
evazion bd3fb7d70e Post#normalized_source: fix for yande.re urls.
Fix regex for yande.re urls like this:

    https://files.yande.re/image/b66909b940e8d77accab7c9b25aa4dc3/yande.re%20377828.png
2018-10-01 20:03:21 -05:00
evazion 0c5452cdc3 Fix #3937: Blank lines in tagbox if certain taggroups are absent 2018-10-01 19:47:39 -05:00
evazion eeadd1ef11 artists/tags subnav: prefer direct tag count over Post#fast_count. 2018-10-01 17:51:48 -05:00
evazion 35eaf28822 Post#fast_count: lower timeout to 1 second. 2018-10-01 12:55:57 -05:00
evazion ccb57e802c Fix #3934: Post#fast_count has very slow worst case behavior. 2018-10-01 12:55:57 -05:00
evazion 215591403e uploads: fix 'rating:safe' not assigning the rating (#3929). 2018-10-01 11:09:00 -05:00
evazion 2ae7ec42df Post#fast_count: raise min cache lifetime to 3 minutes (#3925). 2018-10-01 10:38:15 -05:00
evazion 4425150298 Post#fast_count: fix cache expiry not being set (#3925). 2018-10-01 10:38:15 -05:00
evazion 151fa00933 tests: fix tests for /comments/new, /post_flags/new. 2018-10-01 10:38:15 -05:00
evazion 88a177e1d5 TagSetPresenter: refactor humanized_essential_tag_string.
Move Post#humanized_essential_tag_string to TagSetPresenter#humanized_essential_tag_string.

This allows humanized_essential_tag_string to reuse the same set of tags
already fetched by the tag set presenter for the sidebar.

This avoids fetching the tag categories from memcache again (via
Post#typed_tags) when we're already fetched the tags once before.

This also means it's no longer necessary to cache humanized_essential_tag_string
itself in memcache, since it can be generated as quickly as the sidebar taglist.
2018-09-30 21:52:24 -05:00
evazion 739bb1270c TagSetPresenter: refactor tag string for post edit form.
Move PostPresenter#categorized_tag_groups to TagSetPresenter#split_tag_list_text.

This allows split_tag_list_text to reuse the same set of tags already
fetched by the tag set presenter for the sidebar.

This avoids a memcache call to get the tag categories when rendering the
tag string for the post edit form.
2018-09-30 21:52:24 -05:00
evazion b1f2096d72 TagSetPresenter: refactor *_tag_list_html to avoid memcache calls.
Refactor the tag set presenter to get both the tag categories and the
tag counts in the same call to the database, instead of getting the
counts from the db and the categories from memcache.
2018-09-30 21:52:24 -05:00
evazion 99632d5e8a TagSetPresenter: refactor to pass options explicitly.
Refactor tag_list_html, split_tag_list_html, and inline_tag_list_html to
take the `show_extra_links` and `current_query` options explicitly,
rather than implicitly relying on CurrentUser or taking `params[:tags]`
from the template.
2018-09-30 21:52:24 -05:00
evazion 2cc4e35cc9 Fix #3930: Can't remove children from a parent post through child: metatag.
Add `child:none` and `-child:123` edit metatags. Allow using ranges with
these metatags (e.g. `-child:1..10`, `child:1,3,5`).
2018-09-29 20:42:38 -05:00
evazion 929fd29c33 Fix post archive test failures.
Fix test failures due to Post#versions not being defined when post
archives wasn't enabled because the SQS url wasn't configured.
2018-09-29 20:42:38 -05:00
evazion dfb7cf6994 comments: add standalone new comment form.
Add standalone /comments/new page to allow commenting if javascript is
disabled.
2018-09-29 20:42:38 -05:00
evazion 9ac59aeadd comments.js: clean up comment quoting.
Replace Comment.quote with a remote new.js.erb call.
2018-09-29 20:42:38 -05:00
evazion 75232c0cd4 comments.js: clean up unnecessary code.
* Remove Dtext import (no longer used).
* Remove create.js.erb template (not used).
* Don't hide `#hidden-comments-notice-for-...` (didn't exist).
2018-09-29 20:42:38 -05:00
evazion 984cd0432c Fix #3928: fix case sensitivity in metatags. 2018-09-27 23:42:30 -05:00
evazion 654d40875f posts/show: only render detailed rejection dialog for approvers. 2018-09-26 20:46:22 -05:00
evazion a5438aaf37 Fix dtext preview button to use event delegation.
Fixes the dtext preview button and dtext expandable to work with
dynamically loaded dtext forms.
2018-09-26 20:01:49 -05:00
evazion 6aecbf4a7e posts/show: skip rendering dialogs for anon users.
Anonymous users can't use any of the artist commentary / add to pool /
add to favgroup dialogs, so rendering them is unnecessary.
2018-09-26 20:01:49 -05:00
evazion f886f2fdb3 posts/show: load flag/appeal/replacement dialogs via ajax (#3922). 2018-09-26 20:01:14 -05:00
evazion d1e9f9c3ce Fix broken test artist controller tests, tag alias test. 2018-09-26 00:42:47 -05:00
evazion f5c9616f62 tests: fix broken tests. 2018-09-24 17:34:08 -05:00
evazion d9063a9f2a nijie: support preview urls (#3919). 2018-09-24 17:08:37 -05:00
evazion 99babe19f3 views/static: removed unused templates. 2018-09-23 20:18:34 -05:00
evazion d188e5be33 wiki_page.rb: remove dead methods. 2018-09-23 20:13:51 -05:00
evazion 52704ef6cd nijie: take artist id from url when possible (#3919)
Fixes the artist finder to work on Nijie posts that have been deleted.
2018-09-23 20:08:14 -05:00
evazion 5525bbe1ca nijie: normalize all thumbnail urls (#3919). 2018-09-23 20:08:14 -05:00
evazion d294514dc0 nijie: don't crash on invalid urls or deleted works (#3919). 2018-09-23 20:08:14 -05:00
evazion b6228505aa nijie: fix page_url method.
The id in a bare image url is the member id, not the illust id.
2018-09-23 20:08:13 -05:00
evazion b4cf423058 Clean backtraces in uploads/show, static/error.json. 2018-09-23 20:08:13 -05:00
evazion 3414d27c78 post replacements: fix self-replacements; don't swallow upload errors. 2018-09-23 10:51:16 -05:00
evazion 39fe5273a9 Fix #3917: Uploads: validate file before resizing/distributing. 2018-09-23 10:51:16 -05:00
evazion 39f9e01b13 nicoseiga: fix canonical_url to use the image url. 2018-09-22 11:07:18 -05:00
evazion 09a8198979 /artists: add wildcard, regex search to url field (#3900)
Allow searching the URL field by regex or by wildcard.

If the query looks like `/twitter/` do a regex search, otherwise if it
looks like `http://www.twitter.com/*` do a wildcard search, otherwise if
it looks like an url do an artist finder search, lastly if it looks like
`twitter` do a `*twitter*` search.
2018-09-21 21:19:01 -05:00
evazion a4608daf38 /artists: add more search options for other names, group name.
Add these search params:

* /artists?search[<field>]=
* /artists?search[<field>_eq]=
* /artists?search[<field>_not_eq]=
* /artists?search[<field>_like]=
* /artists?search[<field>_not_like]=
* /artists?search[<field>_ilike]=
* /artists?search[<field>_not_ilike]=
* /artists?search[<field>_regex]=
* /artists?search[<field>_not_regex]=

where `<field>` can be `name`, `group_name`, or `other_names`.

Remove these search params:

* /artists?search[name_matches]=
* /artists?search[other_names_match]=
* /artists?search[group_name_matches]=

`/artists?search[<field>_like]=` effectively does the same thing that
these searches did.
2018-09-21 20:55:14 -05:00
evazion f917b83d6f /artists: drop deprecated search syntax, add regex search for names.
Drop support for the following pseudo-metatags in the Name field in the
artists search form:

* name:<name>
* other:<other name>
* group:<group name>
* status:banned
* status:active
* http://www.example.com

Instead, make the Name field do a wildcard search against the artist
name, group name, or other names. If the query looks like `/regex/`,
then do a regex search against any of these names.

/artists?search[name] now does a literal exact match and
/artists?search{any_name_matches] does the above wildcard/regex search.
2018-09-21 20:51:53 -05:00
evazion 237ab9b782 dmail filters: fix filters being case sensitive. 2018-09-20 19:51:59 -05:00
evazion 03abbd0683 Fix #2894: Use [[:space:]] instead of \s in regexes. 2018-09-20 19:24:38 -05:00
evazion 29cdaddd86 PostSetPresenters::Post#related_posts: clean up metatag parsing (#2894).
* Fix `#related_tags` to use `Tag.has_metatag?`.
* Fix Tag::SUBQUERY_METATAGS and Tag::METATAGS to be arrays instead of regexes.
2018-09-20 19:23:47 -05:00
evazion 6fe883c316 posts index: clean up limit:<n> / order:random parsing (#2894)
* Move the limit:<n> / order:random metatag parsing from the controller
  to the post set.

* Introduce `Tag.has_metatag?` and use it to parse these metatags
  instead of using a regex (#2894).
2018-09-20 19:23:47 -05:00
evazion 235271706b Fix #3913: Remove legacy /post/create.xml endpoint. 2018-09-20 12:25:16 -05:00
evazion 958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page.
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
2018-09-19 23:32:21 -05:00
evazion f4e08ef30d Downloads::File: fix SSRF inside is_cloudflare? (#2498).
Fixes the banned IP check not being applied when sending the HEAD
request for is_cloudflare?.

Also fixes the `#size` method not using the uncached url (which meant
the bookmarklet could report the wrong filesize on artstation uploads).
2018-09-19 20:11:53 -05:00
evazion 2f17082e73 Downloads::File: fix SSRF when following redirects (#2498).
Fixes the banned IP check not being applied when following redirects:

  http://danbooru.donmai.us/uploads/new?url=http://httpbin.org/redirect-to%3Furl=http://127.0.0.1/test.jpg
2018-09-18 16:00:03 -05:00
evazion 99221e4028 Downloads::File: fix SSRF attack when fetching remote size (#2498).
Fixes the banned IP check not being applied when fetching the remote
file size. This allowed one to trick Danbooru into sending HEAD requests
to private IPs:

  http://danbooru.donmai.us/uploads/new?url=http://127.0.0.1/test.jpg
2018-09-18 12:16:27 -05:00
evazion 488b1fd994 Downloads::File: memoize strategy.
Also remove unused options param from constructor
2018-09-18 10:22:15 -05:00
evazion 9cdfbba6c2 Fix #3910: Corrupted images during upload.
Use a fresh tempfile for each download attempt instead of reusing the same
file (and having to rewind/truncate it after each failed attempt).
2018-09-18 10:01:44 -05:00