Commit Graph

703 Commits

Author SHA1 Message Date
evazion
d56b56a6a1 Drop post updates table. 2019-08-29 00:52:23 -05:00
evazion
b283281e5e comments: minimize sql queries.
Certain parts of comment rendering triggered sql queries that we didn't
really need to do. Rework things to avoid this.

* Preload comment creators in order to display commenter names with link_to_user.

* Preload comment votes in order to display "undo vote" links. Only preload
  votes for members since anonymous users can't vote and don't have "undo
  vote" links.

* Rework various conditionals to do the filtering in Ruby so that we
  avoid issuing any extra queries in sql.

* Avoid issuing any queries at all when the post doesn't have any
  comments (when last_commented_at is blank).
2019-08-20 21:55:25 -05:00
evazion
59b277ead1 users: drop id_to_name, name_to_id caching.
Changes:

* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
  to return these fields in the api.

id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.

name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.

Avoiding caching also avoids the need to keep these caches consistent.
2019-08-18 11:24:42 -05:00
evazion
868a2256d1 jobs: migrate file deletion jobs to ActiveJob. 2019-08-16 20:49:35 -05:00
evazion
817f5ecf9c jobs: drop favgroup expunge job.
Make `Post#expunge!` remove favgroups synchronously.
2019-08-16 20:49:34 -05:00
evazion
798d524e60 Post#tag_match: clean up read_only param.
* Drop /posts?ro=true param (broken).
* Clean up tag_match (rescuing PG::ConnectionBad didn't do anything, we
  just build the query here, we don't run it).
2019-08-16 00:26:00 -05:00
evazion
b9d35eaf2c Fix #3272: Unicode tags are still being allowed.
* Don't allow adding tags with invalid names when they already exist in
  the tags table.
* If an invalid tag is added, show an warning and ignore the tag instead
  of failing with a hard error.
* Move the _(cosplay) tag validation into the tag name validator.
2019-08-15 16:42:23 -05:00
evazion
d8749e7dfe search: save a query in fast_count (#4120).
During single-tag searches we look up the tag once in Post.fast_count
and later on when rendering the wiki excerpt. Doing a raw query here
meant that Rails couldn't cache the query, so it got ran again when
rendering the excerpt.
2019-08-12 13:38:45 -05:00
evazion
a6163258bf Post.fast_count: skip alias normalization.
Post.fast_count calls Tag.normalize_query to normalize the tag string
used for the pfc cache key. This slightly improves cache sharing at the
expense of an extra query during tag searches. The extra query isn't
worth it.
2019-08-12 13:38:45 -05:00
evazion
5231371100 posts: drop img.ly source normalization.
There are only two posts from this site and the site itself is defunct:

* https://danbooru.donmai.us/posts/1561685
* https://danbooru.donmai.us/posts/1412508
* https://img.ly/ofxl
2019-08-07 22:11:07 -05:00
evazion
7f482dc35b deviantart: normalize wixmp.com sources to page urls.
Normalize sources like this:

   https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/ab917938-d8c1-4b58-933d-0d38c390461f/ddcyi98-cbd39da2-f528-4b26-aadb-a16fe91442b2.jpg/v1/fill/w_1280,h_1760,q_100,strp/tifa_by_chubymi_ddcyi98-fullview.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsIm9iaiI6W1t7ImhlaWdodCI6Ijw9MTc2MCIsInBhdGgiOiJcL2ZcL2FiOTE3OTM4LWQ4YzEtNGI1OC05MzNkLTBkMzhjMzkwNDYxZlwvZGRjeWk5OC1jYmQzOWRhMi1mNTI4LTRiMjYtYWFkYi1hMTZmZTkxNDQyYjIuanBnIiwid2lkdGgiOiI8PTEyODAifV1dLCJhdWQiOlsidXJuOnNlcnZpY2U6aW1hZ2Uub3BlcmF0aW9ucyJdfQ.Uochlma4QJmLwL2ZGMmvTr4HMva4m4bCeF3vnyPSw4I

to this:

   https://www.deviantart.com/chubymi/art/Tifa-807825644

on the sidebar of the posts show page.
2019-08-06 12:04:05 -05:00
Albert Yi
d8aabff77b remove post keeper references 2019-01-31 15:45:06 -08:00
Albert Yi
d97622d1bb Merge pull request #4007 from evazion/fix-4004
Fix #4004: Add additional order by metatags for posts
2019-01-09 14:43:15 -08:00
evazion
fde398d37b posts: clarify missing artist tag warning (#4029). 2018-12-28 15:52:49 -06:00
evazion
b1335616dd search: add {flag,appeal,approval,replacement,child,pool}_count metatags (#4004). 2018-12-11 18:10:20 -06:00
evazion
4a1f0523a4 search: add comment_count, note_count metatags (#4004).
Add these metatags:

* comment_count
* deleted_comment_count
* active_comment_count
* note_count
* deleted_note_count
* active_note_count
* order:comment_count
* order:deleted_comment_count
* order:active_comment_count
* order:note_count
* order:deleted_note_count
* order:active_note_count
2018-12-11 18:10:20 -06:00
evazion
37b2214472 post_tags_match: replace joins with subqueries.
Refactor various post_tag_match methods to use subqueries instead of joins.

This simplifies things inside PostQueryBuilder, since now we can assume
we're always dealing with a Post relation, rather than some other table
joined with the posts table.
2018-12-11 18:10:20 -06:00
evazion
c7e6c2a44a posts: fix pixiv id parsing (fixup for 5cf6a43) 2018-11-04 14:22:29 -06:00
Albert Yi
e133a598a3 fixes #3936 2018-10-12 15:17:32 -07:00
evazion
e329764276 Fix #3952: Adding animated_gif tag to a ugoira post breaks the player. 2018-10-10 23:50:36 -05:00
evazion
fdb6e4ecee moebooru: rewrite konachan urls for Post#normalized_source (#3911). 2018-10-06 00:58:22 -05:00
evazion
bd3fb7d70e Post#normalized_source: fix for yande.re urls.
Fix regex for yande.re urls like this:

    https://files.yande.re/image/b66909b940e8d77accab7c9b25aa4dc3/yande.re%20377828.png
2018-10-01 20:03:21 -05:00
Albert Yi
02156f3f34 Merge pull request #3933 from evazion/fix-tag-set-presenter
Optimize tag set presenters
2018-10-01 12:55:05 -07:00
evazion
35eaf28822 Post#fast_count: lower timeout to 1 second. 2018-10-01 12:55:57 -05:00
evazion
ccb57e802c Fix #3934: Post#fast_count has very slow worst case behavior. 2018-10-01 12:55:57 -05:00
evazion
2ae7ec42df Post#fast_count: raise min cache lifetime to 3 minutes (#3925). 2018-10-01 10:38:15 -05:00
evazion
4425150298 Post#fast_count: fix cache expiry not being set (#3925). 2018-10-01 10:38:15 -05:00
evazion
88a177e1d5 TagSetPresenter: refactor humanized_essential_tag_string.
Move Post#humanized_essential_tag_string to TagSetPresenter#humanized_essential_tag_string.

This allows humanized_essential_tag_string to reuse the same set of tags
already fetched by the tag set presenter for the sidebar.

This avoids fetching the tag categories from memcache again (via
Post#typed_tags) when we're already fetched the tags once before.

This also means it's no longer necessary to cache humanized_essential_tag_string
itself in memcache, since it can be generated as quickly as the sidebar taglist.
2018-09-30 21:52:24 -05:00
evazion
2cc4e35cc9 Fix #3930: Can't remove children from a parent post through child: metatag.
Add `child:none` and `-child:123` edit metatags. Allow using ranges with
these metatags (e.g. `-child:1..10`, `child:1,3,5`).
2018-09-29 20:42:38 -05:00
evazion
03abbd0683 Fix #2894: Use [[:space:]] instead of \s in regexes. 2018-09-20 19:24:38 -05:00
Albert Yi
ed5b2ad443 throw error on counts/posts endpoint during a timeout (#2170) 2018-09-19 13:34:50 -07:00
evazion
a0ebd90409 Post#normalized_source: add hitomi.la url normalization. 2018-09-10 20:22:10 -05:00
evazion
6fc2d17886 Post#normalized_source: normalize da urls to new format (#3771) 2018-09-02 11:19:15 -05:00
evazion
4f5cb0ba70 Post#has_artist_tag: fix "artist tag is required" triggering incorrectly.
Fixes the "Artist tag is required" upload warning to only trigger when
the source is from a first-party site.
2018-08-28 18:14:21 -05:00
Albert Yi
68c30961ac Merge pull request #3805 from r888888888/refactor-sources
Refactor sources
2018-08-28 12:13:15 -07:00
evazion
0bb787d987 Post.pending_or_flagged: fix ambiguous column reference.
Fixes an ambiguous column reference error when searching for
`status:unmoderated` on the comments page:

    https://danbooru.donmai.us/comments?group_by=comment&search[post_tags_match]=status:unmoderated

Also removes the `id in (...)` subquery checking whether flagged posts
have a flag from the last week; this is always true.
2018-08-25 11:26:00 -05:00
evazion
e33d6f9e84 Post#is_approvable?: fix 'Approve' link appearing on approver's own uploads.
Fixes the 'Approve' link in the modqueue and in the quickmod bar being
shown as available on the approver's own flagged uploads, even though
they can't actually approve these posts.
2018-08-25 11:12:37 -05:00
Albert Yi
762dc3da24 Refactor sources 2018-08-24 12:10:51 -07:00
evazion
4f02c7f70a search: add status:modqueue, status:unmoderated metatags.
* status:modqueue = ~status:pending ~status:flagged
* status:unmoderated = status:modqueue -user:self -approver:self -disapproval:any
2018-08-23 14:48:39 -05:00
evazion
03a97d2962 Post#normalized_source: add toranoana.jp normalization. 2018-08-21 00:07:58 -05:00
evazion
33506839c4 Post#normalized_source: fix gelbooru source normalization.
Didn't handle https or md5-based subdirs:

    https://gelbooru.com//images/57/a3/57a3676a63e5cf4ab05463a3a9a953bb.jpg
        ^                        ^^^^^
2018-08-20 15:02:08 -05:00
evazion
c504ad555d Fix #3822: Move favorites (while deleting) fails if user has also favorited destination 2018-08-19 00:54:17 -05:00
evazion
3419193341 Post#normalized_source: fix twipple.jp normalization. 2018-08-19 00:33:31 -05:00
evazion
03cf4c917e Post#normalized_source: prefer https links where supported. 2018-08-19 00:31:07 -05:00
evazion
4eead40f4d Fix #3823: Add Photozou source normalization. 2018-08-18 23:23:48 -05:00
Albert Yi
9082ddf455 potential fix for #3783 2018-07-26 18:10:27 -07:00
r888888888
cc7a66438f expose recommended posts to everyone 2018-07-23 15:05:43 -07:00
Albert Yi
ea210bdc75 Add matching storage manager 2018-07-12 17:15:41 -07:00
Albert Yi
64446d49e1 add image cropping support 2018-06-22 14:41:57 -07:00
Albert Yi
e551ff9b0c fix tests 2018-06-20 11:11:46 -07:00