Fixes bug described in d3e4ac7c17 (commitcomment-39049351)
When dealing with searches, there are several variables we have to keep
in mind:
* Whether tag aliases should be applied.
* Whether search terms should be sorted.
* Whether the rating:s and -status:deleted metatags should be added by
safe mode and the hide deleted posts setting.
Which of these things we need to do depends on the context:
* We want to apply aliases when actually doing the search, calculating
the count, looking up the wiki excerpt, recording missed/popular
searches in Reportbooru, and calculating related tags for the sidebar,
but not when displaying the raw search as typed by the user (for
example, in the page title or in the tag search box).
* We want to sort the search when calculating cache keys for fast_count
or related tags, and when recording missed/popular searches, but not
in the page title or when displaying the raw search.
* We want to add rating:s and -status:deleted when performing the
search, calculating the count, or recording missed/popular searches,
but not when calculating related tags for the sidebar, or when
displaying the page title or raw search.
Here we introduce normalized_query and try to use it in contexts where
query normalization is necessary. When to use the normalized query
versus the raw unnormalized query is still subtle and prone to error.
Make PostQueryBuilder apply aliases earlier, immediately after parsing
the search.
On the post index page there are multiple places where we need to apply
aliases:
* When running the search with PostQueryBuilder#build.
* When calculating the search count with PostQueryBuilder#fast_count.
* When calculating the related tags for the sidebar.
* When tracking missed searches and popular searches for Reportbooru.
* When looking up wiki excerpts.
Applying aliases after parsing ensures we only have to apply aliases
once for all of these things.
We also normalize the order of tags in searches and strip repeated tags.
This is so that we have consistent cache keys for fast_count.
* Fixes searches for aliased tags being counted as missed searches (fixes#4433).
* Fixes wiki excerpts not showing up when searching for aliased tags.
* Add unaliased:<tag> metatag. This allows you to search for a tag
without applying aliases. This is mainly useful for debugging purposes
and for searching for large tags that are in the process of being
aliased but haven't had all their posts moved yet.
* Remove the "raw" url param from the posts index page. The "raw" param
also caused the search to ignore aliases, but it was undocumented and
exploitable. It was possible to use the raw param to view private
favorites since favorites are treated like a hidden tag.
* Move various search parser helper methods (`has_metatag?`,
`is_single_tag?` et al) from PostSets and the Tag model to
PostQueryBuilder.
* Fix various minor bugs stemming from trying to check if a search query
contains certain metatags using regexes or other adhoc techniques.
Change the title of the post index page to look like this:
"Danbooru: Anime Image Board" (for the front page)
"Kantai Collection Art | Danbooru" (for a tag search)
Change the meta description of the front page to look like this:
Danbooru is the original anime image 'booru. Find over 3.75 million
anime pictures categorized by over 100 million tags.
Change the meta description for a tag search to look like this:
Find over 37,168 Azur Lane images on Danbooru. Azur Lane (碧蓝航线)
(벽람항로) is a Chinese shipgirl-themed side-scrolling shoot 'em up
mobile game developed by Shanghai Manjuu and Xiamen Yongshi...
* Add favorite count beneath recommended posts. Clicking the favcount
loads more recommended posts like that post.
* Increase number of recommendations shown on post show page.
Normally thumbnails have a fixed size of 154x154, but that's not always
desirable outside of the posts index because it creates empty gaps
around thumbnails.
* Add the source (twitter, pixiv, etc) and upload date ("X minutes ago")
to iqdb thumbnails.
* Link the filesize to the full file so you can compare files in new tabs.
* Link the similarity to a iqdb search so you can pivot your search to other posts.
Set the thumbnail height to auto instead of 154px so that there's not a
big empty gap beneath thumbnails that aren't 150px high.
This requires setting dimensions on the <img> tag itself so that the
image height is known before loading, otherwise the tooltip will be
mispositioned after the image loads in.
We set the min-width and min-height instead of the <img> width and
height properties because our calculated dimensions are sometimes
off-by-one compared to the actual dimensions. I'm not sure how libvips
calculates the thumbnail dimensions, but this avoids forcing a slightly
wrong aspect ratio, which produces ugly resizing artifacts.
Changes:
* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
to return these fields in the api.
id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.
name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.
Avoiding caching also avoids the need to keep these caches consistent.
Move Post#humanized_essential_tag_string to TagSetPresenter#humanized_essential_tag_string.
This allows humanized_essential_tag_string to reuse the same set of tags
already fetched by the tag set presenter for the sidebar.
This avoids fetching the tag categories from memcache again (via
Post#typed_tags) when we're already fetched the tags once before.
This also means it's no longer necessary to cache humanized_essential_tag_string
itself in memcache, since it can be generated as quickly as the sidebar taglist.
Move PostPresenter#categorized_tag_groups to TagSetPresenter#split_tag_list_text.
This allows split_tag_list_text to reuse the same set of tags already
fetched by the tag set presenter for the sidebar.
This avoids a memcache call to get the tag categories when rendering the
tag string for the post edit form.
Captioned post previews (previews with the pool name, similarity, or
size beneath) need to have `height: auto` set, otherwise they'll default
to `height: 154px` (just enough for the image) and the caption won't be visible.
Fail loudly if we forget to whitelist a param instead of silently
ignoring it.
misc models: convert to strong params.
artist commentaries: convert to strong params.
* Disallow changing or setting post_id to a nonexistent post.
artists: convert to strong params.
* Disallow setting `is_banned` in create/update actions. Changing it
this way instead of with the ban/unban actions would leave the artist in
a partially banned state.
bans: convert to strong params.
* Disallow changing the user_id after the ban has been created.
comments: convert to strong params.
favorite groups: convert to strong params.
news updates: convert to strong params.
post appeals: convert to strong params.
post flags: convert to strong params.
* Disallow users from setting the `is_deleted` / `is_resolved` flags.
ip bans: convert to strong params.
user feedbacks: convert to strong params.
* Disallow users from setting `disable_dmail_notification` when creating feedbacks.
* Disallow changing the user_id after the feedback has been created.
notes: convert to strong params.
wiki pages: convert to strong params.
* Also fix non-Builders being able to delete wiki pages.
saved searches: convert to strong params.
pools: convert to strong params.
* Disallow setting `post_count` or `is_deleted` in create/update actions.
janitor trials: convert to strong params.
post disapprovals: convert to strong params.
* Factor out quick-mod bar to shared partial.
* Fix quick-mod bar to use `Post#is_approvable?` to determine visibility
of Approve button.
dmail filters: convert to strong params.
password resets: convert to strong params.
user name change requests: convert to strong params.
posts: convert to strong params.
users: convert to strong params.
* Disallow setting password_hash, last_logged_in_at, last_forum_read_at,
has_mail, and dmail_filter_attributes[user_id].
* Remove initialize_default_image_size (dead code).
uploads: convert to strong params.
* Remove `initialize_status` because status already defaults to pending
in the database.
tag aliases/implications: convert to strong params.
tags: convert to strong params.
forum posts: convert to strong params.
* Disallow changing the topic_id after creating the post.
* Disallow setting is_deleted (destroy/undelete actions should be used instead).
* Remove is_sticky / is_locked (nonexistent attributes).
forum topics: convert to strong params.
* merges https://github.com/evazion/danbooru/tree/wip-rails-5.1
* lock pg gem to 0.21 (1.0.0 is incompatible with rails 5.1.4)
* switch to factorybot and change all references
Co-authored-by: r888888888 <r888888888@gmail.com>
Co-authored-by: evazion <noizave@gmail.com>
add diffs
* Remove file_path_for, cropped_file_url (dead code)
* Remove complete_preview_file_url (preview_file_url now returns absolute links)
* Remove `file_name` (only used for Download link in sidebar)
