* Move the source normalization logic out of the post model
and into individual sources' strategies.
* Rewrite normalization tests to be handled into each source's test,
and expand them significantly. Previously we were only testing
a very small subset of domains and variants.
* Fix up normalization for several sites.
* Normalize fav.me urls into normal deviantart urls.
* Refactor fast_count to return nil instead of 1,000,000 if the exact count times out.
* Remove the estimate_post_counts and blank_tag_search_fast_count global config options.
* Replace the hardcoded post count estimates inside fast_count with a
method that parses Postgres's estimated row count from EXPLAIN.
* /counts/posts.json:
** Remove the `raise_on_timeout` parameter.
** Add an `estimate_count=<true|false>` parameter.
** Return null instead of 1,000,000 if the exact count times out.
Bug: If a Member had the hide_deleted_posts option turned on and did a
two tag search, no pages would show up.
Cause: The hide_deleted_posts option implicitly adds the -status:deleted
tag, but this tag wasn't considered a free metatag, so this caused
Post.fast_count to fail and return zero because the search was treated
as a three tag search.
ref: https://danbooru.donmai.us/forum_topics/16829
Bug: searching for "filetype:jpg -user:evazion" would negate the
filetype:jpg metatag too. This was because the -user:<name> metatag was
negating the entire cumulative relation instead of just the user:<name>
clause.
Add:
* commentary:true (posts with commentary)
* commentary:false (posts without commentary)
* commentary:translated (posts with translated commentary)
* commentary:untranslated (posts with untranslated commentary)
* commentary:"text" (posts where any commentary field matches "text")
Known issues:
* There's no way to escape the true, false, translated, or
untranslated keywords to do a literal text search for commentaries
containing one of these keywords.
* Negated searches may be slow. Using a left outer join instead of a
subquery would be faster in most cases, but negating it is harder.
* Fix fav:<user> searches to return no results instead of raising a
UserPrivilege error when the user has private favorites.
* Fix fav:<nonexistent_user> raising a UserPrivilege error instead of
returning no results.
* Fix -ordfav:<user> not being supported.
Change favgroup:<name> searches to return no results instead of raising
a UserPrivilege error when an unpermitted user searches for a private
favgroup.
Partial fix for #4389.
Partial fix for #4389.
* Fix invalid username searches returning all posts instead of no posts.
* Fix "user:A user:B" returning results for user:B instead of no results.
* Fix "approver:A approver:B" returning results for approver:B instead of no results.
* Add support for negated -commenter, -noter, -noteupdater, -upvote, -downvote metatags.
* Add support for "any" and "none" values for all username metatags,
including negated metatags that didn't support "any" or "none" before.
* Change noter:any and commenter:any to include posts with deleted notes
or comments. Note that commenter:<username> already included deleted
comments before. This is so that commenter:any has the same behavior
as commenter:<username>
* Allow tagging a post with a `disapproved:<disinterest|breaks_rules|poor_quality>` to disapprove it.
* Disallow disapproving active posts.
Fixes#4384.
* Move old post archive tests to post version tests.
* Fix pool tests that assumed that multiple edits by the same user
weren't merged.
* Fix references to `is_active` and `notes` on artist model.
* Fix the pool version SQS service to always be mocked before every
test. Before we had to manually set it up before every test dealing
with pool versions.
* Fix it so that we reconnect to the post/pool version databases before
every test. Before using $ARCHIVE_DATABASE_URL to set the database url
failed because environment variables weren't loaded by dotenv yet when
connections were first established.
* Support negated wildcards in searches (e.g. "holding -holding_*")
* Raise wildcard limit to matching 25 tags regardless of user level.
* Fix wildcards potentially matching empty tags.
* Fix wildcard tags being sorted by post count only, and therefore not
having a stable ordering when tags have equal post counts.
* Fix sidebar to calculate wildcards tags the same way the search does.
* Change the `disapproval:<reason>` metatag to `disapproved:<reason>`.
* Change `disapproved:<reason>` to show all posts disapproved for a
given reason, not just those disapproved by the current user.
* Allow searching for your own disapprovals with `disapproved:<my_name>`.
* Drop the `disapproved:<any|none>` metatags. `disapproved:any` is
equivalent to `disapproved:<my_name>` and `disapproved:none` is
equivalent to `-disapproved:<my_name>`.
These changes are so that you can search e.g. disapproved:poor_quality
to find all posts disapproved for poor quality.
* Allow approvers to approve a post by tagging it with status:active.
* Allow approvers to ban a post by tagging it with status:banned.
* Allow approvers to unban a post by tagging it with -status:banned.
The belongs_to_creator macro was used to initialize the creator_id field
to the CurrentUser. This made tests complicated because it meant you had
to create and set the current user every time you wanted to create an
object, when lead to the current user being set over and over again. It
also meant you had to constantly be aware of what the CurrentUser was in
many different contexts, which was often confusing. Setting creators
explicitly simplifies everything greatly.
* Rename 'privacy mode' to 'private favorites'.
* Make the private favorites setting only hide favorites, not favgroups
and not the user's uploads on their profile page.
* Make the favgroup is_public flag default to true instead of false and
fix existing favgroups to be public if the user didn't have privacy mode
enabled before.
* List _all_ public favgroups on the /favorite_groups index, not just
favgroups belonging to the current user.
* Add a /users/<id>/favorite_groups endpoint.
Stop maintaining pool category pseudo tags (pool:series, pool:collection)
in pool strings. They're no longer used and the changes to the
`Post#pools` method in dc4d2e54b caused issues with this.
Also allow Members to change the category of large pools again. This was
only restricted because maintaining these pseudotags forced us to update
every post in the pool whenever a pool's category was changed.
Stop using the pool_string field internally, but keep maintaining it
until we can drop it later.
* Stop using the pool_string for `pool:<name>` metatag searches.
* Stop using the pool_string in the `Post#pools` method. This is used to
get the list of pools on post show pages.
Revert optimization from a6163258b. Turns out that we have to resolve
aliases in fast_count, otherwise for aliased tags we'll return an empty
count.
Fixes#4156.
* Drop support for `source:pixiv/artist-name` searches. This was a hack
that only worked on old pixiv urls that haven't been used for years.
* Replace the old SourcePattern(lower(source)) index with a trigram index.
Changes:
* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
to return these fields in the api.
id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.
name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.
Avoiding caching also avoids the need to keep these caches consistent.
* Don't allow adding tags with invalid names when they already exist in
the tags table.
* If an invalid tag is added, show an warning and ignore the tag instead
of failing with a hard error.
* Move the _(cosplay) tag validation into the tag name validator.
Bug: Searching for an aliased tag returns an empty page instead of
showing the results for the real tag.
Cause: The query parsing code runs the search string through
`.mb_chars.downcase` before calling `TagAlias.to_aliased`, so the input
to `to_aliased` is actually a ActiveSupport::Multibyte::Chars object.
This breaks the `aliases[name]` hash lookup because `name` is not a
plain string.
Fixup for c7bcce429.
* Tighten up illust id parsing to avoid misparsing ids from
non-illust urls (sketch urls and novel urls).
* Move id parsing tests from post_test.rb to sources/pixiv_test.rb.
* Drop support for touch.pixiv.net urls. These urls are no longer used
by Pixiv and aren't present as the source of any posts on Danbooru.
