* Automatically fix all tags with incorrect counts during daily
maintenance (previously only tags with negative counts were fixed).
* Log fixed tags to NewRelic.
* Remove the ability to manually fix tag counts with the "Fix" button on
the /tags listing. This is no longer necessary now that tags are
fixed automatically.
* Only check for conflicts with existing aliases/implications when
requests are created or approved, not when requests are rejected.
* Use `update!(status: "deleted")` instead of `update(status: "deleted")`
so that if rejecting the request fails we fail immediately instead of
continuing on and updating the forum topic.
* Wrap `reject!` and `TagChangeRequestPruner.reject_expired` in
transactions so that if updating either the request or the forum
fails, they both get rolled back.
Replace the `method_attributes` and `hidden_attributes` methods with
`api_attributes`. `api_attributes` can be used as a class macro:
# include only the given attributes.
api_attributes :id, :created_at, :creator_name, ...
# include all default attributes plus the `creator_name` method.
api_attributes including: [:creator_name]
or as an instance method:
def api_attributes
[:id, :created_at, :creator_name, ...]
end
By default, all attributes are included except for IP addresses and
tsvector columns.
In xml responses, if the result is an empty array we want the response
to look like this:
<posts type="array"/>
not like this (the default):
<nil-classes type="array"/>
This refactors controllers so that this is done automatically instead of
having to manually call `@things.to_xml(root: "things")` everywhere. We
do this by overriding the behavior of `respond_with` in `ApplicationResponder`
to set the `root` option by default in xml responses.
Stop using the pool_string field internally, but keep maintaining it
until we can drop it later.
* Stop using the pool_string for `pool:<name>` metatag searches.
* Stop using the pool_string in the `Post#pools` method. This is used to
get the list of pools on post show pages.
* Change the source index on posts from `(lower(source) gin_trgm_ops) WHERE source != ''`
to just `(source gin_trgm_ops)`. The WHERE clause prevented the index
from being used in source:<url> searches because we didn't specify
the `source != ''` clause in the search itself. Excluding blank
sources only saved a marginal amount of space anyway. This fixes
timeouts in source:<url> searches and in the bookmarklet (since we do
a source dupe check on the upload page too).
* Also switch from indexing `lower(name)` to `name` on pools and users.
We don't need to lowercase the column because GIN indexes can be used
with both LIKE and ILIKE queries.
Make /favorites redirect to a ordfav:<user> search instead of having a
separate view just for favorites. This duplicated a lot of code for no
good reason.
Rewrite the implementation of related tags to be simpler, faster, and
more accurate:
* The related tags are now calculated by taking a random sample of 1000
posts, finding the top 250 most frequent tags among those posts, then
ordering those tags by cosine similarity.
* Related tags can generally be calculated in 50-300ms at these sample
sizes. Very high sample sizes (25000+ posts) are still relatively fast
(1-3 seconds), but generally they don't improve accuracy much.
* Related tags are now cached in redis rather than in the tags table.
The related_tags column in the tags table is no longer used.
* Only the related tags in the search taglist are cached. The related
tags returned by the 'Related tags' button are not cached.
* The cache lifetime is a fixed 4 hours.
* The 'Related tags' button now works with metatags.
* The /related_tag page now works with metatags and multitag searches.
Fixes#4134, #4146.
* Drop support for `source:pixiv/artist-name` searches. This was a hack
that only worked on old pixiv urls that haven't been used for years.
* Replace the old SourcePattern(lower(source)) index with a trigram index.
Drop support for https://danbooru.donmai.us/cache/tags.json. This was a
nightly dump of the tags table that was originally added in #1012. It
was never documented and never really used except for by the DanbooruUp
extension.
Fixes POST/PUT API requests failing with InvalidAuthenticityToken errors
due to missing CSRF tokens.
CSRF protection is only necessary for cookie-based authentication. For
non-cookie-based authentication we can safely disable it. That is, if
the user is already passing their login + api_key, then we don't need
to additionally verify the request with a CSRF token.
ref: 2e407fa476 (comments)
Setting the statement timeout at the beginning didn't work because
`PostPruner.new.prune!` clobbers the timeout (it calls `without_timeout`,
which doesn't restore the timeout properly if the timeout was zero).
Also fixes a bug where mod actions weren't logged on mass updates.
Creating the mod action silently failed because it was called when
CurrentUser wasn' set.
Changes:
* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
to return these fields in the api.
id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.
name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.
Avoiding caching also avoids the need to keep these caches consistent.
