Make it so that when a user removes their own vote, the vote is soft
deleted (the is_deleted flag is set) instead of hard deleted.
Changes:
* Add is_deleted flag to comment votes.
* Relax uniqueness constraint so you can have multiple deleted votes on
the same comment. You can still only have one active vote on the comment.
* Add `soft_delete` method to Deletable concern.
Changes:
* Change the `expires_at` field to `duration`.
* Make moderators choose from a fixed set of standard ban lengths,
instead of allowing arbitrary ban lengths.
* List `duration` in seconds in the /bans.json API.
* Dump bans to BigQuery.
Note that some old bans have a negative duration. This is because their
expiration date was before their creation date, which is because in 2013
bans were migrated to Danbooru 2 and the original ban creation dates
were lost.
Optimize searches for non-English phrases in autocomplete. These
searches were pretty slow, and could sometimes cause sitewide lag spikes
when users typed long strings of non-English text into the search box
and caused an unintentional DoS.
The trick is to use an `array_to_tsvector(other_names) USING gin` index
on other_names. This supports fast string prefix matching against all
elements of the array. The downside is that it doesn't allow infix or
suffix matches, so we can't support wildcards in general. Wildcards
didn't quite work anyway, since artist and wiki other names can contain
literal '*' characters.
Fix the `normalize` and `array_attribute` macros conflicting with each
other on the WikiPage model. This meant code like
`wiki_page.other_names = "foo bar"` didn't work. Both macros defined a
`other_names=` method, but one method overrode the other.
The fix is to use anonymous modules and prepend so we can chain method
calls with super.
Add tracking of certain important user actions. These events include:
* Logins
* Logouts
* Failed login attempts
* Account creations
* Account deletions
* Password reset requests
* Password changes
* Email address changes
This is similar to the mod actions log, except for account activity
related to a single user.
The information tracked includes the user, the event type (login,
logout, etc), the timestamp, the user's IP address, IP geolocation
information, the user's browser user agent, and the user's session ID
from their session cookie. This information is visible to mods only.
This is done with three models. The UserEvent model tracks the event
type (login, logout, password change, etc) and the user. The UserEvent
is tied to a UserSession, which contains the user's IP address and
browser metadata. Finally, the IpGeolocation model contains the
geolocation information for IPs, including the city, country, ISP, and
whether the IP is a proxy.
This tracking will be used for a few purposes:
* Letting users view their account history, to detect things like logins
from unrecognized IPs, failed logins attempts, password changes, etc.
* Rate limiting failed login attempts.
* Detecting sockpuppet accounts using their login history.
* Detecting unauthorized account sharing.
* Introduce an abstraction for normalizing attributes. Very loosely
modeled after https://github.com/fnando/normalize_attributes.
* Normalize wiki bodies to Unicode NFC form.
* Normalize Unicode space characters in wiki bodies (strip zero width
spaces, normalize line endings to CRLF, normalize Unicode spaces to
ASCII spaces).
* Trim spaces from the start and end of wiki page bodies. This may cause
wiki page diffs to show spaces being removed even when the user didn't
explicitly remove the spaces themselves.
Expand the tag abbreviation system introduced in b0be8ae45 so that it
works in searches and when tagging posts, not just in autocomplete.
For example, you can tag a post with /evth and it will add the tag
eyebrows_visible_through_hair. You can search for /evth and it will
search for the tag eyebrows_visible_through_hair.
Some more examples:
* /ops is short for one-piece_swimsuit
* /hooe is short for hair_over_one_eye
* /saol is short for standing_on_one_leg
* /tlozbotw is short for the_legend_of_zelda:_breath_of_the_wild
If two tags have the same abbreviation, then the larger tag takes
precedence. For example, /be is short for blue_eyes, not brown_eyes,
because blue_eyes is the bigger tag.
If there is an existing shortcut alias that conflicts with the
abbreviation, then the alias take precedence. For example, /sh is short
for suzumiya_haruhi, not short_hair, because there's an old alias for
/sh -> suzumiya_haruhi.
Add the following search operators:
* /tags?search[post_count_eq]=42
* /tags?search[post_count_not_eq]=42
* /tags?search[post_count_gt]=42
* /tags?search[post_count_gteq]=42
* /tags?search[post_count_lt]=42
* /tags?search[post_count_lteq]=42
Works for all numeric attributes on all index actions.
* Rename the `#negate` and `#and` methods that we monkey patch into
ActiveRecord::Relation. These methods are now defined in Rails 6.1, but
they shadow our methods and have slightly different behavior.
* Fix a call to `invert`. It no longer accepts an argument.
* Factor out the code for moving tags from tag aliases to a separate
TagMover class.
* When aliasing two tags that have conflicting wikis, merge the old wiki
into the new one instead of failing with an error. Merge the other names
fields, replace the old wiki body with a message linking to the new
wiki, and mark the old wiki as deleted.
* When aliasing two tags that have conflicting artist entries, merge the
old artist into the new one instead of silently ignore the conflict.
Merge the group name, other names, and urls fields, and mark the old
artist as deleted.
* When two tags have conflicting wikis or artist entries, but the old
wiki or artist entry is deleted, then just ignore the old wiki or
artist and don't try to merge it.
* Fix it so that when saved searches are rewritten, we rewrite negated
searches too.
Allow searching enum fields by string, by id, or by array of
comma-separated values. The category field in modactions is an example
of an enum field that can be searched this way.
- A generalized search includes function was added
-- The post and user includes functions were changed to use that
- A search function for polymorphic includes was added
- All models are given 3 class functions to control which includes
are searchable, and extra restrictions for the "has_" params
When doing a tag search, we have to be careful about which user we're
running the search as because the results depend on the current user.
Specifically, things like private favorites, private favorite groups,
post votes, saved searches, and flagger names depend on the user's
permissions, and whether non-safe or deleted posts are filtered out
depend on whether the user has safe mode on or the hide deleted posts
setting enabled.
* Refactor internal searches to explicitly state whether they're
running as the system user (DanbooruBot) or as the current user.
* Explicitly pass in the current user to PostQueryBuilder instead of
implicitly relying on the CurrentUser global.
* Get rid of CurrentUser.admin_mode? (used to ignore the hide deleted
post setting) and CurrentUser.without_safe_mode (used to ignore safe
mode).
* Change the /counts/posts.json endpoint to ignore safe mode and the
hide deleted posts settings when counting posts.
* Fix searches not correctly overriding the hide deleted posts setting
when multiple status: metatags were used (e.g. `status:banned status:active`)
* Fix fast_count not respecting the hide deleted posts setting when the
status:banned metatag was used.
Bug: in some cases searching for multiple metatags would cause one
metatag to be ignored. For example, a search for {{user:1 pool:2}} would
be treated as a search for {{pool:2}}.
Cause: we used `ActiveRecord::Relation#merge` to combine two relations,
which was wrong because `merge` doesn't combine `column IN (?)` clauses
correctly. If there are two `column IN (?)` clauses on the same column,
then `#merge` takes only the second clause and ignores the first.
Fix: write our own half-baked `#and` method to work around Rails'
broken-by-design `#merge` method.
ref: https://github.com/rails/rails/issues/33501.
* Fix not being able to use the status: metatag twice in the same search.
* Fix status:active excluding banned posts.
* Fix status:garbage returning all posts.
Support using the same numeric-valued metatag twice in the same search.
Numeric-valued metatags are those taking an integer, float, filesize, or
date argument. Previously using the same metatag twice would cause the
second metatag to overwrite the first metatag.
Examples:
* "id:>5 id:<10"
* "width:>500 width:<1000"
* "date:>2019-01-01 date:<2020-01-01"
Partial fix for #4389.
* Fix invalid username searches returning all posts instead of no posts.
* Fix "user:A user:B" returning results for user:B instead of no results.
* Fix "approver:A approver:B" returning results for approver:B instead of no results.
* Add support for negated -commenter, -noter, -noteupdater, -upvote, -downvote metatags.
* Add support for "any" and "none" values for all username metatags,
including negated metatags that didn't support "any" or "none" before.
* Change noter:any and commenter:any to include posts with deleted notes
or comments. Note that commenter:<username> already included deleted
comments before. This is so that commenter:any has the same behavior
as commenter:<username>
