* Add index on posts.is_deleted. The modqueue was slow because we the
appeal condition wasn't constrained to deleted posts, so it degraded to
a full table scan.
* Avoid extra queries for calculating the page count and disapproval counts.
* Include appealed posts in the modqueue.
* Add `status` field to appeals. Appeals start out as `pending`, then
become `rejected` if the post isn't approved within three days. If the
post is approved, the appeal's status becomes `succeeded`.
* Add `status` field to flags. Flags start out as `pending` then become
`rejected` if the post is approved within three days. If the post
isn't approved, the flag's status becomes `succeeded`.
* Leave behind a "Unapproved in three days" dummy flag when an appeal
goes unapproved, just like when a pending post is unapproved.
* Only allow deleted posts to be appealed. Don't allow flagged posts to be appealed.
* Add `status:appealed` metatag. `status:appealed` is separate from `status:pending`.
* Include appealed posts in `status:modqueue`. Search `status:modqueue order:modqueue`
to view the modqueue as a normal search.
* Retroactively set old flags and appeals as succeeded or rejected. This
may not be correct for posts that were appealed or flagged multiple
times. This is difficult to set correctly because we don't have
approval records for old posts, so we can't tell the actual outcome of
old flags and appeals.
* Deprecate the `is_resolved` field on post flags. A resolved flag is a
flag that isn't pending.
* Known bug: appealed posts have a black border instead of a blue
border. Checking whether a post has been appealed would require either
an extra query on the posts/index page, or an is_appealed flag on
posts, neither of which are very desirable.
* Known bug: you can't use `status:appealed` in blacklists, for the same
reason as above.
`User.find_or_create_by!` calls `User.name_matches` during username
validation, which triggers a deprecation warning for some reason:
DEPRECATION WARNING: Class level methods will no longer inherit scoping
from `create!` in Rails 6.1. To continue using the scoped relation,
pass it into the block directly. To instead access the full set of
models, as Rails 6.1 will, use `User.default_scoped`.
This was a hack to deal with the Cloudflare check sometimes being slow
or timing out during tests. The call to https://api.cloudflare.com/client/v4/ips
could hang if there were IPv6 connectivity problems. If this happens, make
sure that IPv6 is configured properly and that `curl -v --http1.1 -6 https://api.cloudflare.com/client/v4/ips`
works.
Several fixes for the "This tag is under discussion" notice on the post
index page:
* Fix the notice appearing for BURs that aren't pending.
* Fix the notice never going away because of the cache never expiring.
* List all topics when a tag is involved in multiple BURs.
* Link to the forum post instead of the forum topic (fix#4421).
* Optimization: don't check for BURs when the search isn't a simple
single tag search.
* Add a `tags` field to the bulk update requests table for tracking all
tags involved in the request (excluding tags in mass updates that are
negated/optional/wildcards). Known issue: doesn't handle tag type
prefixes in mass updates correctly (e.g. `mass update foo -> artist:bar`
doesn't detect the tag `bar`).
* Allow searching the /bulk_update_requests page by tags.
We don't really need to cache the notice here, but we do it anyway to
reduce queries on the post index page.
* Make IP bans soft deletable.
* Add a hit counter to track how many times an IP ban has blocked someone.
* Add a last hit timestamp to track when the IP ban last blocked someone.
* Add a new type of IP ban, the signup ban. Signup bans restrict new
signups from editing anything until they've verified their email
address.
Also remove non-nullable and default options from migration. The column
didn't originally have these options so they shouldn't be added if the
migration is reverted.
* Move emails from users table to email_addresses table.
* Validate that addresses are formatted correctly and are unique across
users. Existing invalid emails are grandfathered in.
* Add is_verified flag (the address has been confirmed by the user).
* Add is_deliverable flag (an undeliverable address is an address that bounces).
* Normalize addresses to prevent registering multiple accounts with the
same email address (using tricks like Gmail's plus addressing).
The old password reset flow:
* User requests a password reset.
* Danbooru generates a password reset nonce.
* Danbooru emails user a password reset confirmation link.
* User follows link to password reset confirmation page.
* The link contains a nonce authenticating the user.
* User confirms password reset.
* Danbooru resets user's password to a random string.
* Danbooru emails user their new password in plaintext.
The new password reset flow:
* User requests a password reset.
* Danbooru emails user a password reset link.
* User follows link to password edit page.
* The link contains a signed_user_id param authenticating the user.
* User changes their own password.
Rename is_active to is_deleted. This is for better consistency with
other models, and to reduce confusion over what "active" means for
artists. Sometimes users think active is for whether the artist is
actively producing work.
Refactor to use a recursive CTE to calculate implied tags in SQL, rather
than storing them in a descendant_names field. This avoids the
complexity of keeping the stored field up to date. It's also more
flexible, since it allows us to find both descendant tags (tags that
imply a given tag) as well as ancestor tags (tags that are implied by a
given tag).
Remove the creator_id field from artists, notes, and pools. The
creator_id wasn't otherwise used and was inconsistent with the
artist/note/pool history in some cases, especially for old artists.
Few people used forum subscriptions (only around 100), and even fewer
people were subscribed to active threads. Most subscriptions were for
old threads that will never be bumped again. The implementation also had
a few problems:
* Unsubscribe links in emails didn't work (they unset the user's
receive_email_notifications flag, but forum subscriptions didn't
respect this flag).
* Some users had invalid email addresses, which caused notifications to
bounce. There was no mechanism for preventing bounces.
* The implementation wasn't scalable. It involved a daily linear scan
over _all_ forum subscriptions looking for any topics that had been updated.
Few people used dmail filters (~900 users in 5 years) and even fewer
used them correctly. Most people used them to try to block dmail spam,
but usually they either blocked too much (by adding common words that
are present in nearly all dmails, causing all mails to them to be
filtered) or too little (blocking specific email addresses or urls,
which usually are never seen again after the spammer is banned).
Nowadays the spam detection system does a better job of filtering spam.
- Limited to Builders+
-- Moderator+ can also use as they may be too busy ATM
- Only on users, comments, and forum posts
- Multiple reports can be generated per instance
- Primarily posts to a moderator-only topic for viewability
- Secondarily has a table for searchability
-- Viewable only by moderators
* Rename 'privacy mode' to 'private favorites'.
* Make the private favorites setting only hide favorites, not favgroups
and not the user's uploads on their profile page.
* Make the favgroup is_public flag default to true instead of false and
fix existing favgroups to be public if the user didn't have privacy mode
enabled before.
* List _all_ public favgroups on the /favorite_groups index, not just
favgroups belonging to the current user.
* Add a /users/<id>/favorite_groups endpoint.
Don't track IP addresses for post appeals, post flags, tag aliases, tag
implications, or user feedbacks. These things are already tightly
limited. We don't need IPs from them to detect sockpuppets.
Fix various cases where db/structure.sql either didn't match the
production db or didn't match the db structure produced when running
migrations from a fresh install.
Bug: running migrations on a fresh database failed when we got to a
migration adding a posts.uploader_ip_addr index on 2019-11-11. It failed
because the index already existed.
The index already existed because we used to have it at one point, until
it was removed by another migration on 2015-01-20. This migration didn't
correctly remove the index though, because it tried to remove a
posts.source index first, which failed because this index didn't exist
(it probably existed only in production). This error was swallowed,
causing the migration to silently skip removing the posts.uploader_ip_addr
index, which eventually blew up when we tried to add the index again
later on.
Add a new IP address search page at /ip_addresses. Replaces the old
search page at /moderator/ip_addrs.
On user profile pages, show the user's last known IP to mods. Also add
search links for finding other IPs or accounts associated with the user.
IP address search uses a big UNION ALL statement to merge IP addresses
across various tables into a single view. This makes searching easier,
but is known to timeout in certain cases.
Fixes#4207 (the new IP search page supports searching by subnet).
Drop the creator_id and updater_id fields from wiki pages. These fields
had several issues:
* The creator_id field was inconsistent with the wiki_page_versions
table. Apparently during the migration to Danbooru 2 in 2012-2013 the
creator_id field got reset to whoever last updated the wiki at that
point in time.
* Saving a wiki would set the updater_id even when nothing actually
changed. This also caused the updated_at timestamp to get bumped.
Because of this, anything that saved a wiki, including things like
creating aliases or implications, would bump the updater_id and
updated_at even though the wiki didn't actually change. This meant
these fields weren't consistent with the wiki_page_versions history.
Changes:
* Remove `creator_name` field from the /wiki_pages.json API.
* Remove creator name search option from /wiki_pages/search.
Add a dtext_links table for tracking links between wiki pages. This is
to allow for broken link detection and "what links here" searches, among
other uses.
Remove all infrastructure around approving or rejecting user name
changes. Name changes haven't been moderated for several years.
* Remove status, approver_id, change_reason, and rejection_reason fields.
* Remove approve and reject controller actions.
Fixup for 9c34d5cc3. Default to member level in the database because
setting it in `customize_new_user` clobbered user levels set by factory
bot in tests, which broke the test suite.
