Fix three exploits that allowed one to keep using their account after it was deleted:
* It was possible to use session cookies from another computer to login after you deleted your account.
* It was possible to use API keys to make API requests after you deleted your account.
* It was possible to request a password reset, delete your account, then use the password reset link
to change your password and login to your deleted account.
* Don't delete the user's favorites unless private favorites are enabled. The general rule is that
public account activity is kept and private account activity is deleted.
* Delete the user's API keys, forum topics visits, private favgroups, downvotes, and upvotes (if
privacy is enabled).
* Reset all of the user's account settings to default. This means custom CSS is deleted, where it
wasn't before.
* Delete everything but the user's name and password asynchronously.
* Don't log the current user out if it's the owner deleting another user's account.
* Fix#5067 (Mod actions sometimes not created for user deletions) by wrapping the deletion process
in a transaction.
In the past it was possible for users to create multiple accounts with
the same email address. We had about 9000 such accounts. This removes
the email address from these accounts.
When multiple accounts have the same email address, the account that
visited the site last gets to keep the address.
Log the following information in email headers:
* X-Danbooru-User: the user's name and ID.
* X-Danbooru-IP: the user's IP.
* X-Danbooru-Session: the users' session ID.
* X-Danbooru-URL: the page that triggered the email.
* X-Danbooru-Job-Id: the ID of the background job that sent the email.
* X-Danbooru-Enqueued-At: when the email was queued as a background job.
* X-Danbooru-Dmail: for Dmail notifications, the link to the Dmail.
* X-Request-Id: the request ID of the HTTP request that triggered the email.
Also make it so we log an event in the APM when we send an email.
Add a polymorphic `subject` field that records the subject of the mod
action. The subject is the post, user, comment, artist, etc the mod
action is for.
* The subject for the user ban and unban actions is the user, not the ban itself.
* The subject for the user feedback update and deletion actions is the user,
not the feedback itself.
* The subject for the post undeletion action is the post, not the approval itself.
* The subject for the move favorites action is the source post where the
favorites were moved from, not the destination post where the favorites
were moved to.
* The subject for the post permanent delete action is nil, because the
post itself is hard deleted.
* When a post is permanently deleted, all mod actions related to the
post are deleted as well.
* Add a global /post_events page that shows the history of all approvals,
disapprovals, flags, appeals, and replacements on a single page.
* Redesign the /posts/:id/events page to show all approval, disapproval,
flag, appeal, and replacement events for a single post (before it only
showed approvals, flags, and appeals).
* Remove the replacement history link from the post show page. Replacements
are now included in the post events page (closes#4948: Highlighed replacements).
* Add /post_approvals/:id and /post_replacements/:id routes (these are
used by the "Details" link on the post events page).
Make the following fields visible in API responses:
* ip_bans.ip_addr
* ip_geolocations.ip_addr
* ip_geolocations.network
* users.last_ip_addr (mod only)
* user_sessions.ip_addr
* api_keys.last_ip_address
* api_keys.permitted_ip_addresses
Before IP addresses were globally hidden in API responses because IPs were
present in a lot of tables and we didn't want to accidentally leak them.
Now that we've gotten rid of IPs from most tables, it's safe to unhide them.
* Fix it so non-moderators can't search deleted comments using the
`updater`, `body`, `score`, `do_not_bump_post`, or `is_sticky` fields.
Searching for these fields will exclude deleted comments.
* Fix it so non-moderators can search for their own deleted comments using the
`creator` field, but not for deleted comments belonging to other users.
* Fix it so that if a regular user searches `commenter:<username>`, they
can only see posts with undeleted comments by that user. If a moderator or
the commenter themselves searches `commenter:<username>`, they can see all
posts the user has commented on, including posts with deleted comments.
* Fix it so the comment count on user profiles only counts visible
comments. Regular users can only see the number of undeleted comments
a user has, while moderators and the commenter themselves can see the
total number of comments.
Known issue:
* It's still possible to order deleted comments by score, which can let
you infer the score of deleted comments.
Rationale:
* The spoilers tag is the most frequently removed tag from the default blacklist.
* It's frustrating for regular users to have posts randomly hidden because of trivial
spoilers from a series they don't care about.
* The spoilers tag is used way too liberally for things that aren't considered
spoilers on other sites.
* If you're looking up fanart on the internet, you should expect to see a certain
level of spoilers.
* The tag is used very inconsistently, with some characters like Nia_(blade)_(xenoblade)
getting the spoilers tag half the time and the rest of the time not.
Add stricter username rules:
* Only allow usernames to contain basic letters, numbers, CJK characters, underscores, dashes and periods.
* Don't allow names to start or end with punctuation.
* Don't allow names to have multiple underscores in a row.
* Don't allow active users to have names that look like deleted users (e.g. "user_1234").
* Don't allow emoji or any other Unicode characters except for Chinese, Japanese, and Korean
characters. CJK characters are currently grandfathered in but will be disallowed in the future.
Users with an invalid name will be shown a permanent sitewide banner until they change their name.
Add a limit so that users can't upload more if they already have more
than 250 images queued for upload.
For example, if you upload a Pixiv post that has 200 images, then you'll
have 200 queued images for upload. This will go down as the images are
processed. If you exceed the limit, then trying to create new uploads
will return an error.
This is to prevent single users from overwhelming the site by uploading
too many images at once, thereby preventing other users from uploading
because the job queue is backed up and can't process new uploads by
other users until existing uploads are finished.
`string.mb_chars.downcase` was used to correctly downcase Unicode
characters when downcasing strings in Ruby <2.4. This hasn't been needed
since Ruby 2.4.
Fix mailers to not attempt deliveries to invalid or nonexistent email
addresses. This usually happened when someone changed their email, and
we tried to send a confirmation email to a nonexistent address.
Make private favorites and upvotes a Gold-only account option.
Existing Members with private favorites enabled are allowed to keep it
enabled, as long as they don't disable it. If they disable it, then they
can't re-enable it again without upgrading to Gold first.
This is a Gold-only option to prevent uploaders from creating multiple
accounts to upvote their own posts. If private upvotes were allowed for
Members, then it would be too easy to use fake accounts and private
upvotes to upvote your own posts.
Make upvotes public the same way favorites are public:
* Rename the "Private favorites" account setting to "Private favorites and upvotes".
* Make upvotes public, unless the user has private upvotes enabled. Note
that private upvotes are still visible to admins. Downvotes are still
hidden to everyone except for admins.
* Make https://danbooru.donmai.us/post_votes visible to all users. This
page shows all public upvotes. Private upvotes and downvotes are only
visible on the page to admins and to the voter themselves.
* Make votes searchable with the `upvote:username` and `downvote:username`
metatags. These already existed before, but they were only usable by
admins and by people searching for their own votes.
Upvotes are public to discourage users from upvoting with multiple
accounts. Upvote abuse is obvious to everyone when upvotes are public.
The other reason is to make upvotes consistent with favorites, which are
already public.
Merge the 100 favorite subtables into a single table.
Previously the favorites table was partitioned by user id into 100
subtables to try to make searching by user id faster. This wasn't really
necessary and probably slower than just making an index on
(favorites.user_id, favorites.id) to satisfy ordfav searches. BTree
indexes are logarithmic so dividing an index by 100 doesn't make it 100
times faster to search; instead it just removes a layer or two from the
tree.
This also adds a uniqueness index on (user_id, post_id) to prevent
duplicate favorites. Previously we had to check for duplicates at the
application layer, which required careful locking to do it correctly.
Finally, this adds an index on favorites.id, which was surprisingly
missing before. This made ordering and deleting favorites by id really
slow because it degraded to a sequential scan.
Let all users have unlimited favorites. Formerly the limit was 10k
favorites for regular members, 20k for Gold, and unlimited for Platinum.
Limiting favorites doesn't make sense since upvotes are unlimited.
When a user tries to change their email, redirect them to the confirm
password page (like Github's sudo mode) instead of having them re-enter
their password on the change email page. This is the same thing we do
when a user updates their API keys. This way we have can use the same
confirm password authentication flow for everything that needs a
password.
* Tie rate limits to both the user's ID and their IP address.
* Make each endpoint have separate rate limits. This means that, for
example, your post edit rate limit is separate from your post vote
rate limit. Before all write actions had a shared rate limit.
* Make all write endpoints have rate limits. Before some endpoints, such
as voting, favoriting, commenting, or forum posting, weren't subject
to rate limits.
* Add stricter rate limits for some endpoints:
** 1 per 5 minutes for creating new accounts.
** 1 per minute for login attempts, changing your email address, or
for creating mod reports.
** 1 per minute for sending dmails, creating comments, creating forum
posts, or creating forum topics.
** 1 per second for voting, favoriting, or disapproving posts.
** These rate limits all have burst factors high enough that they
shouldn't affect normal, non-automated users.
* Raise the default write rate limit for Gold users from 2 per second to
4 per second, for all other actions not listed above.
* Raise the default burst factor to 200 for all other actions not listed
above. Before it was 10 for Members, 30 for Gold, and 60 for Platinum.
Rework the rate limit implementation to make it more flexible:
* Allow setting different rate limits for different actions. Before we
had a single rate limit for all write actions. Now different
controller endpoints can have different limits.
* Allow actions to be rate limited by user ID, by IP address, or both.
Before actions were only limited by user ID, which meant non-logged-in
actions like creating new accounts or attempting to login couldn't be rate
limited. Also, because actions were limited by user ID only, you could
use multiple accounts with the same IP to get around limits.
Other changes:
* Remove the API Limit field from user profile pages.
* Remove the `remaining_api_limit` field from the `/profile.json` endpoint.
* Rename the `X-Api-Limit` header to `X-Rate-Limit` and change it from a
number to a JSON object containing all the rate limit info
(including the refill rate, the burst factor, the cost of the call,
and the current limits).
* Fix a potential race condition where, if you flooded requests fast
enough, you could exceed the rate limit. This was because we checked
and updated the rate limit in two separate steps, which was racy;
simultaneous requests could pass the check before the update happened.
The new code uses some tricky SQL to check and update multiple limits
in a single statement.
