Allow moderators to forcibly change the username of other users. This is
so mods can change abusive or invalid usernames.
* A mod can only change the username of Builder-level users and below.
* The user can't change their own name again until one week has passed.
* A modaction is logged when a mod changes a user's name.
* A dmail is sent to the user notifying them of the change.
* The dmail does not send the user an email notification. This is so we
don't spam users if their name is changed after they're banned, or if
they haven't visited the site in a long time.
The rename button is on the user's profile page, and when you hover over
the user's name and open the "..." menu.
Don't allow GET requests to pass the request params in the body instead
of in the URL. While Rails can handle GET params passed in the body, it
goes against spec and it may cause problems if the response is a redirect
and the client doesn't send the body params when following the redirect.
This may be a breaking change for broken API clients who were sending
GET params in the body instead of in the URL. This can happen when people
use HTTP libraries incorrectly.
Remove the `CurrentUser.ip_addr` global variable and replace it with
`request.remote_ip`. Before we had to track the current user's IP in a
global variable so that when we edited a post for example, we could pass
down the user's IP to the model and save it in the post_versions table.
Now that we now longer save IPs in version tables, we don't need a global
variable to get access to the current user's IP outside of controllers.
Add a system for upgrading accounts using upgrade codes. Users purchase
an upgrade code off-site then redeem it on-site to upgrade their account
to Gold. Upgrade codes are randomly pre-generated and are one time use
only. Codes have enough randomness that guessing a code is infeasible.
Show a "This page has been removed because of a takedown request" error when
an unauthorized user searches for a banned tag, or tries to view a banned post.
Raise an error if the search is invalid for one of the following reasons:
* It contains multiple conflicting order: metatags (e.g. `order:score order:favcount` or `ordfav:a ordfav:b`).
* It contains a metatag that can't be used more than once: (e.g. `limit:5 limit:10`, `random:5 random:10`).
* It contains a metatag that can't be negated (e.g. `-order:score`, `-limit:20`, or `-random:20`).
* It contains a metatag that can't be used in an OR clause (e.g. ` touhou or order:score`, `touhou or limit:20`, `touhou or random:20`).
Forcibly redirect users to the name change page if their name is
invalid. This means user with invalid names can't do anything or view
any pages until they change their name. API requests are still allowed.
Fix a potential exploit where private information could be leaked if
it was contained in the error message of an unexpected exception.
For example, NoMethodError contains a raw dump of the object in the
error message, which could leak private user data if you could force a
User object to raise a NoMethodError.
Fix the error page to only show known-safe error messages from expected
exceptions, not unknown error messages from unexpected exceptions.
API changes:
* JSON errors now have a `message` param. The message will be blank for unknown exceptions.
* XML errors have a new format. This is a breaking change. They now look like this:
<result>
<success type="boolean">false</success>
<error>PaginationExtension::PaginationError</error>
<message>You cannot go beyond page 5000.</message>
<backtrace type="array">
<backtrace>app/logical/pagination_extension.rb:54:in `paginate'</backtrace>
<backtrace>app/models/application_record.rb:17:in `paginate'</backtrace>
<backtrace>app/logical/post_query_builder.rb:529:in `paginated_posts'</backtrace>
<backtrace>app/logical/post_sets/post.rb:95:in `posts'</backtrace>
<backtrace>app/controllers/posts_controller.rb:22:in `index'</backtrace>
</backtrace>
</result>
instead of like this:
<result success="false">You cannot go beyond page 5000.</result>
Fix an open redirect exploit where if you went to <https://danbooru.donmai.us/login?url=//fakebooru.com>,
then after you logged in you would be redirected to https://fakebooru.com.
This was actually fixed by the upgrade to Rails 7.0. `redirect_to` now
raises an `UnsafeRedirectError` on redirect to an offsite URL. Before we
tried to prevent offsite redirects by checking that the URL started with
a slash, but this was insufficient - it allowed protocol-relative URLs
like `//fakebooru.com`.
Add a test case for protocol-relative URLs and return a 403 error on an
offsite redirect.
Refactor controllers so that endpoint rate limits are declared locally,
with the endpoint, instead of globally, in a single method in ApplicationController.
This way an endpoint's rate limit is declared in the same file as the
endpoint itself.
This is so we can add fine-grained rate limits for certain GET requests.
Before rate limits were only for non-GET requests.
Unlike Unicorn, Puma doesn't have a builtin HTTP request timeout
mechanism, so we have to use Rack::Timeout instead.
See the caveats in the Rack::Timeout documentation [1]. In Unicorn, a
timeout would send a SIGKILL to the worker, immediately killing it. This
would result in a dropped connection and a Cloudflare 502 error to the
user. In Puma, it raises an exception, which we can catch and return a
better error to the user. On the other hand, raising an exception can
potentially corrupt application state if it's sent at the wrong time, or
be delayed indefinitely if the app is stuck in IO or C extension code.
The default request timeout is 65 seconds. 65 seconds is to give things
like HTTP requests on a 60 second timeout enough time to complete. Set
the RACK_REQUEST_TIMEOUT environment variable to change the timeout.
1: https://github.com/sharpstone/rack-timeout#further-documentation
Generate image URLs relative to the site's canonical URL instead of
relative to the domain of the current request.
This means that all subdomains of Danbooru - safebooru.donmai.us,
shima.donmai.us, saitou.donmai.us, and kagamihara.donmai.us - will use
image URLs from https://danbooru.donmai.us, instead of from the current
domain.
The main reason we did this before was so that we could generate either
http:// or https:// image URLs, depending on whether the current request
was HTTP or HTTPS, back when we tried to support both at the same time.
Now we support only HTTPS in production, so there's no need for this. It
was also pretty hacky, since it required storing the URL of the current
request in a per-request global variable in `CurrentUser`.
This also improves caching slightly, since users of safebooru.donmai.us
will receive cached images from danbooru.donmai.us.
Downstream boorus should make sure that the `canonical_url` and
`storage_manager` config options are set correctly. If you don't support
https:// in development, you should make sure to set the canonical_url
option to http:// instead of https://.
* Tie rate limits to both the user's ID and their IP address.
* Make each endpoint have separate rate limits. This means that, for
example, your post edit rate limit is separate from your post vote
rate limit. Before all write actions had a shared rate limit.
* Make all write endpoints have rate limits. Before some endpoints, such
as voting, favoriting, commenting, or forum posting, weren't subject
to rate limits.
* Add stricter rate limits for some endpoints:
** 1 per 5 minutes for creating new accounts.
** 1 per minute for login attempts, changing your email address, or
for creating mod reports.
** 1 per minute for sending dmails, creating comments, creating forum
posts, or creating forum topics.
** 1 per second for voting, favoriting, or disapproving posts.
** These rate limits all have burst factors high enough that they
shouldn't affect normal, non-automated users.
* Raise the default write rate limit for Gold users from 2 per second to
4 per second, for all other actions not listed above.
* Raise the default burst factor to 200 for all other actions not listed
above. Before it was 10 for Members, 30 for Gold, and 60 for Platinum.
Rework the rate limit implementation to make it more flexible:
* Allow setting different rate limits for different actions. Before we
had a single rate limit for all write actions. Now different
controller endpoints can have different limits.
* Allow actions to be rate limited by user ID, by IP address, or both.
Before actions were only limited by user ID, which meant non-logged-in
actions like creating new accounts or attempting to login couldn't be rate
limited. Also, because actions were limited by user ID only, you could
use multiple accounts with the same IP to get around limits.
Other changes:
* Remove the API Limit field from user profile pages.
* Remove the `remaining_api_limit` field from the `/profile.json` endpoint.
* Rename the `X-Api-Limit` header to `X-Rate-Limit` and change it from a
number to a JSON object containing all the rate limit info
(including the refill rate, the burst factor, the cost of the call,
and the current limits).
* Fix a potential race condition where, if you flooded requests fast
enough, you could exceed the rate limit. This was because we checked
and updated the rate limit in two separate steps, which was racy;
simultaneous requests could pass the check before the update happened.
The new code uses some tricky SQL to check and update multiple limits
in a single statement.
Require the user to re-enter their password before they can view,
create, update, or delete their API keys.
This works by tracking the timestamp of the user's last password
re-entry in a `last_authenticated_at` session cookie, and redirecting
the user to a password confirmation page if they haven't re-entered
their password in the last hour.
This is modeled after Github's Sudo mode.
6d867de20 caused an exception in the ApiKeysController, which calls
respond_with with two arguments: `respond_with(CurrentUser.user, @api_key)`.
`options[0]` referred to the second argument, which was incorrect.
The /emails endpoint was passing in the "Email" model because that's
how the emails controller classifies. This was to fix that, and to
allow any other such cases in the future.
This refactors Pundit policies to only rely on the current user, not on
the current user and the current HTTP request. In retrospect, it was a
bad idea to include the current request in the Pundit context. It bleeds
out everywhere and there are many contexts (in tests and models) where
we only have the current user, not the current request. The previous
commit got rid of the only two places where we used it.
Refactor page limits to a) be explicitly listed in the User class (not
hidden away in the Danbooru config) and b) explicitly depend on the
CurrentUser (not implicitly by way of Danbooru.config.max_numbered_pages).
* Fix a bug where non-GET 404 requests weren't handled.
* Fix a bug where non-HTML 404 requests weren't handled.
* Show a random image from a specified pool on the 404 page.
Add a debug mode option. This is useful when debugging failed tests.
Debug mode disables parallel testing so you can set breakpoints in tests
with binding.pry (normally parallel testing makes it hard to set
breakpoints).
Debug mode also disables global exception handling for controllers. This
lets exceptions bubble up to the console during controller tests
(normally exceptions are swallowed by the controller, which prevents you
from seeing backtraces in failed controller tests).
Fix session cookies being sent in publicly cached /autocomplete.json
responses. We can't set any cookies in a response that is being publicly
cached, otherwise they'll be visible to other users. If a user's session
cookies were to be cached, then it would allow their account to be stolen.
In reality, well-behaved caches like Cloudflare will simply refuse to
cache responses that contain cookies to avoid this scenario.
https://support.cloudflare.com/hc/en-us/articles/200172516-Understanding-Cloudflare-s-CDN:
BYPASS is returned when enabling Origin Cache-Control. Cloudflare also
sets BYPASS when your origin web server sends cookies in the response
header.
Bug: when a search timed out we got the generic failbooru page instead
of the search timeout error page.
Cause: when rendering the <link rel="next"> / <link rel="prev"> tags in
the header, we may need to evaluate the search to determine the next or
previous page, but if the searches times out then this fails, which
caused Rails to throw a ActionView::Template::Error because an exception
was thrown while rendering the template.
Likewise, rendering the attributes for the <body> tag could fail with an
ActionView::Template::Error because the call to `current_item.present?`
forced evaluation of the search.
Rename the `error` url param to `cause_error`. Using this param causes
Danbooru to return an error response for testing purposes. Calling this
param `error` caused problems when OAuth2 authorization failed and the
user was redirected back to Danbooru with the `error` param set.
* Make IP bans soft deletable.
* Add a hit counter to track how many times an IP ban has blocked someone.
* Add a last hit timestamp to track when the IP ban last blocked someone.
* Add a new type of IP ban, the signup ban. Signup bans restrict new
signups from editing anything until they've verified their email
address.
- "Current" is now most like the old format
-- It is therefore now the default for post versions
- Only show the actual edits in their own column
- Show the current state at that version in another column
- On the "previous" view, don't double-show full list of tags for
the first post versions, so leave edits blank
- The types are:
-- Previous: The default and the previously used type
-- Subsequent: Compares against the next version
-- Current: Compares against the current version
- Allow switching between comparison types in index and diff views
-- Have links vary depending upon current comparison type
Previously only actions that were marked member_only or above were
subject to IP ban restrictions. This meant that certain actions that
weren't marked member_only, like creating new accounts, could still be
done by IP banned users.
Now IP banned users can't do any non-GET actions, which means they're
not allowed to even login to their accounts.
