jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

rate limits: rework rate limit implementation.

Browse Source 
Rework the rate limit implementation to make it more flexible:

* Allow setting different rate limits for different actions. Before we
  had a single rate limit for all write actions. Now different
  controller endpoints can have different limits.

* Allow actions to be rate limited by user ID, by IP address, or both.
  Before actions were only limited by user ID, which meant non-logged-in
  actions like creating new accounts or attempting to login couldn't be rate
  limited. Also, because actions were limited by user ID only, you could
  use multiple accounts with the same IP to get around limits.

Other changes:

* Remove the API Limit field from user profile pages.
* Remove the `remaining_api_limit` field from the `/profile.json` endpoint.
* Rename the `X-Api-Limit` header to `X-Rate-Limit` and change it from a
  number to a JSON object containing all the rate limit info
  (including the refill rate, the burst factor, the cost of the call,
  and the current limits).
* Fix a potential race condition where, if you flooded requests fast
  enough, you could exceed the rate limit. This was because we checked
  and updated the rate limit in two separate steps, which was racy;
  simultaneous requests could pass the check before the update happened.
  The new code uses some tricky SQL to check and update multiple limits
  in a single statement.
This commit is contained in:
evazion
2021-03-04 20:07:19 -06:00
parent 52adf87489
commit 4492610dfe
15 changed files with 260 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
app/controllers/application_controller.rb
View File

@@ -2,8 +2,6 @@ class ApplicationController < ActionController::Base
include Pundit
helper_method :search_params
class ApiLimitError < StandardError; end
self.responder = ApplicationResponder
skip_forgery_protection if: -> { SessionLoader.new(request).has_api_authentication? }
@@ -74,17 +72,16 @@ class ApplicationController < ActionController::Base
def api_check
return if CurrentUser.is_anonymous? || request.get? || request.head?
if CurrentUser.user.token_bucket.nil?
TokenBucket.create_default(CurrentUser.user)
CurrentUser.user.reload
end
rate_limiter = RateLimiter.new(
"write",
[CurrentUser.user.cache_key],
cost: 1,
rate: CurrentUser.user.api_regen_multiplier,
burst: CurrentUser.user.api_burst_limit
)
throttled = CurrentUser.user.token_bucket.throttled?
headers["X-Api-Limit"] = CurrentUser.user.token_bucket.token_count.to_s
if throttled
raise ApiLimitError, "too many requests"
end
headers["X-Rate-Limit"] = rate_limiter.to_json
rate_limiter.limit!
end
def rescue_exception(exception)
@@ -113,7 +110,7 @@ class ApplicationController < ActionController::Base
render_error_page(410, exception, template: "static/pagination_error", message: "You cannot go beyond page #{CurrentUser.user.page_limit}.")
when Post::SearchError
render_error_page(422, exception, template: "static/tag_limit_error", message: "You cannot search for more than #{CurrentUser.tag_query_limit} tags at a time.")
when ApiLimitError
when RateLimiter::RateLimitError
render_error_page(429, exception)
when NotImplementedError
render_error_page(501, exception, message: "This feature isn't available: #{exception.message}")