nijie: fix parallel test failures.
Nijie tests fail often under parallel testing. This is because every test needs to login to Nijie first, but Nijie rate-limits the login endpoint, so eventually we hit the limit and tests start failing. This is made worse by a thundering herd problem. Eight test processes try to login to Nijie at the same time, but only one succeeds, so the rest sleep and try again, but they all wakeup and try again at the same time, hitting the rate limits again. The workaround is to set the retry limit ridiculously high, higher than we would ideally like in production. Another workaround would be to serialize the Nijie tests in the test suite. This can be done with lockfiles and flock(2). This helps, but we can still hit the rate limit even under serialized execution.
This commit is contained in:
@@ -182,7 +182,7 @@ module Sources
|
||||
form = { email: Danbooru.config.nijie_login, password: Danbooru.config.nijie_password }
|
||||
|
||||
# XXX `retriable` must come after `cache` so that retries don't return cached error responses.
|
||||
response = http.cache(1.hour).use(:retriable).post("https://nijie.info/login_int.php", form: form)
|
||||
response = http.cache(1.hour).use(retriable: { max_retries: 20 }).post("https://nijie.info/login_int.php", form: form)
|
||||
DanbooruLogger.info "Nijie login failed (#{url}, #{response.status})" if response.status != 200
|
||||
return nil unless response.status == 200
|
||||
|
||||
|
||||
Reference in New Issue
Block a user