Replace http.rb's builtin redirect following option with our own
redirect follower. This fixes an issue with http.rb losing cookies after
following a redirect.
Allow cookies to be saved and sent back when making several requests in
a row. Usage:
http = Danbooru::Http.use(:session)
# saves the foo=42 cookie sent by the response.
http.get("https://httpbin.org/cookies/set/foo/42")
# sends back the foo=42 cookie from the previous request.
http.get("https://httpbin.org/cookies")
Remove the Downloads::File class. Move download methods to
Danbooru::Http instead. This means that:
* HTTParty has been replaced with http.rb for downloading files.
* Downloading is no longer tightly coupled to source strategies. Before
Downloads::File tried to automatically look up the source and download
the full size image instead if we gave it a sample url. Now we can
do plain downloads without source strategies altering the url.
* The Cloudflare Polish check has been changed from checking for a
Cloudflare IP to checking for the CF-Polished header. Looking up the
list of Cloudflare IPs was slow and flaky during testing.
* The SSRF protection code has been factored out so it can be used for
normal http requests, not just for downloads.
* The Webmock gem can be removed, since it was only used for stubbing
out certain HTTParty requests in the download tests. The Webmock gem
is buggy and caused certain tests to fail during CI.
* The retriable gem can be removed, since we no longer autoretry failed
downloads. We assume that if a download fails once then retrying
probably won't help.
Revert back to previous workaround of fetching previous day if current
day returns no result. A terrible hack, really we should convert dates
to Reportbooru's timezone, but that has other complications.
Increase timeout to 30 seconds when uploading files to IQDB. Previously
we used the default timeout of 3 seconds, which could cause 599 timeout
errors sometimes if the upload took too long.
Fix "cannot determine size of body" errors on upload page. Caused by
exception during IQDB lookup. We were posting the form data wrong, we
need to wrap the file with HTTP::FormData::File and pass it through the
`form` parameter.
Fix the sidebar on the /posts index page sometimes being blank. This
could happen when either the related tag calculation was too slow and
timed out, or when Reporbooru was unavailable and we couldn't fetch the
list of popular tags.
In the tag list would otherwise be blank, we fall back to frequent tags
(the most common tags on the current page of results).
Also change it so that if Reportbooru is unconfigured, we fail
gracefully by returning blank results instead of failing with an
exception. This is so we can still view the popular searches and missed
searches pages during testing (even though they'll be blank).
3cdf67920 changed it so that Danbooru::Http follows redirects by
default. This broke some things in the Nico Seiga strategy, so disable
following redirects in the Nico Seiga API client for now.
Also change it so that Danbooru::Http follows redirects after a POST
request (by setting `strict: false`). Nico Seiga needs this because it
sends a redirect after we POST the login form.
* Get rid of mechanize, fully switch to Danbooru::Http
* Switch to mobile api, improving speed
* Merge main and manga clients
* Add full support for manga pages
* Add support for anonymous and r-15 images
* Don't fail when attempting to upload oekaki direct links
* Various misc fixes
* Combine MissedSearchService, PostViewCountService, and
PopularSearchService into single ReportbooruService class.
* Use Danbooru::Http for these services instead of HTTParty.
Bug: Replacing posts hosted on cdn.donmai.us didn't work.
Cause: Original files on cdn.donmai.us are hosted under /var/www/danbooru/original/, but replacements
were trying to store them directly under /var/www/danbooru, which failed with a permission error.
We were trying to store them in the wrong directory because we didn't respect the `original_subdir`
option when generating file paths.
Refactor models so that we define attribute API permissions in policy
files instead of directly in models.
This is cleaner because a) permissions are better handled by policies
and b) which attributes are visible to the API is an API-level concern
that models shouldn't have to care about.
This fixes an issue with not being able to precompile CSS/JS assets
unless the database was up and running. This was a problem when building
Docker images because we don't have a database at build time. We needed
the database because `api_attributes` was a class-level macro in some
places, which meant it ran at boot time, but this triggered a database
call because api_attributes used database introspection to get the list
of allowed API attributes.
Notes are not given percent coordinates when created, however moving
or resizing would update the coordinates to percentage. This is why
it was appearing as if the type of coordinates given were random.
* Inline Javascript in the <head> is a minor bottleneck because it
blocks page rendering.
* Fix antiproxying protection (proxy sites like to strip <script> tags
from the header).
Don't embed config options directly in Javascript. Including them in the
Javascript forces us to recompile the Javascript whenever these options
change, which requires us to redeploy the entire site instead of just
restarting the app server.
* Remove `banned_ip_for_download?` config option. This isn't something that usually needs
to be configured.
* Replace the `ipaddress` gem with `ipaddress_2`. The `ipaddress` gem has several methods
we need (`link_local?`, etc) that are only available in master because the gem hasn't had
an official release in several years. `ipaddress_2` is a fork that is more actively
maintained.
