Break the hourly/daily/weekly/monthly maintenance tasks down into
individual delayed jobs. This way if one task fails, it won't prevent
other tasks from running. Also, jobs can be run in parallel, and can be
individually retried if they fail.
Move the metadata parsing code from MediaAsset to ExifTool::Metadata so
we can use it outside the context of a MediaAsset, in particular when
dealing with a MediaFile that hasn't been saved to disk yet.
Set `crossorigin="anonymous"` on the <img> tag for thumbnails. This
makes it so we don't send session cookies in requests for thumbnails.
Since images are served from cdn.donmai.us, and since session cookies
are set on *.donmai.us, session cookies are sent by default with every
thumbnail request.
Not sending cookies saves up to 1kb of overhead per thumbnail request
(in reality, it's less than this because of HTTP/2 multiplexing and
header compression).
* Listen on port 3000 instead of port 80. Port 80 is prone to conflicts
with other webservers.
* Use the production version of the Danbooru Docker image. It's less
likely to have bugs than master.
* Fix the autoinstall script to work with `curl ... | sh`.
* Lower PUMA_WORKERS to 1 to reduce memory usage.
Caused by d854bf6b. Banning an artist would deadlock because it was
performed in a transaction, which didn't work with the `parallel_each`
inside the "create an implication to banned_artist" step.
Fix a bug where where PNG images could be incorrectly detected as
exif-rotated. This would happen when a PNG contained the
IFD0:Orientation flag. It's technically possible for a PNG to contain
this flag, but it's ignored by libvips and by browsers.
post #3762340 (nsfw) is an example of a PNG like this.
The fix is to use `autorot` to let libvips apply the rotation instead of
trying to interpret the exif data ourselves. Note that libvips-8.9 has a
bug where it doesn't strip the orientation flag after applying
`autorot`, which leads to the image being incorrectly rotated a second
time when generating the thumbnail. Use libvips-8.11 instead.
Hourly pruning of expired uploads was failing because of nil deference
errors in `media_asset.destroy!`. There are various cases where an
upload doesn't have a media asset, for example when the source url
fails to download or when the upload is of an invalid filetype.
Just trigger on pushes to the betabooru branch instead of on the Docker
image being built. The workflow_run event keeps triggering on the master
branch even though it's supposed to only trigger on the betabooru
branch. This makes it fail because it doesn't have access to the secrets
in the betabooru environment.
This does mean the deploy will fail if the image hasn't been previously
built by a push to master first.
`IFD0:Orientation` is the orientation of the main image.
`IFD1:Orientation` is the orientation of the embedded thumbnail, if it
has one. Using `IFD1:Orientation` was incorrect here because some images
have a non-rotated main image but a rotated thumbnail. Post #1023563 is
an example.
Rotate the image based on the EXIF orientation flag when generating
thumbnails and samples.
Also fix the width and height to be calculated correctly for rotated
images. Vips gives us the unrotated width and height of the image; we
have to detect whether the image is rotated and swap the width and
height manually to correct them. For example, if an image with the
"Rotate 90 CW" flag is 100x500 before rotation, then after rotation it's
500x100. This should fix#4883 (Exif rotation breaks Javascript fit-to-window)
We also have to fix it so that regenerating a post updates the width and
height of the post, in the event that it's a rotated image.
Finally we set `image-orientation: from-image;` even though it's
probably not necessary.
* Make danbooru:images:validate task run in parallel.
* Remove task for regenerating thumbnails.
* Move all rake tasks into same file.
* Add usage instructions.
Fix the test suite failing when trying to run it in the default state
with no config file or API keys configured. Most source sites require
API keys or login credentials to be set in order to work. Skip these
tests when credentials aren't configured.
Add `less` to the Docker image to fix an issue with running `bin/rails console`.
The console uses Pry[1], which has an issue where it pipes long output
through `less`, but it tries to use the -X option, which is only
supported by GNU less, not Busybox less. There's a open bug about this
in the Pry repo dating back to 2014[2].
Add `tini` and use it as the Docker entrypoint to ensure we forward
signals to child processes and reap zombie children properly. This fixes
an issue where if you ran something like:
docker run ghcr.io/danbooru/danbooru bash -c 'bin/rails db:test:prepare && bin/rails test'
Then you couldn't use control-C to stop the container. This was because
bash wasn't forwarding signals to its children, and because by default,
programs running as PID 1 ignore SIGINT and SIGTERM. See [3][4] for details.
1: https://github.com/pry/pry
2: https://github.com/pry/pry/issues.1248
3: https://github.com/krallin/tini/issues.8
4: https://gist.github.com/StevenACoffman/41fee08e8782b411a4a26b9700ad7af5#dont-run-pid-1
Make it so that when ARCHIVE_DATABASE_URL isn't set, it defaults to
DATABASE_URL. In other words, if you don't have a separate archive
database configured, then default to using the main database for
post/pool versions.
Fixes an issue where running the test suite would fail if you didn't
explicitly set ARCHIVE_DATABASE_URL because it tried to use
`archive_test` as the post/pool versions database name.
Autotag `greyscale`, `non-repeating_animation`, and `exif_rotation`.
Note that this does not detect all (or even most) greyscale images.
Artists often save greyscale images as RGB instead of as greyscale.
Automatically tag animated_gif and animated_png when a post is edited.
Add them back if the user tries to remove them from an animated post,
or remove them if the user tries to add them to a non-animated post.
Before we added these tags at upload time, but it was possible for users
to remove them after upload, or to incorrectly add them to non-animated
posts. They were added at upload time because we couldn't afford to open
the file and parse the metadata on every tag edit. Now that we save the
metadata in the database, we can do this.
This also makes it so you can't tag ugoira on non-ugoira files.
Known bug: it's possible to have an animated GIF where every frame is
identical. Post #3770975 is an example. This will be detected as an
animated GIF even though visually it doesn't appear to be animated.
Fixes#4041: Animated_gif tag not added to preprocessed uploads
Remove code for working with older versions of libvips. This makes
libvips 8.10+ a hard requirement. Older versions were already broken and
failed certain tests in the test suite.
Fix it so that you can reapprove a failed BUR to run it again. Before
this would fail because it would end up trying to create the aliases
or implications again, which would fail because they already existed.
Now it ignores when an alias or implication already exists. It will
however finish tagging the posts if they haven't been fully moved.
No longer used now that we use Puma in production. If you still used
Unicorn in your install, switch to `bin/rails server` instead. See
config/puma.rb for config settings.
No longer used now that we use Kubernetes to deploy the site instead of
Capistrano.
If you run your own installation of Danbooru, and you used Capistrano to
deploy your site, it is recommended that you switch to either the Docker
Compose file (for personal installs), the Procfile (for non-Dockerized,
development environments), or Kubernetes (for production environments;
see https://github.com/danbooru/danbooru-infrastructure/tree/master/k8s
for Danbooru's production configuration).
The artist ban tests deadlocked because of a weird interaction between
threads and database transactions when tagging posts in parallel. Add a
hack to work around it.
