Add `foreman` to the base Docker image. This way you can do this:
docker run --rm -it -v $PWD:/danbooru ghcr.io/danbooru/danbooru foreman start
to start everything needed to run Danbooru in development mode (except
for the Postgres database). This will start everything listed in the
Procfile:
bin/rails server
bin/good_job start
bin/rails danbooru:cron
bin/webpack-dev-server
Switch the Ruby memory allocator from Glibc malloc to Jemalloc. Jemalloc
supposedly uses less memory than Glibc malloc because it's better at
handling memory fragmentation. It also has detailed internal statistics
to help monitor allocator behavior.
We use the LD_PRELOAD method of loading Jemalloc instead of building it
into Ruby so that we can switch allocators at runtime.
Upgrade bootsnap to 1.9.3 too because Ruby 3.0.3 has a bug that causes
Rails to fail to boot when bootsnap is enabled. Bootsnap 1.9.3 works
around this bug.
Also add libgmp to build with bignum support.
Add a Ruby wrapper library around the libseccomp library. Seccomp is
used to restrict the syscalls a program can make. See comments in
app/logical/seccomp.rb for further details.
This is not used for anything yet. It's simply adding part of the
sandboxing infrastructure for later use.
Update the Postgres client binaries (psql et al) to version 14.0. This
is so they match the server version, and so that pg_amcheck is
available, which was introduced in 14.0.
This requires updating the base image to Ubuntu 21.04 at the same time
because the Postgres repo doesn't support version 14.0 on Ubuntu 20.10.
Add `less` to the Docker image to fix an issue with running `bin/rails console`.
The console uses Pry[1], which has an issue where it pipes long output
through `less`, but it tries to use the -X option, which is only
supported by GNU less, not Busybox less. There's a open bug about this
in the Pry repo dating back to 2014[2].
Add `tini` and use it as the Docker entrypoint to ensure we forward
signals to child processes and reap zombie children properly. This fixes
an issue where if you ran something like:
docker run ghcr.io/danbooru/danbooru bash -c 'bin/rails db:test:prepare && bin/rails test'
Then you couldn't use control-C to stop the container. This was because
bash wasn't forwarding signals to its children, and because by default,
programs running as PID 1 ignore SIGINT and SIGTERM. See [3][4] for details.
1: https://github.com/pry/pry
2: https://github.com/pry/pry/issues.1248
3: https://github.com/krallin/tini/issues.8
4: https://gist.github.com/StevenACoffman/41fee08e8782b411a4a26b9700ad7af5#dont-run-pid-1
Fix Exiftool not being able to get the metadata for compressed SWF
files. Exiftool requires Compress::Zlib as an optional dependency to
decompress compressed SWF files, but it wasn't in the Docker image.
Archive::Zip is required for Zip files and Digest::MD5 for certain other
metadata (see "DEPENDENCIES" in exiftool README).
Include OpenResty in the base Docker image. This is so we can run
OpenResty in front of Danbooru as a reverse proxy to serve static assets
(CSS, JS, and static images living in public/images).
Including the proxy in the same container as the static assets avoids a
lot of problems with trying to share files across separate containers.
Add a model for storing image and video metadata for uploaded files.
Metadata is extracted using ExifTool. You will need to install ExifTool
after this commit. ExifTool 12.22 is the minimum required version
because we use the `--binary` option, which was added in this release.
The MediaMetadata model is separate from the MediaAsset model because
some files contain tons of metadata, and most of it is non-essential.
The MediaAsset model represents an uploaded file and contains essential
metadata, like the file's size and type, while the MediaMetadata model
represents all the other non-essential metadata associated with a file.
Metadata is stored as a JSON column in the database.
ExifTool returns all the file's metadata, not just the EXIF metadata.
EXIF is one of several types of image metadata, hence why we call
it MediaMetadata instead of EXIFMetadata.
Make nokogiri use the bundled version of libxml2 instead of the system
version. In the past installing nokogiri was slow because it had to
compile the bundled version of libxml2, which is partly why we switched
to the system library. Now it's faster because the bundled version comes
pre-compiled with the nokogiri gem.
https://nokogiri.org/#native-gems-faster-more-reliable-installation
Reverts 440bbbb28.
Fix the ca-certificates package not being installed inside the base
Docker image. This caused uploads from HTTPS sites to fail because TLS
certificates couldn't be validated.
* Optimize Dockerfile to minimize size of the Docker image.
* Specify exact versions of important dependencies (Ruby, Node, Vips) to
ensure our dependencies are up to date and locked to known versions.
* Install Vips from source because the version that ships with Ubuntu is too old.
* Install FFmpeg from source because otherwise using the Ubuntu package
pulls in tons of video libraries we don't need, bloating the image.
