Fix `Cannot write log file 'ffmpeg2pass-0.log' for pass-1 encoding: Permission denied` error
when uploading ugoira files. Caused by the fact that 2-pass encoding tries to write a log file in
the current directory by default, which fails in production because the default working directory in
the Docker image is /danbooru, which is read-only.
Fix certain ugoiras having very low quality webm samples. This was
because we had a target bitrate of 5 Mbps, but this wasn't enough for
videos that were high resolution or that had choppy, hard-to-compress
motion, such as post 5081776 (nsfw).
This is used to provide higher resolution thumbnails for high pixel
density displays, such as phones or laptops. If your screen has a 2x
pixel density ratio, then 360x360 thumbnails will be rendered at 720x720
resolution.
We use WebP here because it's about 15% smaller than the equivalent
JPEG, and because if a device has a high enough pixel density to use
this, then it probably supports WebP.
720x720 thumbnails average about 36kb in size, compared to 20.35kb for
360x360 thumbnails and 7.55kb for 180x180 thumbnails.
These other formats aren't actually generated during upload, but support
for creating them is there.
Also tune the parameters for generating JPEGs:
* Use Q=85 instead of Q=90 because Q=85 enables 4:2:0 chroma
subsampling, while Q=90 doesn't use subsampling. Subsampling reduces
filesize by ~30% in most cases. It does reduce quality for certain
images, particularly for images with lots of bright red, but in most
cases the quality difference isn't noticeable.
* Enable several MozJPEG-specific options, including trellis
quantization and scan optimization. These reduce filesize without
reducing quality, at the cost of slower encoding times.
Calculate the dimensions of thumbnails ourselves instead of letting
libvips calculate them for us. This way we know the exact size of
thumbnails, so we can set the right width and height for <img> tags. If
we let libvips calculate thumbnail sizes for us, then we can't predict
the exact size of thumbnails, because sometimes libvips rounds numbers
differently than us.
The problem was that we were stripping color profiles from thumbnails,
but we weren't setting `export_profile: "srgb"` to convert images to
sRGB first. This resulted in wrong colors for images with non-sRGB color
profiles, such as Adobe RGB.
The fix is to convert images to sRGB when possible, while leaving CMYK
and greyscale images alone. We leave CMYK images alone because we can't
convert CMYK to sRGB without losing color. We leave greyscale images
alone if they don't have a color profile, that way they stay as
one-channel greyscale (or two-channel greyscale, in case of alpha)
instead of being converted to three-channel sRGB. However, if a
greyscale image has a color profile, then we have to convert to sRGB,
otherwise the colors would be wrong when we strip the profile.
We also have to set the import profile, otherwise images with broken
embedded color profiles won't have a fallback profile and may get
incorrect colors. In this case we also have to be careful, because we
can't specify an sRGB fallback for greyscale or CMYK images.
* Make it so replacing a post doesn't generate a dummy upload as a side effect.
* Make it so you can't replace a post with itself (the post should be regenerated instead).
* Refactor uploads and replacements to save the ugoira frame data when
the MediaAsset is created, not when the post is created. This way it's
possible to view the ugoira before the post is created.
* Make `download_file!` in the Pixiv source strategy return a MediaFile
with the ugoira frame data already attached to it, instead of returning it
in the `data` field then passing it around separately in the `context`
field of the upload.
Fix how the duration of videos and animated GIFs / PNGs is calculated.
If we can't determine the duration from the file metadata, then play the
entire video or animation back using FFmpeg and scrape the duration and
frame count.
This is necessary for things like WebM files where the duration metadata
is optional, or animated GIFs and PNGs that don't have a duration field
in the metadata, only a frame count and a sequence of frame delays.
Fix certain animated PNGs returning NaN as the duration because the
frame rate was being reported as "0/0" by FFMpeg. This happens when the
animation has zero delay between frames. This is supposed to mean a PNG
with an infinitely fast frame rate, but in practice browsers limit it to
around 10FPS. The exact frame rate browsers will use is unknown and
implementation defined.
Add methods to MediaFile to calculate the duration, frame count, and
frame rate of animated GIFs, PNGs, Ugoiras, and videos.
Some considerations:
* It's possible to have a GIF or PNG that's technically animated but
just has one frame. These are treated as non-animated images.
* It's possible to have an animated GIF that has an unspecified
frame rate. In this case we assume the frame rate is 10 FPS; this is
browser dependent and may not be correct.
* Animated GIFs, PNGs, and Ugoiras all support variable frame rates.
Technically, each frame has a separate delay, and the delays can be
different frame-to-frame. We report only the average frame rate.
* Getting the duration of an APNG is surprisingly hard. Most tools don't
have good support for APNGs since it's a rare and non-standardized
format. The best we can do is get the frame count using ExifTool and the
frame rate using ffprobe, then calculate the duration from that.
Fix a bug where where PNG images could be incorrectly detected as
exif-rotated. This would happen when a PNG contained the
IFD0:Orientation flag. It's technically possible for a PNG to contain
this flag, but it's ignored by libvips and by browsers.
post #3762340 (nsfw) is an example of a PNG like this.
The fix is to use `autorot` to let libvips apply the rotation instead of
trying to interpret the exif data ourselves. Note that libvips-8.9 has a
bug where it doesn't strip the orientation flag after applying
`autorot`, which leads to the image being incorrectly rotated a second
time when generating the thumbnail. Use libvips-8.11 instead.
Rotate the image based on the EXIF orientation flag when generating
thumbnails and samples.
Also fix the width and height to be calculated correctly for rotated
images. Vips gives us the unrotated width and height of the image; we
have to detect whether the image is rotated and swap the width and
height manually to correct them. For example, if an image with the
"Rotate 90 CW" flag is 100x500 before rotation, then after rotation it's
500x100. This should fix#4883 (Exif rotation breaks Javascript fit-to-window)
We also have to fix it so that regenerating a post updates the width and
height of the post, in the event that it's a rotated image.
Finally we set `image-orientation: from-image;` even though it's
probably not necessary.
Remove code for working with older versions of libvips. This makes
libvips 8.10+ a hard requirement. Older versions were already broken and
failed certain tests in the test suite.
Fix a bug where generating thumbnails failed for certain images when
using libvips 8.10. Specifically, it failed for single-channel greyscale
images and four-channel CMYK images without an embedded color profile.
In these cases we specified an sRGB fallback profile, but under libvips
8.10 this failed because the sRGB profile was incompatible with
single-channel and four-channel images. Before libvips 8.10 this worked,
but as of 8.10 it's a hard error.
The way libvips handles fallback color profiles differs across versions,
so we have to use different arguments for different versions. In 8.7,
vips doesn't have builtin color profiles, so we have to specify our own
manually. In 8.9, it has builtin profiles, so we can omit the import
profile, but we're still required to set the export profile to sRGB,
otherwise it will leave CMYK images as CMYK when generating thumbnails.
In 8.10, we have to _not_ to set the import or export profile to sRGB,
otherwise it will fail with an incompatible profile error when it tries
to convert CMYK images to RGB.
The builtin sRGB profile used by libvips[1] is different than the one we
used previously[2]. The builtin one comes from LCMS[3], whereas ours
came from ArgyllCMS.[4] Not all sRGB profiles are created the same[5],
so this may result in some imperceptible differences in thumbnail
output. The ArgyllCMS profile was used before because it seemed to be
the best one[6], but realistically it probably doesn't matter.
1: https://github.com/libvips/libvips/blob/v8.10.6/libvips/colour/profiles/sRGB.icm
2: 906eec190d/config/sRGB.icm
3: https://www.littlecms.com/
4: https://www.argyllcms.com/
5: https://ninedegreesbelow.com/photography/srgb-profile-comparison.html
6: https://ninedegreesbelow.com/photography/srgb-profile-comparison.html#addendum
Fix regression in ef2857667 that caused animated GIFs and PNGs to
generate thumbnails that were larger than 150x150.
Also fix a bug with cropped previews not being generated for animated
GIFs and PNGs.
flash files can be quite big (the biggest on danbooru.donmai.us being
68.6MB atm). Reading it and applying complex transformations twice seems
unnecessary.
MediaFile#dimensions is called twice - in #width and in #height but
it only works on the first call because the file is read to the end and
consumed the first time so when #read is called the second time it only
returns the empty string
* Move image thumbnail generation code to MediaFile::Image.
* Move video thumbnail generation code to MediaFile::Video.
* Move ugoira->webm conversion code to MediaFile::Ugoira.
This separates thumbnail generation from the upload process so that it's
possible to generate thumbnails outside of uploads.
* Use libvips instead of ruby-imagespec for reading dimensions of jpeg, png, and gif files.
* Copy the code for reading the dimensions of flash files from ruby-imagespec.
Fixes an incompatibility between ruby-imagespec and the rubocop gem that
prevented us from including rubocop in the Gemfile.
* Add MediaFile abstraction. A MediaFile represents an image or video file.
* Move filetype detection and dimension parsing code from uploads to MediaFile.
