Files
danbooru/app/logical/media_file/image.rb
evazion 48ecb80d6b Fix #5230: video upload 500 error (StatementInvalid) & empty error panel on page
Fix StatementInvalid exception when uploading https://files.catbox.moe/vxoe2p.mp4.

This was a result of multiple bugs:

* First, generating thumbnails for the video failed. This was because
  the video uses the AV1 codec, which FFmpeg failed to decode. It failed
  because our version of FFmpeg was built without the `--enable-libdav1d`
  flag, so it uses the builtin AV1 decoder, which apparently can't
  handle this particular video (it spews a bunch of errors about "Failed
  to get pixel format" and "missing sequence header" and "failed to get
  reference frame").

* Because generating the thumbnails failed, an exception was raised. We
  tried to save the error message in the upload_media_assets.error
  field. However, this also failed because the error message was 77kb
  long (it contained the entire output of the ffmpeg command), but the
  `upload_media_assets` table had a btree index on the `error` column,
  which meant the maximum length of the error column was limited to
  ~2.7kb. This lead to a StatementInvalid exception being raised.

* Because the StatementInvalid exception was raised while we were trying
  to set the upload media asset's status to `failed`, the upload was
  left stuck in the `processing` state rather than being set to the
  `failed` state.

* Because the upload was stuck in the `processing` state, the upload
  page would hang forever waiting for the upload to complete.

The fixes are to:

* Build FFmpeg with `--enable-libdav1d` to use libdav1d for decoding AV1
  videos instead of the builtin AV1 decoder.

* Remove the index on the `upload_media_assets.error` column so that
  setting overly long error messages won't fail.

* Catch unexpected exceptions in ProcessUploadMediaAssetJob so we can
  mark uploads as failed, even if `process_upload!` itself fails because
  it raises an unexpected exception inside its own exception handler.

* Check that the video is playable with `MediaFile::Video#is_corrupt?` before
  allowing it to be uploaded. This way we can return a better error
  message if we can't generate thumbnails because the video isn't
  playable. This requires decoding the entire video, so it means uploads
  may take several seconds longer for long videos. It's also a security
  risk in case ffmpeg has any bugs.

* Define `MediaAsset#preview!` as raising an exception on error, so
  it's clear that generating thumbnails can fail. Define `MediaAsset#preview`
  as returning nil on error for when we don't care about the cause of
  the error.
2022-10-26 22:49:55 -05:00

159 lines
4.6 KiB
Ruby

# frozen_string_literal: true
# A MediaFile for a JPEG, PNG, or GIF file. Uses libvips for resizing images.
#
# @see https://github.com/libvips/ruby-vips
# @see https://libvips.github.io/libvips/API/current
class MediaFile::Image < MediaFile
delegate :thumbnail_image, to: :image
def dimensions
image.size
rescue Vips::Error
[0, 0]
end
def is_supported?
case file_ext
when :avif
# XXX Mirrored AVIFs should be unsupported too, but we currently can't detect the mirrored flag using exiftool or ffprobe.
!metadata.is_rotated? && !metadata.is_cropped? && !metadata.is_grid_image? && !metadata.is_animated_avif?
when :webp
!is_animated?
else
true
end
end
def is_corrupt?
image.stats
false
rescue Vips::Error
true
end
def duration
return nil if !is_animated?
video.duration
end
def frame_count
case file_ext
when :gif, :webp
image.get("n-pages") if image.get_fields.include?("n-pages")
when :png
metadata.fetch("PNG:AnimationFrames", 1)
when :avif
video.frame_count
else
nil
end
end
def frame_rate
return nil if !is_animated? || frame_count.nil? || duration.nil? || duration == 0
frame_count / duration
end
def channels
image.bands
end
def colorspace
image.interpretation
end
def resize!(max_width, max_height, format: :jpeg, quality: 85, **options)
# @see https://www.libvips.org/API/current/Using-vipsthumbnail.md.html
# @see https://www.libvips.org/API/current/libvips-resample.html#vips-thumbnail
if colorspace.in?(%i[srgb rgb16])
resized_image = thumbnail_image(max_width, height: max_height, import_profile: "srgb", export_profile: "srgb", **options)
elsif colorspace == :cmyk
# Leave CMYK as CMYK for better color accuracy than sRGB.
resized_image = thumbnail_image(max_width, height: max_height, import_profile: "cmyk", export_profile: "cmyk", intent: :relative, **options)
elsif colorspace.in?(%i[b-w grey16]) && has_embedded_profile?
# Convert greyscale to sRGB so that the color profile is properly applied before we strip it.
resized_image = thumbnail_image(max_width, height: max_height, export_profile: "srgb", **options)
elsif colorspace.in?(%i[b-w grey16])
# Otherwise, leave greyscale without a profile as greyscale because
# converting it to sRGB would change it from 1 channel to 3 channels.
resized_image = thumbnail_image(max_width, height: max_height, **options)
else
raise NotImplementedError
end
if resized_image.has_alpha?
resized_image = resized_image.flatten(background: 255)
end
output_file = Tempfile.new(["image-preview-#{md5}", ".#{format.to_s}"])
case format.to_sym
when :jpeg
# https://www.libvips.org/API/current/VipsForeignSave.html#vips-jpegsave
resized_image.jpegsave(output_file.path, Q: quality, strip: true, interlace: true, optimize_coding: true, optimize_scans: true, trellis_quant: true, overshoot_deringing: true, quant_table: 3)
when :webp
# https://www.libvips.org/API/current/VipsForeignSave.html#vips-webpsave
resized_image.webpsave(output_file.path, Q: quality, preset: :drawing, smart_subsample: false, effort: 4, strip: true)
when :avif
# https://www.libvips.org/API/current/VipsForeignSave.html#vips-heifsave
resized_image.heifsave(output_file.path, Q: quality, compression: :av1, effort: 4, strip: true)
else
raise NotImplementedError
end
MediaFile::Image.new(output_file)
end
def preview!(max_width, max_height, **options)
w, h = MediaFile.scale_dimensions(width, height, max_width, max_height)
preview_frame.resize!(w, h, size: :force, **options)
end
def preview_frame
if is_animated?
FFmpeg.new(file).smart_video_preview
else
self
end
end
def is_animated?
frame_count.to_i > 1
end
def is_animated_gif?
file_ext == :gif && is_animated?
end
def is_animated_png?
file_ext == :png && is_animated?
end
def is_animated_webp?
file_ext == :webp && is_animated?
end
def is_animated_avif?
file_ext == :avif && is_animated?
end
# Return true if the image has an embedded ICC color profile.
def has_embedded_profile?
image.icc_import(embedded: true)
true
rescue Vips::Error
false
end
# @return [Vips::Image] the Vips image object for the file
def image
Vips::Image.new_from_file(file.path, fail: strict).autorot
end
def video
FFmpeg.new(file)
end
memoize :image, :video, :dimensions, :is_corrupt?, :is_animated_gif?, :is_animated_png?
end