uploads: fix temp files not being cleaned up quickly enough.

Fix temp files generated during the upload process not being cleaned up quickly enough. This included
downloaded files, generated preview images, and Ugoira video conversions.

Before we relied on `Tempfile` cleaning up files automatically. But this only happened when the
Tempfile object was garbage collected, which could take a long time. In the meantime we could have
hundreds of megabytes of temp files hanging around.

The fix is to explicitly close temp files when we're done with them. But the standard `Tempfile`
class doesn't immediately delete the file when it's closed. So we also have to introduce a
Danbooru::Tempfile wrapper that deletes the tempfile as soon as it's closed.
This commit is contained in:
evazion
2022-11-15 17:34:59 -06:00
parent 21a779455f
commit e935f01358
14 changed files with 67 additions and 27 deletions

View File

@@ -17,8 +17,9 @@ class MediaFile::Ugoira < MediaFile
end
def close
file.close
preview_frame.close
super
@preview_frame&.close
# XXX should clean up `convert` too
end
def metadata
@@ -52,7 +53,7 @@ class MediaFile::Ugoira < MediaFile
raise RuntimeError, "can't convert ugoira to webm: no ugoira frame data was provided" unless frame_delays.present?
Danbooru::Archive.extract!(file) do |tmpdir, filenames|
output_file = Tempfile.new(["ugoira-conversion", ".webm"], binmode: true)
output_file = Danbooru::Tempfile.new(["danbooru-ugoira-conversion-#{md5}-", ".webm"], binmode: true)
# Duplicate last frame to avoid it being displayed only for a very short amount of time.
last_file_name = File.basename(filenames.last)
@@ -90,8 +91,8 @@ class MediaFile::Ugoira < MediaFile
private
def preview_frame
FFmpeg.new(convert).smart_video_preview
@preview_frame ||= FFmpeg.new(convert).smart_video_preview
end
memoize :preview_frame, :dimensions, :convert, :metadata
memoize :dimensions, :convert, :metadata
end