Add support for uploading posts from Gelbooru. Note that the translated
tags will include both the Gelbooru tags and the tags from the Gelbooru
post's source. The commentary and artist information will also be taken
from the Gelbooru post's source. The source of the Danbooru post however
will be left as the Gelbooru post itself, not as the Gelbooru post's source.
Remove the last remaining uses of the PixivUgoiraFrameData model. As of
32bfb8407, Ugoira frame data is now stored in the MediaMetadata model,
under the `Ugoira:FrameDelays` EXIF field.
The pixiv_ugoira_frame_data table still exists, but it can be removed
after this commit is deployed.
Fixes#5264: Error when replacing with ugoira.
Automatically add the AI-generated tag to posts that have the
`PNG:Software=NovelAI` EXIF attribute.
This is not foolproof because this metadata may get removed if an
AI-generated post is resaved or uploaded to a site that strips EXIF
metadata. It also only works for NovelAI. Currently it detects 29 out of
177 AI-generated uploads on Danbooru.
Switch the Ugoira Javascript player to using frame delays from the
MediaMetadata model instead of from the PixivUgoiraFrameData model.
After this is deployed it will be safe to remove the PixivUgoiraFrameData
model.
Store Ugoira frame delays in the MediaMetadata model as a fake EXIF
field instead of in the PixivUgoiraFrameData model. This way we can get
rid of the PixivUgoiraFrameData model completely. This is a step towards
fixing #5264.
Remove various fallbacks and workarounds from the Ugoira Javascript
player for legacy browser issues.
* Remove fallbacks for URL, Blob, and ArrayBuffer (these have been
supported in most browsers for years).
* Remove workaround for iOS <6 not supporting Blob URLs.
* Remove workaround for a bug in Safari's handling of the Range HTTP
header. This was fixed in 2015.
* Remove support for playing a list of image URLs instead of playing
images from the .zip file (this was never used).
* Remove support for limiting loadahead of images (it was hardcoded to
always use unlimited loadahead anyways).
The frame data for Ugoira files is stored like this:
[{"file"=>"000000.jpg", "delay"=>65},
{"file"=>"000001.jpg", "delay"=>65},
{"file"=>"000002.jpg", "delay"=>65},
{"file"=>"000003.jpg", "delay"=>65},
{"file"=>"000004.jpg", "delay"=>65},
{"file"=>"000005.jpg", "delay"=>65},
{"file"=>"000006.jpg", "delay"=>65},
{"file"=>"000007.jpg", "delay"=>65},
{"file"=>"000008.jpg", "delay"=>65},
{"file"=>"000009.jpg", "delay"=>65},
{"file"=>"000010.jpg", "delay"=>65}]
This is stored in the pixiv_ugoira_frame_data table in YAML format. This
is a problem because a) we only need the frame delays to play the Ugoira,
not the filenames, and b) storing the data in YAML format is a security
issue that's blocking the upgrade to Rails 7.0.4 (see [1]).
This commit changes the Ugoira Javascript player so that it only uses
the list of frame delays, not the filenames, to play the Ugoira. This
paves the way for storing the frame delays as a simple integer array
instead as a serialized YAML object.
This assumes that the images in a Ugoira zip file are stored in the same
order they should be played back in. This was confirmed by checking every
zip file and verifying that files are actually stored in filename order.
[1]: https://discuss.rubyonrails.org/t/cve-2022-32224-possible-rce-escalation-bug-with-serialized-columns-in-active-record/81017
Whenever the email address normalization procedure changes, the
`normalized_address` column of the email address table must be updated.
This is normally when the list of canonical domain mappings changes.
Renormalizing addresses may also require deleting duplicates.
In the past it was possible for users to create multiple accounts with
the same email address. We had about 9000 such accounts. This removes
the email address from these accounts.
When multiple accounts have the same email address, the account that
visited the site last gets to keep the address.
Add a fix script that fixes invalid email addresses if they can be
fixed, otherwise they're deleted.
For a long time we didn't have any email validation, so we ended up with
a lot of invalid email addresses containing typos or other random garbage.
This tries to fix the most common typos when possible, otherwise the
email address is deleted.
In many cases the user created two accounts, one with a typo in the
email and one with the correct email. In these cases we can't fix the
invalid email, so we just delete it.
Allow moderators to forcibly change the username of other users. This is
so mods can change abusive or invalid usernames.
* A mod can only change the username of Builder-level users and below.
* The user can't change their own name again until one week has passed.
* A modaction is logged when a mod changes a user's name.
* A dmail is sent to the user notifying them of the change.
* The dmail does not send the user an email notification. This is so we
don't spam users if their name is changed after they're banned, or if
they haven't visited the site in a long time.
The rename button is on the user's profile page, and when you hover over
the user's name and open the "..." menu.
* Fixed a bug where manga posts with a single tag would raise an error
* Fixed a bug where dic.nicovideo.jp/oekaki posts weren't uploadable due
to SSL issues
* Added support for more manga corner cases
Fix not being able to use the full set of search operators on polymorphic `model_id` and
`model_type` attributes. Before things like `search[model_type]=Post` worked, but
`search[model_type_not_eq]=Post` or other `model_type_*` operators didn't.
Log the following information in email headers:
* X-Danbooru-User: the user's name and ID.
* X-Danbooru-IP: the user's IP.
* X-Danbooru-Session: the users' session ID.
* X-Danbooru-URL: the page that triggered the email.
* X-Danbooru-Job-Id: the ID of the background job that sent the email.
* X-Danbooru-Enqueued-At: when the email was queued as a background job.
* X-Danbooru-Dmail: for Dmail notifications, the link to the Dmail.
* X-Request-Id: the request ID of the HTTP request that triggered the email.
Also make it so we log an event in the APM when we send an email.
Fix it so that emails are (hopefully) able to show the one-click
unsubscribe button in Gmail and other mail providers that support the
List-Unsubscribe header. This way users can unsubscribe instead of
marking emails as spam.
* Add the List-Unsubscribe-Post header.
* Fix the disable email notifications endpoint to support POST as well as DELETE requests.
* Fix the disable email notifications endpoint to disable XSRF protection (we don't need
users to be logged in because we use a signed URL instead).
https://www.rfc-editor.org/rfc/rfc8058#section-3.1https://www.rfc-editor.org/rfc/rfc8058#section-8.1
