Upload files in natural order rather than archive order when uploading archive files.
Before files were listed in the same order they appeared in the zip file. This could be in
non-alphabetical order, or even with files from different directories interleaved between each
other. Now files are uploaded in natural order, which is alphabetical order but with numbers sorted
properly, so that `file-9.jpg` appears before `file-10.jpg`.
A couple non-obvious consequences:
* Users can't flag non-rating:G posts in safe mode.
* Non-Gold users can flag Gold-only posts if they're the uploader.
* Add og:image:width, og:image:height, and og:image:type tags.
* Use og:video tags for videos.
* Use 720x720 instead of 150x150 preview images for videos.
* Add duration tag to JSON-LD data for videos.
* Add OpenGraph tags to media assets show page.
* Respect Twitter max image size limits.
* Don't include OpenGraph image tags when someone shares a plain https://danbooru.donmai.us link
with no tag search. This caused random potentially NSFW images to be shown when someone shared a
https://danbooru.donmai.us link on social media, which could be cached for long periods of time.
Fix bug where jobs had the opposite of the intended priority. Populating saved searches had the
highest priority, while processing uploads had the lowest priority.
Caused by Delayed::Job and GoodJob having opposite interpretations of job priorities. In
Delayed::Job, lower numbers had higher priority, while in GoodJob, higher numbers have higher
priority. This was missed when migrating from Delayed::Job to GoodJob.
* Fix exception in 65a17979c due to reference to `@media_asset` instead of `media_asset.
* Fix 05143dc9f not removing links to deleted images on the upload page.
On the upload page, change the file size info beneath the image to be formatted the same way as on
the media assets page, and to include a menu with links to offsite reverse image search tools.
Allow admins to delete media asset files.
This only deletes the image file itself, not the upload or media asset record. The upload will still
be in the user's upload list, but the image will be gone. The media asset page will still exist, but
it will only show the file's metadata, not the image itself. We don't delete the metadata so we have
a record of what the file's MD5 was and who uploaded it, to prevent the file from being uploaded
again and to take action against the user if necessary.
On the show page, add a icon beneath the image linking to the Danbooru post if the asset has been posted.
FIXME: Downstream boorus should change the icon to something else.
Show sources on the media asset show page. An asset can have more than one source if the same
file is uploaded from multiple sites.
Only sources from known sites are shown. Sources from unknown sites aren't shown because they
could potentially contain private information or identify the uploader in some way.
Known issue: Twitter posts often show two sources, the direct image URL and the page URL. This is
because someone uploaded the direct image URL first, and we're not able to tell that the image URL
and the page URL are for the same tweet.
Redesign the media assets show page to:
* Include sidebar with AI tags and image metadata.
* Include next and previous image buttons.
* Make the image use 100% of the available screen space and to scroll with the window.
* When a tag's category is changed, also change the category of any aliases pointing to it. For
example, if "ff7" is aliased to "final_fantasy_vii", and "final_fantasy_vii" is changed to a
copyright tag, then change the empty "ff7" tag to be a copyright tag too.
* Don't allow changing the category of an aliased tag. For example, if "ff7" is aliased to
"final_fantasy_vii", then don't allow changing the "ff7" tag to be a non-copyright tag.
This ensures that the categories of aliased tags stay in sync with that of their parent tags. This
way aliased tags are colored correctly in wikis and other places.
When regenerating thumbnails for a media asset, don't redistribute the original file. This is
unnecessary and also slow if it's a large file on remote storage.
Fix placeholder names like weibo_1234, bilibili_1234, nijie1234, and nicoseiga1234 being suggested
as Other Names when creating new artist entries. These are meant to be placeholders for the tag
name, not used as other names.
Hide the "Formatting help" link for single-line DText fields in dialog boxes, such as after the
"Reason" field in the flag, appeal, and report dialog boxes.
The `date:` metatag depends on the current user's time zone, so the post count for `date:` searches
needs to be cached separately for different users, otherwise it could lead to incorrect page counts
if two users with different time zones search for the same `date:` metatag at the same time.
Fix the email deliverability check to return undeliverable when the MX lookup returns a domain or IP
address that is invalid or that refuses connections. For example, `yeah.com` returns the invalid IP
address 0.0.0.0 for its MX record.
Don't allow favgroup names that:
* Start or end with underscores.
* Contain multiple underscores in a row.
* Contain asterisks or non-printable characters.
* Consist of only underscores.
* Consist of only digits (conflicts with `favgroup:1234` syntax).
Add a fix script that fixes favgroups that violate these rules and notifies the user.
Fix the ban! and unban! methods to:
* Lock the artist while it is being banned or unbanned.
* Perform the edits as a mass update, so that the posts are updated in parallel.
* Edit the artist as the banner rather than as the current user.
* Soft delete the banned_artist implication when an artist is unbanned instead of hard deleting it.
* Ignore the banned_artist implication if it's deleted.
When a user is banned, send them a "You have been banned" dmail instead of a "Your user record has
been updated" dmail.
When a user loses approver status due to inactivity, don't seen them a "Your user record has been
updated" dmail for the "Lost approver privileges" neutral feedback they receive.
Fix a bug where, if you were uploading an entire 4chan thread, then the source of each post would
get set to the 4chan thread, rather than to the individual 4chan post.
Fix API calls returning HTTP 412 (Precondition Failed) errors. It seems we need to spoof the Referer as
`https://bilibili.com` for API calls to succeed.
Allow uploading .zip, .rar, and .7z files from disk. The archive will be extracted and the images
inside will be uploaded.
This only works for archive files uploaded from disk, not from a source URL.
Post source URLs will look something like this: "file://foo.zip/1.jpg", "file://foo.zip/2.jpg", etc.
Sometimes artists uses Shift JIS or other encodings instead of UTF-8 for filenames. In these cases
we just assume the filename is UTF-8 and replace invalid characters with '?', so filenames might be
wrong in some cases.
There are various protections to prevent uploading malicious archive files:
* Archives with more than 100 files aren't allowed.
* Archives that decompress to more than 100MB aren't allowed.
* Archives with filenames containing '..' components aren't allowed (e.g. '../../../../../etc/passwd').
* Archives with filenames containing absolute paths aren't allowed (e.g. '/etc/passwd').
* Archives containing symlinks aren't allowed (e.g. 'foo -> /etc/passwd').
* Archive types other than .zip, .rar, and .7z aren't allowed (e.g. .tar.gz, .cpio).
* File permissions, owners, and other metadata are ignored.
Partial fix for #5340: Add support for extracting archive attachments from certain sources
Fix /posts/random?tags=touhou to redirect to /posts/1234?q=touhou instead of /posts/1234?tags=touhou.
Fixes the search bar on random posts not containing the current search.
Fix temp files generated during the upload process not being cleaned up quickly enough. This included
downloaded files, generated preview images, and Ugoira video conversions.
Before we relied on `Tempfile` cleaning up files automatically. But this only happened when the
Tempfile object was garbage collected, which could take a long time. In the meantime we could have
hundreds of megabytes of temp files hanging around.
The fix is to explicitly close temp files when we're done with them. But the standard `Tempfile`
class doesn't immediately delete the file when it's closed. So we also have to introduce a
Danbooru::Tempfile wrapper that deletes the tempfile as soon as it's closed.
The `pry` gem was removed in e698bf91 because we replaced `pry-byebug` with the standard debugger.
Add it back because `pry` is better than `irb` and we still can use the standard debugger with it.
* Call it "Changes" when referring to the global list of changes (pool changes, artist changes, etc).
* Call it "History" when referring to the history of a single item (pool history, artist history, etc).
* Put the artist history search form on the /artist_versions page, and remove the /artist_versions/search endpoint.
Fix external HTTP requests not working when the HTTP proxy was enabled. Caused by the `public_only`
option (which prevents SSRF attacks by validating that the URL doesn't resolve to a local IP) being
incompatible with the `proxy` option.
Don't log a mod action when a user deletes their own account. This isn't a moderator action, so it
doesn't belong here. Account deletions are still logged on the /user_events page (visible to mods only).
A mod action is still logged when the Owner-level user deletes someone else's account.
Add media_asset_id and old_media_asset_id columns for associating replacements with media assets.
This way we can easily tell which replacements don't have a media asset (with the md5 alone we can't
tell whether the media asset actually exists).
Add a fix script to download images from Gelbooru for old replacements where we deleted the original
image. For archival purposes, we want to try to find the original file for every replacement.
These images will be uploaded as unposted assets under DanbooruBot's name.
Add a fix script that imports the md5 for old post replacements from the corresponding DanbooruBot
replacement comment, then deletes all replacement comments.
There are about 250 replacements left that still have a null md5 because they don't have a matching
comment. This is because if a post was replaced but the file didn't change, it didn't leave a comment.
evazion
evazion