Allow searching the /uploads and /media_assets pages by the following metatags:
* id:
* md5:
* width:
* height:
* duration:
* mpixels:
* ratio:
* filesize:
* filetype:
* date:
* age:
* status:<processing|active|deleted|expunged|failed> (for /media_assets)
* status:<pending|processing|active|failed> (for /uploads)
* is:<filetype>, is:<status>
* exif:
Examples:
* https://betabooru.donmai.us/media_assets?search[ai_tags_match]=filetype:png
* https://betabooru.donmai.us/uploads?search[ai_tags_match]=filetype:png
Note that in /uploads search, the id:, date:, and age: metatags refer to the upload media asset, not
the upload itself.
Note also that uploads may contain multiple assets, so for example searching uploads by
`filetype:png` will return all uploads containing at least one PNG file, even if they contain other
non-PNG files.
Fix bug where it was possible to submit blank text in various text fields.
Caused by `String#blank?` not considering certain Unicode characters as blank. `blank?` is defined
as `match?(/\A[[:space:]]*\z/)`, where `[[:space:]]` matches ASCII spaces (space, tab, newline, etc)
and Unicode characters in the Space category ([1]). However, there are other space-like characters
not in the Space category. This includes U+200B (Zero-Width Space), and many more.
It turns out the "Default ignorable code points" [2][3] are what we're after. These are the set of 400
or so formatting and control characters that are invisible when displayed.
Note that there are other control characters that aren't invisible when rendered, instead they're
shown with a placeholder glyph. These include the ASCII C0 and C1 control codes [4], certain Unicode
control characters [5], and unassigned, reserved, and private use codepoints.
There is one outlier: the Braille pattern blank (U+2800) [6]. This character is visually blank, but is
not considered to be a space or an ignorable code point.
[1]: https://codepoints.net/search?gc[]=Z
[2]: https://codepoints.net/search?DI=1
[3]: https://www.unicode.org/review/pr-5.html
[4]: https://codepoints.net/search?gc[]=Cc
[5]: https://codepoints.net/search?gc[]=Cf
[6]: https://codepoints.net/U+2800
[7]: https://en.wikipedia.org/wiki/Whitespace_character
[8]: https://character.construction/blanks
[9]: https://invisible-characters.com
Add ability to undelete accounts from within the console. Their password is reset, their name is
restored to their last known user name, and a mod action is logged.
Upload files in natural order rather than archive order when uploading archive files.
Before files were listed in the same order they appeared in the zip file. This could be in
non-alphabetical order, or even with files from different directories interleaved between each
other. Now files are uploaded in natural order, which is alphabetical order but with numbers sorted
properly, so that `file-9.jpg` appears before `file-10.jpg`.
A couple non-obvious consequences:
* Users can't flag non-rating:G posts in safe mode.
* Non-Gold users can flag Gold-only posts if they're the uploader.
Allow admins to delete media asset files.
This only deletes the image file itself, not the upload or media asset record. The upload will still
be in the user's upload list, but the image will be gone. The media asset page will still exist, but
it will only show the file's metadata, not the image itself. We don't delete the metadata so we have
a record of what the file's MD5 was and who uploaded it, to prevent the file from being uploaded
again and to take action against the user if necessary.
* When a tag's category is changed, also change the category of any aliases pointing to it. For
example, if "ff7" is aliased to "final_fantasy_vii", and "final_fantasy_vii" is changed to a
copyright tag, then change the empty "ff7" tag to be a copyright tag too.
* Don't allow changing the category of an aliased tag. For example, if "ff7" is aliased to
"final_fantasy_vii", then don't allow changing the "ff7" tag to be a non-copyright tag.
This ensures that the categories of aliased tags stay in sync with that of their parent tags. This
way aliased tags are colored correctly in wikis and other places.
Fix placeholder names like weibo_1234, bilibili_1234, nijie1234, and nicoseiga1234 being suggested
as Other Names when creating new artist entries. These are meant to be placeholders for the tag
name, not used as other names.
Don't allow favgroup names that:
* Start or end with underscores.
* Contain multiple underscores in a row.
* Contain asterisks or non-printable characters.
* Consist of only underscores.
* Consist of only digits (conflicts with `favgroup:1234` syntax).
Add a fix script that fixes favgroups that violate these rules and notifies the user.
Fix the ban! and unban! methods to:
* Lock the artist while it is being banned or unbanned.
* Perform the edits as a mass update, so that the posts are updated in parallel.
* Edit the artist as the banner rather than as the current user.
* Soft delete the banned_artist implication when an artist is unbanned instead of hard deleting it.
* Ignore the banned_artist implication if it's deleted.
When a user is banned, send them a "You have been banned" dmail instead of a "Your user record has
been updated" dmail.
When a user loses approver status due to inactivity, don't seen them a "Your user record has been
updated" dmail for the "Lost approver privileges" neutral feedback they receive.
Fix a bug where, if you were uploading an entire 4chan thread, then the source of each post would
get set to the 4chan thread, rather than to the individual 4chan post.
Allow uploading .zip, .rar, and .7z files from disk. The archive will be extracted and the images
inside will be uploaded.
This only works for archive files uploaded from disk, not from a source URL.
Post source URLs will look something like this: "file://foo.zip/1.jpg", "file://foo.zip/2.jpg", etc.
Sometimes artists uses Shift JIS or other encodings instead of UTF-8 for filenames. In these cases
we just assume the filename is UTF-8 and replace invalid characters with '?', so filenames might be
wrong in some cases.
There are various protections to prevent uploading malicious archive files:
* Archives with more than 100 files aren't allowed.
* Archives that decompress to more than 100MB aren't allowed.
* Archives with filenames containing '..' components aren't allowed (e.g. '../../../../../etc/passwd').
* Archives with filenames containing absolute paths aren't allowed (e.g. '/etc/passwd').
* Archives containing symlinks aren't allowed (e.g. 'foo -> /etc/passwd').
* Archive types other than .zip, .rar, and .7z aren't allowed (e.g. .tar.gz, .cpio).
* File permissions, owners, and other metadata are ignored.
Partial fix for #5340: Add support for extracting archive attachments from certain sources
Fix /posts/random?tags=touhou to redirect to /posts/1234?q=touhou instead of /posts/1234?tags=touhou.
Fixes the search bar on random posts not containing the current search.
Fix temp files generated during the upload process not being cleaned up quickly enough. This included
downloaded files, generated preview images, and Ugoira video conversions.
Before we relied on `Tempfile` cleaning up files automatically. But this only happened when the
Tempfile object was garbage collected, which could take a long time. In the meantime we could have
hundreds of megabytes of temp files hanging around.
The fix is to explicitly close temp files when we're done with them. But the standard `Tempfile`
class doesn't immediately delete the file when it's closed. So we also have to introduce a
Danbooru::Tempfile wrapper that deletes the tempfile as soon as it's closed.
Don't log a mod action when a user deletes their own account. This isn't a moderator action, so it
doesn't belong here. Account deletions are still logged on the /user_events page (visible to mods only).
A mod action is still logged when the Owner-level user deletes someone else's account.
Fix `Relation passed to #and must be structurally compatible. Incompatible values: [:joins] (ArgumentError)`
exception in `ordfav:evazion ratio:4:3` search. Broken by e849d8f1c.
We were effectively doing this:
q1 = Post.joins(:favorites, :media_asset).where("favorites.user_id = ?", 52664).order("favorites.id DESC")
q2 = Post.joins(:media_asset, :favorites).where("ROUND(media_assets.image_width::numeric / media_assets.image_height::numeric, 2) = 1.33")
q3 = q1.and(q2)
This failed because Rails didn't like the fact that the joins were in a different order when the
queries were `and`-ed together.
Don't allow users to choose names that conflict with search syntax, like `any` or `none`, or names
that impersonate user levels, like `Admin`, `Moderator`, `Anonymous`, etc.
