Allow logged out users to call https://danbooru.donmai.us/profile.json.
This allows getting information on default settings and limits for
anonymous users.
May be a breaking API change if users were using the HTTP response code
from /profile.json to check if they were successfully logged in.
Add a Suggested tags list to the Related Tags box. The suggested tags
are just the AI tags for the post.
Suggested tags are currently hidden in CSS for beta testing. Use custom
CSS to unhide them.
Change the /ai_tags page to show only posts by default, not both posts
and unposted media assets mixed together. Showing media assets tended to
confuse users about why they couldn't add tags to these images. It also
distracted from the page's primary use case, which is gardening posts.
Add "Add" and "Remove" buttons beneath thumbnails on the /ai_tags page.
These let you add the tag to the post if it's correct, or remove it if
it's wrong.
Add a database model for storing AI-predicted tags, and add a UI for browsing and searching these tags.
AI tags are generated by the Danbooru Autotagger (https://github.com/danbooru/autotagger). See that
repo for details about the model.
The database schema is `ai_tags (media_asset_id integer, tag_id integer, score smallint)`. This is
designed to be as space-efficient as possible, since in production we have over 300 million
AI-generated tags (6 million images and 50 tags per post). This amounts to over 10GB in size, plus
indexes.
You can search for AI tags using e.g. `ai:scenery`. You can do `ai:scenery -scenery` to find posts
where the scenery tag is potentially missing, or `scenery -ai:scenery` to find posts that are
potentially mistagged (or more likely where the AI missed the tag).
You can browse AI tags at https://danbooru.donmai.us/ai_tags. On this page you can filter by
confidence level. You can also search unposted media assets by AI tag.
To generate tags, use the `autotag` script from the Autotagger repo, something like this:
docker run --rm -v ~/danbooru/public/data/360x360:/images ghcr.io/danbooru/autotagger ./autotag -c -f /images | gzip > tags.csv.gz
To import tags, use the fix script in script/fixes/. Expect a Danbooru-size dataset to take
hours to days to generate tags, then 20-30 minutes to import. Currently this all has to be done by hand.
Add a system for upgrading accounts using upgrade codes. Users purchase
an upgrade code off-site then redeem it on-site to upgrade their account
to Gold. Upgrade codes are randomly pre-generated and are one time use
only. Codes have enough randomness that guessing a code is infeasible.
Add options to disable comments, the forum, and autocomplete. This is
for personal boorus and potentially for safe mode. Note that disabling
the forum may cause difficulties with creating and approving BURs.
Disabling comments and the forum merely hides them from most areas,
rather than completely removing them.
Factor out the Stripe code from the UserUpgrade class. Introduce a new
PaymentTransaction abstract class that represents a payment with some
payment processor, and a PaymentTransaction::Stripe class that
implements transactions with Stripe.
Note that we can't completely eliminate Stripe even though we no longer
accept payments with it because we still need to be able to look up old
payments in Stripe.
Show a "This page has been removed because of a takedown request" error when
an unauthorized user searches for a banned tag, or tries to view a banned post.
Raise an error if the search is invalid for one of the following reasons:
* It contains multiple conflicting order: metatags (e.g. `order:score order:favcount` or `ordfav:a ordfav:b`).
* It contains a metatag that can't be used more than once: (e.g. `limit:5 limit:10`, `random:5 random:10`).
* It contains a metatag that can't be negated (e.g. `-order:score`, `-limit:20`, or `-random:20`).
* It contains a metatag that can't be used in an OR clause (e.g. ` touhou or order:score`, `touhou or limit:20`, `touhou or random:20`).
Show a warning when creating a duplicate artist; that is, when adding a
URL that already belongs to another artist.
This is a soft warning rather than a hard error because there are some
cases where multiple artists legitimately share the same site or account.
Forcibly redirect users to the name change page if their name is
invalid. This means user with invalid names can't do anything or view
any pages until they change their name. API requests are still allowed.
* Add a "Size" menu to the My Uploads / All Uploads pages to allow
changing the thumbnail size.
* Make the My Uploads / All Uploads pages use the same thumbnail size as
the post index page.
* Change the "Gallery | Table" links on the My Uploads page to icons.
Fix the "My Uploads" page showing Admins all uploads, not just their own
uploads.
Changes the URL of the My Uploads page from /uploads to /users/:id/uploads.
This page shows each individual file you've uploaded. This is different
from the regular uploads page because files in multi-file uploads are
not grouped together.
* Make thumbnails on the "My Uploads" page show an icon with an image
count when an upload contains multiple files.
* Make the "My Uploads" page show each upload, not each individual file.
If an upload contains multiple files, they're shown grouped together
under a single upload. This does mean that failed or duplicate uploads
will show up on this page now. This is because this page shows each
upload attempt, not each uniquely uploaded file.
Include media assets in /uploads.json and /uploads/:id.json API responses, like this:
{
"id": 4983629,
"source": "https://www.pixiv.net/en/artworks/96198438",
"uploader_id": 52664,
"status": "completed",
"created_at": "2022-02-12T16:26:04.680-06:00",
"updated_at": "2022-02-12T16:26:08.071-06:00",
"referer_url": "",
"error": null,
"media_asset_count": 1,
"upload_media_assets": [
{
"id": 9370,
"created_at": "2022-02-12T16:26:08.068-06:00",
"updated_at": "2022-02-12T16:26:08.068-06:00",
"upload_id": 4983629,
"media_asset_id": 5206552,
"status": "pending",
"source_url": "https://i.pximg.net/img-original/img/2022/02/13/01/20/19/96198438_p0.jpg",
"error": null,
"page_url": "https://www.pixiv.net/artworks/96198438",
"media_asset": {
"id": 5206552,
"created_at": "2022-02-12T16:26:07.980-06:00",
"updated_at": "2022-02-12T16:26:08.061-06:00",
"md5": "90a85a5fae5f0e86bdb2501229af05b7",
"file_ext": "jpg",
"file_size": 1055775,
"image_width": 1052,
"image_height": 1545,
"duration": null,
"status": "active"
}
}
]
}
This is needed so you can check for upload errors in the API, since in a multi-file
upload, each asset can have a separate error message. This is a stopgap solution until
something like /uploads.json?include=upload_media_assets.media_asset works.
Fix requests for non-existent .js pages, for example https://danbooru.donmai.us/oaisfj.js,
raising AbstractController::DoubleRenderError when trying to render the 404 response.
Fix two issues that could lead to duplicate errors when creating posts:
* Fix the submit button on the upload form to disable itself on submit, to prevent
accidental double submit errors.
* Fix a race condition when checking for MD5 duplicates. MD5 uniqueness is checked on both
the Rails level, with a uniqueness validation, and on the database level, with a unique
index on the md5 column. Creating a post could fail with an ActiveRecord::RecordNotUnique
error if the uniqueness validation in Rails passed, but the uniqueness constraint in the
database failed. In this case, we catch the RecordNotUnique error and convert it to a
Rails validation error so we can treat it like a normal validation failure.
Fix a potential exploit where private information could be leaked if
it was contained in the error message of an unexpected exception.
For example, NoMethodError contains a raw dump of the object in the
error message, which could leak private user data if you could force a
User object to raise a NoMethodError.
Fix the error page to only show known-safe error messages from expected
exceptions, not unknown error messages from unexpected exceptions.
API changes:
* JSON errors now have a `message` param. The message will be blank for unknown exceptions.
* XML errors have a new format. This is a breaking change. They now look like this:
<result>
<success type="boolean">false</success>
<error>PaginationExtension::PaginationError</error>
<message>You cannot go beyond page 5000.</message>
<backtrace type="array">
<backtrace>app/logical/pagination_extension.rb:54:in `paginate'</backtrace>
<backtrace>app/models/application_record.rb:17:in `paginate'</backtrace>
<backtrace>app/logical/post_query_builder.rb:529:in `paginated_posts'</backtrace>
<backtrace>app/logical/post_sets/post.rb:95:in `posts'</backtrace>
<backtrace>app/controllers/posts_controller.rb:22:in `index'</backtrace>
</backtrace>
</result>
instead of like this:
<result success="false">You cannot go beyond page 5000.</result>
