Don't allow GET requests to pass the request params in the body instead
of in the URL. While Rails can handle GET params passed in the body, it
goes against spec and it may cause problems if the response is a redirect
and the client doesn't send the body params when following the redirect.
This may be a breaking change for broken API clients who were sending
GET params in the body instead of in the URL. This can happen when people
use HTTP libraries incorrectly.
Add a fix script to delete all accounts with invalid usernames. Also
change it so the owner-level user can delete accounts belonging to other
users.
Users who have logged in in the last year and who have a valid email
address will be given a one week warning. After that all accounts with
invalid names will be deleted. Anyone who has visited the site in the
last 6 months will have already seen a warning page that their name must
be changed to keep using the site.
Fix mod actions to use the same message format everywhere.
Before mod actions were formatted in various inconsistent ways:
* "deleted post #1234"
* "comment #1234 updated by <user>"
* "<user> updated forum #1234"
* "<user> level changed Member -> Builder"
Now all mod actions consistently use this format:
* "deleted post #1234"
* "updated comment #1234"
* "updated forum #1234"
* "promoted <user> from Member to Builder"
This way mod actions are formatted consistently with other actions on
the /user_actions page, where everything is written as "<user> did X".
Also add a fix script to fix existing mod actions.
Allow approvers to bypass the rule that you can't flag a post again if
it was flagged less than 3 days ago. This rule was intended to prevent
flag warring among regular users, which hopefully shouldn't be a problem
among approvers. It was also useless because approvers could always
just directly delete the post even if they couldn't flag it.
Allowing approvers to reflag posts allows them to reinstate flags that
were accidentally approved.
Remove the `CurrentUser.ip_addr` global variable and replace it with
`request.remote_ip`. Before we had to track the current user's IP in a
global variable so that when we edited a post for example, we could pass
down the user's IP to the model and save it in the post_versions table.
Now that we now longer save IPs in version tables, we don't need a global
variable to get access to the current user's IP outside of controllers.
Set various IP address columns to nullable in preparation for dropping them.
In production, some of these tables already contained null values even
though it violated the constraint.
Remove the /ip_addresses page. This page allowed moderators to search
users by IP, and to see recent activity tied to an IP. However, it was
limited to IPs tied to uploads, comments, dmails, artist edits, note
edits, and wiki edits.
Remove this page because it was limited in scope and because there are
better ways of doing what it did. The /user_events page is better at
catching sockpuppets because it tracks IPs for every login, not just for
certain types of edits. And the /user_actions page is better at
monitoring user activity because it shows all activity associated with
an account, not just for certain types of edits.
Removing this allows us to drop IP addresses from all tables besides the
user_events table. This is good because these IPs are no longer necessary
for any purpose, and because storing them forever is a liability.
Make all forum post votes visible to everyone.
When forum votes were first introduced, it was technically possible to
vote on any forum post, including on posts in mod-only threads.
Accordingly, forum post votes were only visible if the forum post itself
was visible. However, there doesn't actually exist any votes on private
forum posts, and trying to filter them out makes the /user_actions page
much slower, so just make them visible to everyone.
Allow moderators to see all events on the /user_events page. Before only
admins could see when a user changed their email, changed their
password, or had a failed login attempt. Now moderators can see these
events too.
Filtering these events out made the /user_actions page slower, and it
wasn't really necessary since merely knowing that a user changed their
email or password isn't that much more sensitive than knowing when they
logged in or out.
Add a /user_actions page. This page shows you a global timeline of
(almost) all activity on the site, including uploads, comments, votes,
edits, forum posts, and so on.
The main things it doesn't include are post edits, pool edits, and
favorites (posts and pools live in a separate database, and favorites
don't have the timestamps we need for ordering).
This page is useful for moderation purposes because it lets you see a
history of almost all of a user's activity on a single page.
Currently this page is mod-only. In the future it will be open to all
users, so you can view the history of your own site activity, or the
activity of others.
Add a user_actions view. This view unions together a bunch of tables to
produce an event log of every action taken by a user.
Also add a bunch of indexes to make queries on this table efficient.
Even though the view is an enormous query combining together about 30
different tables, queries are very efficient as long as every table has
`created_at` and `(user_id, created)` indexes.
Change ID columns from `bigint` (64-bits) to `integer` (32-bits) on various tables.
Rails 6.0 switched the default from bigint to integer for IDs on new
tables, so now we have a mix of tables with integer IDs and bigint IDs.
Switch back to integer IDs on certain tables because we're going to
build a view that unions a bunch of tables together to build a user
activity timeline, and for this purpose all the tables need to have IDs
of the same type in order for Postgres to optimize the query effectively.
Remove the IP address search option from the /moderator/dashboard page.
This was an obsolete way of searching for sockpuppet accounts by IP.
The /user_events page should be used instead.
Unlike other models, notes had a special rule where if you deleted or
undeleted a note, it would always create a new version instead of
merging it into the previous version. Remove this rule since it didn't
have a purpose and it was inconsistent with other versioned models.
Fix a nil reference exception in the sidebar when a wiki page doesn't
belong to a tag.
Also hide the options sidebar on the new wiki page since none of the
options are relevant when creating a new wiki.
Automatically retire artist aliases after they're more than 2 years old.
Before the rule was that artist aliases were only retired if they didn't
have any new posts in the last 2 years.
Replace the "Edit | History | Related | Similar" links on the /tags
index with a popup menu containing more actions, including links to the
tag's version history, links to the tag's aliases and implications,
links to comments and forum discussions mentioning the tag, and links to
find untagged and mistagged posts using AI tags.
Track the history of the tag `category` and `is_deprecated` fields in
the `tag_versions` table.
Adds generic Versionable and VersionFor concerns that encapsulate most
of the history tracking logic. These concerns are designed to make it
easy to add history to any model.
There are a couple notable differences between tag versions and other versions:
* There is no 1 hour edit merge window. All changes to the `category`
and `is_deprecated` fields produce a new version in the tag history.
* New versions aren't created when a tag is created. Versions are only
created when a tag is edited for the first time. The tag's initial
version isn't created until *after* the tag is edited for the first time.
For example, if you change the category of a tag that was last updated
10 years ago, that will create an initial version of the tag backdated
to 10 years ago, plus a new version for your edit.
This is for a few reasons:
* So that we don't have to create new tag versions every time a new tag
is created. This would be wasteful because most tags never have their
category or deprecation status change.
* So that if you make a typo tag, your name isn't recorded in the tag's
history forever.
* So that we can create new tags in various places without having to know
who created the tag (which may be unknown if the current user isn't set).
* Because we don't know the full history of most tags, so we have to
deal with incomplete histories anyway.
This has a few important consequences:
* Most tags won't have any tag versions. They only gain tag versions if
they're edited.
* You can't track /tag_versions to see newly created tags. It only
shows changes to already existing tags.
* Tag version IDs won't be in strict chronological order. Higher IDs may
have created_at timestamps before lower IDs. For example, if you
change the category of a tag that is 10 years old, that will create an
initial version with a high ID, but with a created_at timestamp dated
to 10 years ago.
Fixes#4402: Track tag category changes
Add a tag_versions table for tracking the history of tags.
A couple notable differences from other version tables:
* There is a previous_version_id column that points to the previous
version. This allows finding the first, last, previous, or next
version efficiently in SQL.
* There is a `version` column that tracks the revision number (1, 2, 3, etc).
Post versions and note versions have this, but other version tables don't.
* The `updater_id` column is optional. This is because we don't know who
the last updater was before we started tracking the history of tags,
so the initial updater will be NULL in the first version of the tag.
Fixup for 015c6dc7d. Show a warning about failure to add a tag instead
of raising an exception when trying to tag a post with `char:copy:foo`.
This tries to create a tag named `copy:foo` then set the category to
character, which doesn't work because `copy:foo` isn't a valid tag name.
Drop the ability to write e.g. `create alias foo -> char:bar` in a BUR
to change the tag's type as a side effect. You can only use these
tag type prefixes in tag edits now.
This feature was only intended to be used in tag edits. The fact it
worked elsewhere was unintended behavior.
This feature was problematic because it relied on `Tag.find_or_create_by_name`
automagically changing the tag's category when the tag name contained a
tag category prefix, e.g. `char:hatsune_miku`. This meant that merely
looking up a tag could have the side effect of changing its category.
It was also bad because `find_or_create_by_name` had a hidden dependency
on the current user, which may not be set or available in all contexts.
Fix a bug where searching for `sakana~` ranked `sakana~_(meme)` beneath
random artist tags containing the word `sakana`. Now, if the search contains
punctuation, we rank exact matches first, even for small tags. Before we
ranked exact matches for small tags lower than inexact matches for large
tags. If the search contains punctuation, it's a strong signal the user
is looking for an exact match.
Fix tags like `short_shorts` or `hunter_x_hunter` being highlighted
incorrectly. Typing `short_sh` would highlight it as SHort_SHorts
instead of as SHORT_SHorts.
Fix an exception when trying to fetch source data for URLs like
https://va.media.tumblr.com/tumblr_pgohk0TjhS1u7mrsl.mp4.
For these URLs it's not possible to use the trick where we try to open
the URL as a HTML page and scrape the post id from the HTML. Instead we
get the raw video if we try to to this.
Lowercase searches clientside before sending them to the server. This is
to improve cache hit rates when users accidentally type in uppercase,
especially on mobile where the first letter often gets capitalized.
Fix usernames not being highlighted when completing @mentions.
This also changes it so the autocomplete results don't include the '@'
in front of the name.
Minor breaking change to the /autocomplete.json API. Autocomplete
results for mentions now have the type `mention` instead of `user`.
Fix /autocomplete.json returning results like this:
[
{
"table": {
"type": "tag-word",
"label": "long hair",
"value": "long_hair",
"category": 0,
"post_count": 2818211,
"antecedent": null
}
},
]
instead of like this:
[
{
"type": "tag-word",
"label": "long hair",
"value": "long_hair",
"category": 0,
"post_count": 2818211,
},
]
Also change it so that optional attributes like `antecedent` aren't
returned if they're null. This could be a minor breaking change if users
rely on these attributes always being present.
Lastly change XML results to look like this:
<autocomplete-service-results type="array">
<autocomplete-service-result>
<type>tag-word</type>
<label>long hair</label>
<value>long_hair</value>
<category type="integer">0</category>
<post-count type="integer">2801117</post-count>
</autocomplete-service-result>
</autocomplete-service-results>
instead of like this:
<objects type="array">
<object>
<type>tag</type>
<label>hair ornament</label>
<value>hair_ornament</value>
<category type="integer">0</category>
<post-count type="integer">883340</post-count>
<antecedent nil="true"/>
</object>
</objects>
Fix a bug where the word `lazy` wasn't highlighted in `don't_say_"lazy"`
when searching for `lazy`. The bug was that punctuation surrounding
words wasn't properly split from the word.
When completing a static metatag, such as rating: or order:,
don't include the metatag name in the autocomplete menu.
For example, when completing `rating:g`, show `general` in the
autocomplete menu, not `rating:general`.
This makes static metatags consistent with other metatags.
Remove the `data-autocomplete-antecedent`, `data-autocomplete-category`,
and `data-autocomplete-post-count` data attributes from the autocomplete
menu. These weren't used by Danbooru itself and they bloat the HTML.
Limit the width of the autocomplete menu to 480px. This is so that
stupidly long isekai titles don't blow out the size of the menu when
searching for common words.
For example, when searching for "look", one of the results is:
no_matter_how_you_look_at_it_it's_your_fault_that_i'm_not_popular! -> watashi_ga_motenai_no_wa_dou_kangaetemo_omaera_ga_warui!
The new autocomplete system tends to return more results, so increase
the number of tags shown from 10 to 20. This is useful so you can do things
like search for 'skirt' during tagging to find all skirt-related tags.
Highlight the matching part of the tag in the autocomplete menu. For
example, if you search "hair", then the word "hair" will be bolded in
every matching tag. This is so users can tell why a particular tag was
matched.
Switch autocomplete to match individual words in the tag, instead of
only matching the start of the tag.
For example, "hair" matches any tag containing the word "hair", not just tags
starting with "hair". "long_hair" matches all tags containing the words "long"
and "hair", which includes "very_long_hair" and "absurdly_long_hair".
Words can be in any order and words can be left out. So "closed_eye" matches
"one_eye_closed". "asuka_langley_souryuu" matches "souryuu_asuka_langley".
This has several advantages:
* You can search characters by first name. For example, "miku" matches "hatsune_miku".
"zelda" matches both "princess_zelda" and "the_legend_of_zelda".
* You can find the right tag even if you get the word order wrong, or forget a word.
For example, "eyes_closed" matches "closed_eyes". "hair_over_eye" matches "hair_over_one_eye".
* You can find more related tags. For example, searching "skirt" shows all tags
containing the word "skirt", not just tags starting with "skirt".
The downside is this may break muscle memory by changing the autocomplete order of
some tags. This is an acceptable trade-off.
You can get the old behavior by writing a "*" at the end of the tag. For
example, searching "skirt*" gives the same results as before.
Add a `words` column to the tags table. This will be used for parsing
tags into words for word-based matching in autocomplete.
For example, "very_long_hair" can be parsed into ["very", "long", "hair"].
The `array_to_tsvector(words)` index is for performing wildcard
searches. It lets us do e.g
SELECT * FROM tags WHERE array_to_tsvector(words) @@ 'hand:* & hold:*'
to find tags containing the words "hand*" and "hold*".
Render the HTML for autocomplete results server-side instead of in
Javascript. This is cleaner than building HTML in Javascript, but it may
hurt caching because the HTTP responses are larger.
Fixes#4698: user autocomplete contains links to /posts
Also fixes a bug where tag counts in the autocomplete menu were different
from tag counts displayed elsewhere because of differences in rounding.
Previously if you typed e.g. "/tr" in autocomplete we would first check
if "/tr" was aliased to another tag before expanding out the abbreviation.
This was for compatibility with legacy shortcut aliases. These aliases
have been removed so this is dead code now.
Show the search navbar when searching for `pool:1234 tag`. Before the
search navbar would always be hidden if the search contained a pool:
metatag, even if it was a multi-tag search.
Fix the pool name not being bolded in the pool navbar when doing a
pool:<name> search. Previously the selected pool was only bolded when
doing a pool:<id> search.
Also change the /autocomplete.json API to no longer strip '-' and '~'
from the start of the tag. This may be a breaking change if third-party
scripts relied on this behavior.
This was an obsolete URL format briefly used by Pixiv around 2019-2020.
There were only ~80 posts with sources using this format. They have been
manually fixed.
Bug: When uploading a direct Pixiv image URL, we ignored it in favor of the
image URL returned by the Pixiv API. This meant if you tried to upload the
original version of a revised image, we would get the revised version instead.
Fix: When given a direct Pixiv image URL, use it as-is if it's a full
image URL. If it's a sample image URL, ignore it in favor of the full image
URL as returned by the API, unless the post is deleted and the API data
is unavailable.
Allow logged out users to call https://danbooru.donmai.us/profile.json.
This allows getting information on default settings and limits for
anonymous users.
May be a breaking API change if users were using the HTTP response code
from /profile.json to check if they were successfully logged in.
Foundation changed their HTML page format and we can no longer scrape
the image URL directly from the page. Instead we have to build it based
on API data.
evazion
evazion