Don't allow gifting Gold or Platinum upgrades to users above Platinum
level. Fixes an exploit where you could demote Builders and above by
gifting them an upgrade.
New rules for user promotions:
* Moderators can no longer promote other users to moderator level. Only
Admins can promote users to Mod level. Mods can only promote up to Builder level.
* Admins can no longer promote other users to Admin level. Only Owners
can promote users to Admin. Admins can only promote up to Mod level.
* Admins can no longer demote themselves or other admins.
These rules are being changed to account for the new Owner user level.
Also change it so that when a user upgrades their account, the promotion
is done by DanbooruBot. This means that the inviter and the mod action
will show DanbooruBot as the promoter instead of the user themselves.
Add a new Owner user level for the site owner. Highly sensitive
operations like manually changing the passwords of other users will be
restricted to the site owner.
* Rename the `#negate` and `#and` methods that we monkey patch into
ActiveRecord::Relation. These methods are now defined in Rails 6.1, but
they shadow our methods and have slightly different behavior.
* Fix a call to `invert`. It no longer accepts an argument.
Display autocorrected tags similar to aliases, with an arrow pointing at
the corrected tag, but with a dotted underline beneath the misspelled
tag to indicate that it's misspelled.
Tune autocorrect to produce fewer false positives. Before we used
trigram similarity. Now we use Levenshtein edit distance with a dynamic
typo threshold. Trigram similarity was able to correct large
transpositions (e.g. `miku_hatsune` -> `hatsune_miku`), but it was bad
at correcting small typos. Levenshtein is good at small typos, but can't
correct large transpositions.
Fix an exception that was thrown when trying to autocomplete saved
search labels (e.g. `search:all`) as an anonymous user. This was a
pre-existing bug.
The previous cache policy was that all autocomplete results were cached
for a fixed 7 days. The new policy is that if autocomplete returns more
than 10 results they're cached for 24 hours, otherwise if it returns
less than 10 results they're cached for 1 hour.
The rationale is that if autocomplete returns a lot of results, then the
top 10 results are relatively stable and unlikely to change, but if it
returns less than 10 results, then the results are unstable and can be
easily changed.
We also change it so that autocomplete calls can be cached publicly.
Public caching means that HTTP requests are cached by Cloudflare. This
will ideally reduce load on the server and reduce latency for end users.
This is only safe for calls that return the same results for all users
(i.e. the results don't depend on the current user), since the cache is
publicly shared by all users. Currently username, favgroup, and saved
search autocomplete results depend on the current user, so they can't be
publicly cached.
Reworks tag autocomplete to work the same way for all users. Previously
autocomplete for Builders worked differently than autocomplete for
regular users.
This is how it works now:
* If the search starts with a slash (/), then do a tag abbreviation
match. For example, `/evth` matches eyebrows_visible_through_hair.
* Otherwise if the search contains a wildcard (*), then just do a simple
wildcard search.
* Otherwise do a tag prefix match against tags and aliases. For example,
`black` matches all tags or aliases beginning with `black`.
* If the tag prefix match returns no results, then do a autocorrect match.
The differences for regular users:
* You can abbreviate tags with a slash (/).
The differences for Builders:
* Now tag abbreviations have to start with a slash (/).
* Autocorrect isn't performed unless a regular search returns no results.
* Results are always sorted by tag count. Before different types of
results (regular tag matches, alias matches, abbreviation matches,
and autocorrect matches) were all mixed together based on a tag
weighting scheme.
This refactors the autocomplete Javascript to use a single dedicated
/autocomplete.json endpoint instead of a bunch of separate endpoints.
This simplifies the autocomplete Javascript by making it so that instead
of calling a different endpoint for each type of query (for users, wiki
pages, pools, artists, etc), then having to parse the results of each
call to get the data we need, we can call a single endpoint that returns
exactly what we need.
This also means we don't have to parse searches clientside in order to
autocomplete metatags. Instead we can just pass the search term to the
server and let it parse the search, which is easy to do serverside.
Finally, this makes autocomplete easier to test, and it makes it easier
to add more sophisticated autocomplete behavior, since most of the logic
lives serverside.
Bug: when aliasing a tag that implied another tag, it was possible for
the alias to fail. Moving the implication could fail because we checked
that the tag category of both tags in the implication was the same, but
we did this before the alias moved the category of the old tag to the
new tag.
When approving or rejecting a BUR, don't edit the OP forum post to add
an EDIT: line stating the request has been approved. Instead just let
the embedded BUR state who it was approved by, and post a reply saying
that the request has been approved.
Move the validation that the tags in an implication must have wiki pages
back into the TagImplication model. Use validation contexts to only run
the validation when the BUR is created, not when the BUR is approved.
Usage:
* `nuke touhou`
* `nuke pool:Disgustingly_Adorable`
Add a command for nuking tags. `nuke A` is a shortcut for `mass update A -> -A`.
This means it also works for pools.
Bug: When validating a BUR, we didn't properly simulate running each
line of the BUR in order, which could cause validation to incorrectly
fail in multi-line BURs where some lines depended on previous lines.
This bug meant you couldn't reverse an alias in a single BUR. The old
alias wasn't removed before validating the new alias, so the BUR would
fail with an alias conflict.
This bug also meant that BURs containing duplicate aliases or
redundant implications weren't caught.
The fix is for BUR validation to actually simulate creating and removing
aliases in sequential order, just as they would be when the BUR is
approved. This is done by running the BUR in a transaction, then
rolling back the transaction at the end. This is hacky but it works.
Remove the error status from aliases and implications. Aliases and
implications normally shouldn't fail because they're validated
beforehand. If they do, just let the delayed job itself record the
failure.
Also disable the delayed job from retrying if the alias/implication
somehow fails.
Remove the pending status from tag aliases and implications.
Previously aliases would be created first in the pending state then
changed to active when the alias was later processed in a delayed job.
This meant that BURs weren't processed completely sequentially; first
all the aliases in a BUR would be created in one go, then later they
would be processed and set to active sequentially.
This was problematic in complex BURs that tried to reverse or swap
around aliases, since new pending aliases could be created before old
conflicting aliases were removed.
Remove the ability to skip secondary validations when creating a BUR.
The only skippable validation that still existed was the requirement
that both tags in an implication must have wiki pages. It's now
mandatory to write wiki pages for tags before you can request an
implication. This doesn't apply to empty tags.
Remove the `processing` state from aliases and implications. This state
was used to mark when an alias or implication had been approved but the
alias or implication was still being processed. Aliases in the
processing state were still considered active, so there was no
functional difference between the active state and the processing state.
This fixes a problem where it was possible for implications to get stuck
in the processing state. This happened when a BUR contained a duplicate
implication. Transitioning from the processing state to the active state
failed in this case because we used `update` instead of `update!`, which
meant validation errors were silently ignored.
Make the tag rename command also move any aliases or implications from
the old tag to the new tag. Previously only the create alias command
moved aliases and implications.
Bug: if a BUR contained a mass update followed by an alias, then the
alias would become active before the mass update, which could cause
the mass update to return incorrect results if both the alias and mass
update touched the same tags.
This happened because all aliases and implications in the BUR were set
to a queued state before the mass update was processed, but putting an
alias in the queued state effectively made it active.
The fix is to remove the queued state. This was only used anyway as a
debugging tool anyway to monitor the state of BURs as they were being
processed.
Decrease the HTTP timeout to 0.5 seconds when fetching popular tags from
Reportbooru. Increase the length of time that popular tags are cached
from 1 minute to 1 hour. This is for the list of popular searches in the
front page sidebar.
When renaming a tag and the new tag has a qualifier, use the pipe trick
to hide the qualifier in wiki links. For example, renaming Fallout to
Fallout_(series) should change wiki links from [[Fallout]] to [[Fallout
(series)|]].
Bug: if `hatsune_miku` is an empty general tag, then tagging a post with
`hatsune_miku_(cosplay)` would fail because `hatsune_miku` wasn't a
character tag. This could cause tagging posts to fail when aliasing a
character tag back to an old name that was used in the past, but was now
an empty general tag.
Fix: only treat the *_(cosplay) tag as being in conflict with the base
tag when the base tag is nonempty.
* Add a `rename A -> B` command for bulk update requests.
* Change mass updates to only retag the posts, not to move saved
searches or blacklists.
A tag rename does the same thing an alias does, except it doesn't
create a permanent alias. More precisely, a tag rename:
* Moves the wiki.
* Moves the artist entry.
* Moves saved searches.
* Moves blacklists.
* Merges the wikis, if both tags have wiki pages.
* Merges the artist entries, if both tags have artist pages.
* Fixes links in wiki pages to point to the new tag.
* Retags the posts.
When aliasing A to B, update any wikis linking to [[A]] to link to [[B]]
instead.
This is a best-effort process based on rough heuristics. There are a few
known problems:
* We don't always know how to capitalize the new tag. We try to mimic
the capitalization of the old tag, such that if the old tag was
capitalized (because it was at the beginning of a sentence), or if
every word in the old link was capitalized (because it's a proper
noun), then the new link will be capitalized in the same way. This can
handle simple general tags and character tags, but will fail for
copyright tags with mixed capitalization. For example, we don't know
that [[jojo_no_kimyou_na_bouken]] should be capitalized as [[JoJo no
Kimyou na Bouken]]. If we don't know how to capitalize the new tag, we
leave the old tag as-is so it can manually be fixed.
* Some aliases might require changing how a tag is pluralized. If we
changed [[rat]] to [[mouse]], then we should change `[[rat]]s` to
[[mice]]. We don't try to deal with this.
* In general, some changes might require entire sentences to be
rewritten to keep the grammar correct. Changing something like
[[skirt lift]] to [[lifting skirt]] could break the grammar of the
sentence. We don't try to deal with this.
* Factor out the code for moving tags from tag aliases to a separate
TagMover class.
* When aliasing two tags that have conflicting wikis, merge the old wiki
into the new one instead of failing with an error. Merge the other names
fields, replace the old wiki body with a message linking to the new
wiki, and mark the old wiki as deleted.
* When aliasing two tags that have conflicting artist entries, merge the
old artist into the new one instead of silently ignore the conflict.
Merge the group name, other names, and urls fields, and mark the old
artist as deleted.
* When two tags have conflicting wikis or artist entries, but the old
wiki or artist entry is deleted, then just ignore the old wiki or
artist and don't try to merge it.
* Fix it so that when saved searches are rewritten, we rewrite negated
searches too.
Remove the following alternate forms of commands:
* aliasing A -> B
* unaliasing A -> B
* implicating A -> B
* implicate A -> B
* unimplicating A -> B
* unimplicate A -> B
* updating A -> B
* change A -> B
The following forms are accepted:
* create alias A -> B
* alias A -> B
* create implication A -> B
* imply A -> B
* remove alias A -> B
* unalias A -> B
* remove implication A -> B
* unimply A -> B
* update A -> B
Only downcase tags in aliases, implications, and category change
commands. Don't downcase mass update commands. Mass updates are
potentially case sensitive (for example: `mass update source:imageboard -> source:Imageboard`).
* Don't raise exceptions when a BUR is invalid. Instead, use Rails
validations to return errors. Fixes invalid BURs potentially raising
exceptions in views. Also makes it so that each error in a BUR is
reported, not just the first one.
* Revalidate the BUR whenever the script is edited, not just when the
BUR is created. Ensures the BUR can't be broken by editing. Fixes a bug
where forum threads could be broken by someone editing a BUR and
breaking the syntax, thereby causing the BUR to raise an unparseable
script error when the forum thread was viewed.
* Validate that removed aliases and implication actually exist.
* Validate that the tag actually exists when changing a tag's category.
* Combine bulk update request processor unit tests with main bulk update
request unit tests.
Prevent age-restricted fanbox posts from raising errors when source data
is fetched. This prevents error messages from being shown to users when
switching to the edit tab on a post.
This will cause uploads of age-restricted posts to fail with an
unrelated error because we either can't find the image url (if we were
given only the html page) or we can't download the image (because we're
not logged in to Fanbox).
