Fix wiki pages and artists to normalize other names more consistently
and correctly.
For wiki pages, we strip leading / trailing / repeated underscores to
fix user typos, and we normalize to NFKC form to make search more consistent.
For artists, we allow leading / trailing / repeated underscores because
some artist names have these, and we normalize to NFC form because some
artists have weird names that would be lost by NFKC form. This does make
search less consistent.
This refactors the autocomplete Javascript to use a single dedicated
/autocomplete.json endpoint instead of a bunch of separate endpoints.
This simplifies the autocomplete Javascript by making it so that instead
of calling a different endpoint for each type of query (for users, wiki
pages, pools, artists, etc), then having to parse the results of each
call to get the data we need, we can call a single endpoint that returns
exactly what we need.
This also means we don't have to parse searches clientside in order to
autocomplete metatags. Instead we can just pass the search term to the
server and let it parse the search, which is easy to do serverside.
Finally, this makes autocomplete easier to test, and it makes it easier
to add more sophisticated autocomplete behavior, since most of the logic
lives serverside.
Remove the pending status from tag aliases and implications.
Previously aliases would be created first in the pending state then
changed to active when the alias was later processed in a delayed job.
This meant that BURs weren't processed completely sequentially; first
all the aliases in a BUR would be created in one go, then later they
would be processed and set to active sequentially.
This was problematic in complex BURs that tried to reverse or swap
around aliases, since new pending aliases could be created before old
conflicting aliases were removed.
Remove the ability to skip secondary validations when creating a BUR.
The only skippable validation that still existed was the requirement
that both tags in an implication must have wiki pages. It's now
mandatory to write wiki pages for tags before you can request an
implication. This doesn't apply to empty tags.
Bug: if a BUR contained a mass update followed by an alias, then the
alias would become active before the mass update, which could cause
the mass update to return incorrect results if both the alias and mass
update touched the same tags.
This happened because all aliases and implications in the BUR were set
to a queued state before the mass update was processed, but putting an
alias in the queued state effectively made it active.
The fix is to remove the queued state. This was only used anyway as a
debugging tool anyway to monitor the state of BURs as they were being
processed.
Bug: if you created an artist with the name of an existing general tag,
then the gentag would be changed to an artist tag, no matter how big the
gentag was.
Now we only allow creating artist entries for non-artist tags if the tag
is empty.
Ref: https://danbooru.donmai.us/forum_topics/17095
Remove the ability to edit an artist's wiki page directly from the
artist edit page. Instead the artist edit page has a link to open the
wiki edit page if you need to edit the wiki too.
Fixes an error being thrown when renaming an artist with a wiki page.
The problem is that changing the artist's name breaks the artist's
association with the old wiki page. Rails really wants nested
associations to be based on immutable IDs, not on mutable names, so
dealing with this correctly is difficult.
We don't really want to encourage people to create wiki pages for
artists to begin with, since they're usually just used to duplicate
the artist urls. Making it less convenient to edit artist wiki pages is
an intentional change to discourage creating unnecessary artist wikis.
Finally, this fixes an exploit where it was possible to edit locked wiki
pages through the artist edit page.
When doing a tag search, we have to be careful about which user we're
running the search as because the results depend on the current user.
Specifically, things like private favorites, private favorite groups,
post votes, saved searches, and flagger names depend on the user's
permissions, and whether non-safe or deleted posts are filtered out
depend on whether the user has safe mode on or the hide deleted posts
setting enabled.
* Refactor internal searches to explicitly state whether they're
running as the system user (DanbooruBot) or as the current user.
* Explicitly pass in the current user to PostQueryBuilder instead of
implicitly relying on the CurrentUser global.
* Get rid of CurrentUser.admin_mode? (used to ignore the hide deleted
post setting) and CurrentUser.without_safe_mode (used to ignore safe
mode).
* Change the /counts/posts.json endpoint to ignore safe mode and the
hide deleted posts settings when counting posts.
* Fix searches not correctly overriding the hide deleted posts setting
when multiple status: metatags were used (e.g. `status:banned status:active`)
* Fix fast_count not respecting the hide deleted posts setting when the
status:banned metatag was used.
Bug: using the has_tag param caused the paginator to calculate the wrong
page count.
Caused by the join having a DISTINCT clause in the OFFSET/LIMIT query,
but not in the COUNT(*) query.
Rename is_active to is_deleted. This is for better consistency with
other models, and to reduce confusion over what "active" means for
artists. Sometimes users think active is for whether the artist is
actively producing work.
Remove the creator_id field from artists, notes, and pools. The
creator_id wasn't otherwise used and was inconsistent with the
artist/note/pool history in some cases, especially for old artists.
Refactor to use accepts_nested_attributes_for instead of the notes
attribute to facilitate editing wikis on the artist edit page.
This fixes the notes attribute unintentionally showing up in the API.
This also changes it so that renaming an artist entry doesn't
automatically rename the corresponding wiki page. This had bad behavior
when there was a conflict between wiki pages (the wikis would be
silently merged, which usually isn't what you want). It also didn't warn
about wiki links being broken by renames.
* Replace the .category-N CSS classes on tags with .tag-type-N. Before
we were inconsistent about whether tag colors were indicated with
.category-N or .tag-type-N. Now it's always .tag-type-N.
* Fix various places to not use Tag.category_for. Tag.category_for does
one Redis call per tag lookup, which leads to N Redis calls on many
pages. This was inefficient because usually we either already had the
tags from the database, or we could fetch them easily.
Remove various associated fields that were included by default on
certain endpoints. API users can use the only param to include the
full association if they need these fields.
* /artists.json: urls.
* /artist_urls.json: artist.
* /comments.json: creator_name and updater_name.
* /notes.json: creator_name.
* /pools.json: creator_name.
* /posts.json: uploader_name, children_ids, pixiv_ugoira_frame_data.
* /post_appeals.json: is_resolved.
* /post_versions.json: updater_name.
* /uploads.json: uploader_name.
- The only string works much the same as before with its comma separation
-- Nested includes are indicated with square brackets "[ ]"
-- The nested include is the value immediately preceding the square brackets
-- The only string is the comma separated string inside those brackets
- Default includes are split between format types when necessary
-- This prevents unnecessary includes from being added on page load
- Available includes are those items which are allowed to be accessible to the user
-- Some aren't because they are sensitive, such as the creator of a flag
-- Some aren't because the number of associated items is too large
- The amount of times the same model can be included to prevent recursions
-- One exception is the root model may include the same model once
--- e.g. the user model can include the inviter which is also the user model
-- Another exception is if the include is a has_many association
--- e.g. artist urls can include the artist, and then artist urls again
Remove the list of most-used source domains from artist summaries. This
took up a lot of space and usually wasn't very useful. It was also slow.
We had to calculate this on every artist tag search so we could display
it in the Artist tab, even though usually the user didn't open the tab.
Remove support for the `search[sort]` param on certain index pages. This
hasn't been used for years, and it caused the `search[order]=` param to
be added to pagination links even when the order was blank.
Allow all users to view and edit artist entries and wiki pages belonging
to banned artists. There was little need to hide these pages from
Members, it was mainly to appease artists who didn't like us even
linking to their sites.
These restrictions also had multiple flaws:
* Banned artist information was still visible in the API.
* It was still possible to edit banned artists using the API.
* It was still possible for unprivileged users to revert banned
artist entries or wiki pages to previous versions.
* The restrictions were inconsistent: in various places they were
either Member-only, Gold-only, or Builder-only.
The belongs_to_creator macro was used to initialize the creator_id field
to the CurrentUser. This made tests complicated because it meant you had
to create and set the current user every time you wanted to create an
object, when lead to the current user being set over and over again. It
also meant you had to constantly be aware of what the CurrentUser was in
many different contexts, which was often confusing. Setting creators
explicitly simplifies everything greatly.
Validate that artist entries belong to an artist tag. Don't allow
creating artist entries for character/copyright/meta tags, but do allow
entries for general tags (the gentag will be automatically changed to an
artist tag).
