* Pick the largest character or copyright tags by post count. Previously
we picked the tags with the longest names, which was nonsensical.
* Remove tag cateogory logic from config file. We can't avoid hardcoding
some knowledge about tag categories here, so there's no point in trying.
This affects tab titles on post show pages as well as filenames in
downloaded images.
Drop the creator_id and updater_id fields from wiki pages. These fields
had several issues:
* The creator_id field was inconsistent with the wiki_page_versions
table. Apparently during the migration to Danbooru 2 in 2012-2013 the
creator_id field got reset to whoever last updated the wiki at that
point in time.
* Saving a wiki would set the updater_id even when nothing actually
changed. This also caused the updated_at timestamp to get bumped.
Because of this, anything that saved a wiki, including things like
creating aliases or implications, would bump the updater_id and
updated_at even though the wiki didn't actually change. This meant
these fields weren't consistent with the wiki_page_versions history.
Changes:
* Remove `creator_name` field from the /wiki_pages.json API.
* Remove creator name search option from /wiki_pages/search.
Add a dtext_links table for tracking links between wiki pages. This is
to allow for broken link detection and "what links here" searches, among
other uses.
https://danbooru.donmai.us/forum_topics/9127?page=283#forum_post_160508
There was a recent outage that was caused by the read replica
(yukinoshita.donmai.us) being temporarily unavailable. The pg driver in
rails got hardstuck trying to connect to the replica, which brought down
the whole site. The app servers stopped responding and could only be
brought down with SIGKILL. Even try to boot the rails console didn't
work.
We only really used this to calculate tag counts inside Post.fast_count,
which wasn't really beneficial since the read replica is slower than the
main database.
DText is processed in three phases: a preprocessing phase, the regular
parsing phases, and a postprocessing phase.
In the preprocessing phase we extract all the wiki links from all the
dtext messages on the page (more precisely, we do this in forum threads
and on comment pages, because these are the main places with lots of
dtext). This is so we can lookup all the tags and wiki pages in one
query, which is necessary because in the worst case (in certain forum
threads and in certain list_of_* wiki pages) there can be hundreds of
tags per page.
In the postprocessing phase we fixup the html generated by the ragel
parser to add CSS classes to wiki links. We do this in a postprocessing
step because it's easier than doing it in the ragel parser itself.
* Parse the wiki page with the actual dtext parser instead of by hand.
This is so that wiki links inside things like [nodtext] or [code]
blocks are handled properly.
* Only include tags that exist and are nonempty. Don't include links to
dead pages or blank tags.
Replace this common pattern in controllers:
@tags = Tag.search(search_params).paginate(params[:page], :limit => params[:limit], :search_count => params[:search])
with this:
@tags = Tag.paginated_search(params)
`search_count` is used to skip doing a full page count when we're not
doing a search (on the assumption that the number of results will be
high when not constrained by a search). We didn't do this consistently
though. Refactor to do this in every controller.
Previously the page-based (numbered) paginator would always count the
total_pages, even in API calls when it wasn't needed. This could be very
slow in some cases. Refactor so that total_pages isn't calculated unless
it's called.
While we're at it, refactor to condense all the sequential vs. numbered
pagination logic into one module. This incidentally fixes a couple more
bugs:
* "page=b0" returned all pages rather than nothing.
* Bad parameters like "page=blaha123" and "page=a123blah" were accepted.
Disable database timeouts durings daily maintenance. Fixes
`regenerate_post_counts!` timing out. Remove calls to without_timeout
because otherwise it will reenable the timeout when trying to restore
the old timeout (see 97cc873a3f).
* Add search form above table.
* Move thumbnail to left of table when viewing history of single post.
* Remove unrelated links from subnav menu.
* Fix bugs with changed_tags search.
* Add ability to search /post_versions by added tags, removed tags, or
changed tags (added or removed).
* Add 'History' link to the sidebar of the /posts index. This is a
shortcut for a /post_versions search of the current tag.
Don't create a neutral feedback, create a mod action, or dmail the user
after changing a user's name. The name change is already recorded in
/user_name_change_requests, so creating feedbacks and mod actions is
redundant. They also expose private information (when a user deletes
their account, old name changes aren't supposed to be visible any more).
Remove all infrastructure around approving or rejecting user name
changes. Name changes haven't been moderated for several years.
* Remove status, approver_id, change_reason, and rejection_reason fields.
* Remove approve and reject controller actions.
* Automatically fix all tags with incorrect counts during daily
maintenance (previously only tags with negative counts were fixed).
* Log fixed tags to NewRelic.
* Remove the ability to manually fix tag counts with the "Fix" button on
the /tags listing. This is no longer necessary now that tags are
fixed automatically.
DEPRECATION WARNING: Dangerous query method (method whose arguments
are used as raw SQL) called with non-attribute argument(s): "trunc(3
* similarity(name, 'two')) DESC". Non-attribute arguments will be
disallowed in Rails 6.1. This method should not be called with
user-provided values, such as request parameters or model
attributes. Known-safe values can be passed by wrapping them in
Arel.sql(). (called from order_similarity at
/home/admin/src/danbooru/app/models/tag.rb:817)
Fixup for 9c34d5cc3. Default to member level in the database because
setting it in `customize_new_user` clobbered user levels set by factory
bot in tests, which broke the test suite.
`User.find_by_name` used `where_ilike` to do a case-insensitve name
search, but it didn't escape `*` or `\` characters first, so it didn't
handle names containing these characters properly.
* Always display 'Saved searches' link in subnav bar, even if the user
hasn't created any saved searches yet.
* Eliminate use of `has_saved_searches` bitpref on users.
Bug: in the test environment, new users defaulted to the Anonymous level
instead of the Member level. This broke signing up as a new user with
Capybara during system tests.
* Only check for conflicts with existing aliases/implications when
requests are created or approved, not when requests are rejected.
* Use `update!(status: "deleted")` instead of `update(status: "deleted")`
so that if rejecting the request fails we fail immediately instead of
continuing on and updating the forum topic.
* Wrap `reject!` and `TagChangeRequestPruner.reject_expired` in
transactions so that if updating either the request or the forum
fails, they both get rolled back.
Stop maintaining pool category pseudo tags (pool:series, pool:collection)
in pool strings. They're no longer used and the changes to the
`Post#pools` method in dc4d2e54b caused issues with this.
Also allow Members to change the category of large pools again. This was
only restricted because maintaining these pseudotags forced us to update
every post in the pool whenever a pool's category was changed.
Replace the `method_attributes` and `hidden_attributes` methods with
`api_attributes`. `api_attributes` can be used as a class macro:
# include only the given attributes.
api_attributes :id, :created_at, :creator_name, ...
# include all default attributes plus the `creator_name` method.
api_attributes including: [:creator_name]
or as an instance method:
def api_attributes
[:id, :created_at, :creator_name, ...]
end
By default, all attributes are included except for IP addresses and
tsvector columns.
