Fix various cases where db/structure.sql either didn't match the
production db or didn't match the db structure produced when running
migrations from a fresh install.
Bug: running migrations on a fresh database failed when we got to a
migration adding a posts.uploader_ip_addr index on 2019-11-11. It failed
because the index already existed.
The index already existed because we used to have it at one point, until
it was removed by another migration on 2015-01-20. This migration didn't
correctly remove the index though, because it tried to remove a
posts.source index first, which failed because this index didn't exist
(it probably existed only in production). This error was swallowed,
causing the migration to silently skip removing the posts.uploader_ip_addr
index, which eventually blew up when we tried to add the index again
later on.
Add `Danbooru.Autocomplete.MAX_RESULTS` param that controls the number
of results returned by autocomplete.
This does not work with tag autocomplete for Builders. Builder
autocomplete still needs to be refactored and unified with Member
autocomplete.
Also fix a bug in the /saved_searches/labels endpoint where the limit
param wasn't respected.
Add a new IP address search page at /ip_addresses. Replaces the old
search page at /moderator/ip_addrs.
On user profile pages, show the user's last known IP to mods. Also add
search links for finding other IPs or accounts associated with the user.
IP address search uses a big UNION ALL statement to merge IP addresses
across various tables into a single view. This makes searching easier,
but is known to timeout in certain cases.
Fixes#4207 (the new IP search page supports searching by subnet).
Add these endpoints:
* /note_versions/1234
* /artist_versions/1234
* /artist_commentary_versions/1234
This is so the /ip_addresses listing can link to these endpoints.
This gem uses a native extension that requires a C++ compiler to build.
Removing this gem removes the need to have a C++ toolchain to install Danbooru.
Bug: if a post had a large number of child posts, the child post preview
box would expand beyond the width of the screen.
Setting `min-width: 0` makes the flexbox column shrink instead of
expand.
Bug: sending a dmail containing a wiki link (ex: [[tagme]]) failed when
the recipient had email notifications turned on.
Cause: wiki links inside email notifications use absolute urls, which
the dtext postprocessor didn't parse correctly.
Bug: links like these returned 404s:
* https://danbooru.donmai.us/wiki_pages/...
* https://danbooru.donmai.us/wiki_pages/.hack//
* https://danbooru.donmai.us/wiki_pages/ssss.gridman
Cause: by default, Rails uses dots in route segments to separate the id
from the format. For example, in /wiki_pages/ssss.gridman, the id is
parsed as "ssss" and the format is "gridman" (as if "gridman" were a
format like "json" or "xml").
We work around this by specifying the regex for the id param manually.
The trick here is to use a non-greedy match-all combined with a positive
lookahead to detect the extension but not include it in the match.
Sometimes uploads fail with this error:
ActiveRecord::QueryCanceled - PG::QueryCanceled: ERROR: canceling
statement due to statement timeout CONTEXT: while updating tuple
(2808,110) in relation "tags"
This is caused by two uploads trying to update tag counts for the same
tags at the same time. The UPDATE statement doesn't guarantee the tags
will be updated in any particular order, which can cause deadlocks when
two UPDATEs try to update the same tags in a conflicting order. For
example, if one process tries to increment '1girl' and 'solo' (in that
order), while at the same time another process tries to increment 'solo'
and '1girl' (in that order), the result will be a deadlock.
The solution is to explicitly lock the tags in a consistent order before
the update statement.
* Use more compact timestamps in post tooltips ("39 minutes ago" -> "39m ago")
* Move timestamps to the right (after favcount + score).
* Switch favorite icon from star to heart.
* Redirect the show_or_new action to either the show page or the new
page. Don't use show_or_new to render nonexistent wikis; do that in the
regular show action instead.
* Make the show action return 404 for nonexistent wikis.
* Fix inconsistencies in how wiki pages were linked.
* Link directly to the wiki instead of to a title search that is expected
to redirect to the wiki.
Change wiki page search to redirect to exact matches only when using the
quick search bar. Fixes searches sometimes unexpectedly redirecting when
doing a regular (non-quick) search that happens to return a single result.
Also remove the logic that tries to expand the search when no results
are found. This will eventually be replaced with a smarter "did you mean?"
search.
Fix an exception that is triggered by mousing out of a thumbnail when
tooltips are disabled. Caused by trying to access `qtip.cache` when
`qtip` is null.
* Move search form to index page instead of a separate page.
* Merge creator + timestamp into one column.
* Add category column.
* Fix N+1 query issue.
Move the parsing for the [bur:<id>], [ta:<id>], [ti:<id>] pseudo tags to
the main parser in `DText.format_text`. This fixes a bug where wiki
links inside bulk update requests on the forum weren't properly
colorized because the text of the BUR was embedded after we scanned for
wiki links, not before.
This also ensures that tags inside bulk update requests will be recorded
in the dtext_links table, meaning that forum posts can be properly
searched by tags.
This incidentally means that these request pseudo tags can now be used
outside the forum.
Normally we skip doing page counts on index pages when there aren't any
search filters. This is on the assumption that most index pages have
more than 1000 pages (20,000 results), so it's not worth counting them
exactly. This isn't always true, so here we turn on full counts on
certain index pages known to be small.
* Add post/comment/forum vote counts to user profiles.
* Show uploaders on post votes index and allow searching by uploader.
* Show forum posters on forum votes index and allow searching by poster.
* Add unvote link to forum votes index.
* Only show unvote links to current user.
* Let gold users use upvote:self, downvote:self metatags to search for
their own votes.
* Don't let mods use upvote:<user>, downvote:<user> metatags to see
votes by other users. Only let admins see other users' votes.
* Add vote count to profile page.
* Move forum post vote tests from test/controllers to test/functional.
* Fix forum post vote tests to work with new routes.
* Fix obsolete wiki page tests dealing with updater_id.
* Remove the single alias and implication request forms. From now
on, bulk update requests are the only way to request aliases or
implications.
* Remove the forum topic ID field from the bulk update request form.
Instead, to attach a BUR to an existing topic you go to the topic then
you click "Request alias/implication" at the top of the page.
* Update the bulk update request form to give better examples for the
script format and to explain the difference between aliases and
implications.
Instead of sending IQDB the image url and letting it download the file,
download the file on Danbooru's end and send IQDB the downloaded file.
This fixes several issues:
* Some sites need the referer header set to avoid hotlink protection
when downloading the file. Danbooru knows how to deal with this but
IQDB doesn't.
* We need to enforce certain restrictions when downloading files,
including setting max filesize limits, setting max timeouts, and not
allowing downloads from forbidden IPs (to avoid SSRF attacks).
Danbooru knows how to handle these things but IQDB doesn't.
Fix the moebooru strategy to fallback to returning the image url if we
can't find the preview url. Fixes iqdb lookups failing in some cases
because the strategy didn't return a valid url for preview_url.
* Pick the largest character or copyright tags by post count. Previously
we picked the tags with the longest names, which was nonsensical.
* Remove tag cateogory logic from config file. We can't avoid hardcoding
some knowledge about tag categories here, so there's no point in trying.
This affects tab titles on post show pages as well as filenames in
downloaded images.
Drop the creator_id and updater_id fields from wiki pages. These fields
had several issues:
* The creator_id field was inconsistent with the wiki_page_versions
table. Apparently during the migration to Danbooru 2 in 2012-2013 the
creator_id field got reset to whoever last updated the wiki at that
point in time.
* Saving a wiki would set the updater_id even when nothing actually
changed. This also caused the updated_at timestamp to get bumped.
Because of this, anything that saved a wiki, including things like
creating aliases or implications, would bump the updater_id and
updated_at even though the wiki didn't actually change. This meant
these fields weren't consistent with the wiki_page_versions history.
Changes:
* Remove `creator_name` field from the /wiki_pages.json API.
* Remove creator name search option from /wiki_pages/search.
Add a dtext_links table for tracking links between wiki pages. This is
to allow for broken link detection and "what links here" searches, among
other uses.
DText was changed so that the .dtext-external-link class is now applied to all external links.
Previously it applied only to named links (ex: "google":[http://www.google.com]), not bare
links (ex: http://www.google.com).
https://fontawesome.com/how-to-use/on-the-web/other-topics/performance
The default version of Font Awesome uses Javascript to replace <i> tags
with dynamically generated SVG elements. This adds a lot of weight to
the Javascript bundle (at least 1MB+), even when using subsetting to
load only the icons we actually use. The web font version is less
featureful than the JS version, but much lighter weight.
* Fix the IQDB lookup to handle error messages returned by IQDB.
* Fix `undefined method `>=' for nil:NilClass` error when trying to upload
ugoiras. Using the preview image for IQDB lookups should be faster and
give the same results as the full image, plus it should work with things
like ugoiras and videos that IQDB can't handle.
Also add an image_url param so that API users can continue using the
full image if needed.
https://danbooru.donmai.us/forum_topics/9127?page=283#forum_post_160508
There was a recent outage that was caused by the read replica
(yukinoshita.donmai.us) being temporarily unavailable. The pg driver in
rails got hardstuck trying to connect to the replica, which brought down
the whole site. The app servers stopped responding and could only be
brought down with SIGKILL. Even try to boot the rails console didn't
work.
We only really used this to calculate tag counts inside Post.fast_count,
which wasn't really beneficial since the read replica is slower than the
main database.
evazion
evazion
