This refactors Pundit policies to only rely on the current user, not on
the current user and the current HTTP request. In retrospect, it was a
bad idea to include the current request in the Pundit context. It bleeds
out everywhere and there are many contexts (in tests and models) where
we only have the current user, not the current request. The previous
commit got rid of the only two places where we used it.
This option was originally added in issue #1747. But only ~350 users
ever disabled autocomplete, only ~120 of these were seen in the last
year, and only 9 new users who signed up in the last year disabled it.
Users wishing to disable autocomplete can use this CSS:
.ui-autocomplete { display: none !important: }
or this Javascript:
$("[data-autocomplete]").autocomplete("disable");
Specify the default settings for new users inside the User model instead
of inside the database. This makes it easier to change defaults, and it
makes the code clearer.
Remove the enable_post_navigation option. This option was originally
added to disable the next/prev post navbar beneath posts. It was later
repurposed to disable keyboard shortcuts.
Users who don't want keyboard shortcuts are advised to not press random
buttons on the keyboard like a caveman.
Only ~1200 users disabled this option and only ~600 were seen in the
last year.
Remove the enable_sequential_post_navigation option. This option was
used to disable the next/previous post navbar below posts.
This option was originally added in issue #674 because of people
complaining about the navbar when it was originally added. Also there
were complaints about URLs being uglier because of search params in the
URL (e.g. /posts/1234?q=touhou). There were also various minor bugs with
it at the time, such as keyboard shortcuts not working correctly, or the
page not remembering your search after a tag edit.
These complaints are irrelevant nowadays because a) people are used to
the navbar by now (and more often complain about it *not* being there
for order:score searches), b) post URLs always contain the search now,
this option hasn't disabled that for years, and c) the initial bugs with
it were fixed years ago.
Only ~1000 users disabled this option and only ~600 were seen in the last year.
Users still wishing to hide the search navbar can use custom CSS instead.
Include the "q" URL param (as in `/posts/1234?q=<search>`) on thumbnails
for all users. Previously it was only for logged in users. This lets the
next/previous post navbar beneath posts be used by logged out users.
Refactor the post preview html to use the ViewComponent framework. This
lets us encapsulate all the HTML, CSS, and helper methods for a UI
component in a single place.
See https://viewcomponent.org.
Adjust permissions on user events to let Moderators only see login,
logout, and user creation events, not other types of events (password
changes, etc). Admins can see everything. These other types of events
are meant for account security purposes and aren't very relevant for
sockpuppet detection purposes.
When a user does a tag search, log a few more things, including the normalized
search string, the number of tags in the search string, and the number of results.
Refactor page limits to a) be explicitly listed in the User class (not
hidden away in the Danbooru config) and b) explicitly depend on the
CurrentUser (not implicitly by way of Danbooru.config.max_numbered_pages).
Optimize autocomplete to ignore various types of bogus input that will
never match anything. It turns out it's not uncommon for people to do
things like paste random URLs into autocomplete, or hold down keys, or
enter long strings of gibberish text (sometimes in other languages).
Some things, like autocorrect and slash abbreviations, become
pathologically slow when fed certain types of bad input.
Autocomplete will abort and return nothing in the following situations:
* Searching for URLs (tags that start with http:// or https://).
* Overly long tags (strings longer than the 170 char tag name limit).
* Slash abbreviations longer than 10 chars (e.g. typing `/qwoijqoiqogirqewgoi`).
* Slash abbreviations that aren't alphanumeric (e.g. typing `/////////`).
* Autocorrect input that contains too much punctuation and not enough actual letters.
Optimize searches for non-English phrases in autocomplete. These
searches were pretty slow, and could sometimes cause sitewide lag spikes
when users typed long strings of non-English text into the search box
and caused an unintentional DoS.
The trick is to use an `array_to_tsvector(other_names) USING gin` index
on other_names. This supports fast string prefix matching against all
elements of the array. The downside is that it doesn't allow infix or
suffix matches, so we can't support wildcards in general. Wildcards
didn't quite work anyway, since artist and wiki other names can contain
literal '*' characters.
Fix the `normalize` and `array_attribute` macros conflicting with each
other on the WikiPage model. This meant code like
`wiki_page.other_names = "foo bar"` didn't work. Both macros defined a
`other_names=` method, but one method overrode the other.
The fix is to use anonymous modules and prepend so we can chain method
calls with super.
Fix wiki pages and artists to normalize other names more consistently
and correctly.
For wiki pages, we strip leading / trailing / repeated underscores to
fix user typos, and we normalize to NFKC form to make search more consistent.
For artists, we allow leading / trailing / repeated underscores because
some artist names have these, and we normalize to NFC form because some
artists have weird names that would be lost by NFKC form. This does make
search less consistent.
Fix saved searches to remove additional invalid characters from labels:
* Remove repeated spaces or underscores.
* Remove leading and trailing spaces or underscores.
* Normalize Unicode characters to NFC form.
Also add a fix script to renormalize labels in old saved searches. A few
problems with existing searches:
* Some saved searches somehow had labels containing NULL elements.
* Some had leading or trailing underscores.
* Some had repeated underscores.
* Some had non-English characters in uppercase.
Add tracking of certain important user actions. These events include:
* Logins
* Logouts
* Failed login attempts
* Account creations
* Account deletions
* Password reset requests
* Password changes
* Email address changes
This is similar to the mod actions log, except for account activity
related to a single user.
The information tracked includes the user, the event type (login,
logout, etc), the timestamp, the user's IP address, IP geolocation
information, the user's browser user agent, and the user's session ID
from their session cookie. This information is visible to mods only.
This is done with three models. The UserEvent model tracks the event
type (login, logout, password change, etc) and the user. The UserEvent
is tied to a UserSession, which contains the user's IP address and
browser metadata. Finally, the IpGeolocation model contains the
geolocation information for IPs, including the city, country, ISP, and
whether the IP is a proxy.
This tracking will be used for a few purposes:
* Letting users view their account history, to detect things like logins
from unrecognized IPs, failed logins attempts, password changes, etc.
* Rate limiting failed login attempts.
* Detecting sockpuppet accounts using their login history.
* Detecting unauthorized account sharing.
Add a Restricted user level. Restricted users are level 10, below
Members. New users start out as Restricted if they sign up from a proxy
or an IP recently used by another user.
Restricted users can't update or edit any public content on the site
until they verify their email address, at which point they're promoted
to Member. Restricted users are only allowed to do personal actions
like keep favorites, keep favgroups and saved searches, mark dmails as
read or deleted, or mark forum posts as read.
The restricted state already existed before, the only change here is
that now it's an actual user level instead of a hidden state. Before it
was based on two hidden flags on the user, the `requires_verification`
flag (set when a user signs up from a proxy, etc), and the `is_verified`
flag (set after the user verifies their email). Making it a user level
means that now the Restricted status will be shown publicly.
Introducing a new level below Member means that we have to change every
`is_member?` check to `!is_anonymous` for every place where we used
`is_member?` to check that the current user is logged in.
Remove the WikiPageVersion#artist and Post#updater associations. Neither
of these existed and they caused problems with searching includable
associations in the API.
In Danbooru 1, aliases (and implications) had a `reason` field where
either the admin or the alias requester gave a reason for the alias.
This field was removed from the code and the database schema, but it
still existed in the production database. This adds the field back, so
that the dev schema is consistent with the production schema, and so
that legacy reasons can be viewed on site again.
* Add back legacy tag_aliases.reason and tag_implications.reason field.
* Make /tag_aliases and /tag_implications show legacy reasons.
* Add the reason field to the search form.
Purge cached thumbnails from Cloudflare when a post is regenerated.
This is necessary because regenerating a post may change the thumbnail,
and if we don't purge the cache from Cloudflare then users will still
see the old thumbnail.
We have do this before updating IQDB because if we don't, IQDB will see
the old cached thumbnail and index the wrong image. This may be racy
because the thumbnail might not be completely purged from Cloudflare
before it's downloaded by IQDB.
Regenerate posts asynchronously using a delayed job.
Regenerating a post can be slow because it involves downloading the
original file, regenerating the thumbnails, and redistributing the new
thumbnails back to the image servers. It's better to run this in the
background, especially if a user is trying to regenerate posts in bulk.
The downside is there's no notification to the user when the regeneration
is complete. You have to check the modactions log to see when it's finished.
