Most of the new settings aren't relevant to us. We do have to fix some
tests to work around a Rails bug. `assert_enqueued_email_with` uses the
wrong queue, so we have to specify it explicitly. This is fixed in Rails
HEAD but not yet released.
* Introduce an abstraction for normalizing attributes. Very loosely
modeled after https://github.com/fnando/normalize_attributes.
* Normalize wiki bodies to Unicode NFC form.
* Normalize Unicode space characters in wiki bodies (strip zero width
spaces, normalize line endings to CRLF, normalize Unicode spaces to
ASCII spaces).
* Trim spaces from the start and end of wiki page bodies. This may cause
wiki page diffs to show spaces being removed even when the user didn't
explicitly remove the spaces themselves.
The `source` command is a bash-ism and doesn't work in a strictly POSIX
shell like dash, which is the /bin/sh on Debian/Ubuntu. Use `.` instead.
https://en.wikipedia.org/wiki/Dot_(command)
Don't allow wiki pages to have invalid names.
This incidentally means that you can't create wiki pages for pools. For
example, you can't create a wiki titled "pool:almost_heart-warming".
This is not a valid tag name, so it's not a valid wiki name either. This
was done in a handful of cases to translate Pixiv tags to Danbooru pools
(see: <https://danbooru.donmai.us/wiki_page_versions?search[title_like]=pool:*>)
Also fix it so that titles are normalized before validation, not before save.
Rename the following post replacement attributes:
* file_size_was -> old_file_size
* file_ext_was -> old_file_ext
* image_width_was -> old_image_width
* image_height_was -> old_image_height
* md5_was -> old_md5
In Rails 6.1, having attributes named `file_size` and `file_size_was` on
the same model breaks things because it conflicts with Rails' dirty
attribute tracking.
* Swap out activerecord-hierarchical_query gem for some guy's patched
version because the mainline version is incompatible with 6.1.
* Disable meta_request gem because it hangs puma on startup on 6.1.
Expand the tag abbreviation system introduced in b0be8ae45 so that it
works in searches and when tagging posts, not just in autocomplete.
For example, you can tag a post with /evth and it will add the tag
eyebrows_visible_through_hair. You can search for /evth and it will
search for the tag eyebrows_visible_through_hair.
Some more examples:
* /ops is short for one-piece_swimsuit
* /hooe is short for hair_over_one_eye
* /saol is short for standing_on_one_leg
* /tlozbotw is short for the_legend_of_zelda:_breath_of_the_wild
If two tags have the same abbreviation, then the larger tag takes
precedence. For example, /be is short for blue_eyes, not brown_eyes,
because blue_eyes is the bigger tag.
If there is an existing shortcut alias that conflicts with the
abbreviation, then the alias take precedence. For example, /sh is short
for suzumiya_haruhi, not short_hair, because there's an old alias for
/sh -> suzumiya_haruhi.
Limit tag length to 170 chars. 170 chars was chosen because it's
longer than the longest active tag on Danbooru.
Tag length is limited because in some contexts we can't deal with
excessively long tags. Tag autocorrect for example uses the levenshtein
function in Postgres, which can't handle strings more than 255 chars long.
Disallow tags from starting with a '/' character. This is so that tag
abbreviations in autocomplete, which start with a '/', don't conflict
with regular tags.
Also disallow some other punctuation characters: `%{})]. Currently no
tags start with these characters. This is to reserve other special
characters in case we need them for other future syntax extensions.
Add the following search operators:
* /tags?search[post_count_eq]=42
* /tags?search[post_count_not_eq]=42
* /tags?search[post_count_gt]=42
* /tags?search[post_count_gteq]=42
* /tags?search[post_count_lt]=42
* /tags?search[post_count_lteq]=42
Works for all numeric attributes on all index actions.
This is the scenario:
* You type something in autocomplete, let's say 'touhou'.
* Autocomplete calls /autocomplete?search[query]=touhou&search[type]=tag_query
* The endpoint returns JSON, because the autocomplete call sets an
`Accept: application/json` header requesting JSON.
* Visit /autocomplete?search[query]=touhou&search[type]=tag_query in your browser.
* Notice that the cached JSON response is incorrectly returned, not an
HTML response like the browser requested.
The problem is that the response type is chosen based on the Accept
header, but the response didn't set the `Vary: Accept` header, so the
browser doesn't know the response type can vary and so it incorrectly
returns the cached response.
This issue is partially fixed by Rails 6.1 ([1]), which properly sets the
`Vary: Accept` header when the response depends on the Accept header.
However, the next issue is that Cloudflare doesn't respect the Vary
header at all ([2], [3]). Therefore we can't use the Accept header to
pick the format, instead we have explicitly specify the format with
/autocomplete.json.
This is clearer and better for caching anyway. Using the `Vary: Accept`
header reduces the cache hit rate, because the exact format of the
Accept header varies across browsers, which fragments the cache.
Whew.
[1] https://github.com/rails/rails/pull.36213
[2] https://community.cloudflare.com/t/cloudflare-cdn-cache-to-support-http-vary-header/160802
[3] https://support.cloudflare.com/hc/en-us/articles/115003206852
[4] https://www.smashingmagazine.com/2017/11/understanding-vary-header/
Fix session cookies being sent in publicly cached /autocomplete.json
responses. We can't set any cookies in a response that is being publicly
cached, otherwise they'll be visible to other users. If a user's session
cookies were to be cached, then it would allow their account to be stolen.
In reality, well-behaved caches like Cloudflare will simply refuse to
cache responses that contain cookies to avoid this scenario.
https://support.cloudflare.com/hc/en-us/articles/200172516-Understanding-Cloudflare-s-CDN:
BYPASS is returned when enabling Origin Cache-Control. Cloudflare also
sets BYPASS when your origin web server sends cookies in the response
header.
Fix an exception that could occur when typing "/" by itself in
autocomplete and a regular tag starting with "/" was returned. This
caused an exception in `r[:antecedent].length` because the tag's
antecedent was nil.
* Remove the `source` and `weight` html data attributes (no longer used).
* Make the `type` html data attribute properly indicate the completion
type. Valid types: `tag`, `tag-alias`, `tag-abbreviation`,
`tag-autocorrect`, `tag-other-name`.
Allowing typing Japanese tags in autocomplete. For example, typing 東方
in autocomplete will be completed to the touhou tag. Typing ぶくぶ will
complete to the bkub tag.
This works using wiki page and artist other names. Effectively, any name
listed as an other name in a wiki or artist page will be treated like an
alias for autocomplete purposes. This is limited to non-ASCII other names,
to prevent English other names from interfering with regular tag searches.
* Set Max-Age= flag instead of Expires= flag.
* Set Secure flag when using HTTPS.
* Extend default cookie lifetime from 1 year to 20 years.
* Remove "session" expiration option (unused).
* Remove max cookie size check.
The cookie size check was previously added in #2518 to deal with running
out of space due to tag scripts and blacklists. This should no longer
happen since we no longer use cookies for these things. Remove the
warning because it should never happen, we can't fix it if it does, and
the user probably won't know how to fix it either.
* Let Mods and Admins see the email addresses of users below their level.
* Let users see their own email address on their profile.
* Let users verify or edit their email address from their profile.
This is to make catching sockpuppets easier, and to make it easier for
users to fix their email.
Don't allow admins to bypass promotion restrictions by manually updating
user levels with a `PUT /users/:id` API call. Level changes have to go
through the /admin/users/:id/edit page.
Don't allow gifting Gold or Platinum upgrades to users above Platinum
level. Fixes an exploit where you could demote Builders and above by
gifting them an upgrade.
* Test that the user upgrade process integrates with Stripe correctly.
* Replace a deprecated `card` param with `source` in `Stripe::Charge.create`.
* Rescue Stripe::StripeError instead of Stripe::CardError so that we
handle failures outside of card failures, such as network errors.
New rules for user promotions:
* Moderators can no longer promote other users to moderator level. Only
Admins can promote users to Mod level. Mods can only promote up to Builder level.
* Admins can no longer promote other users to Admin level. Only Owners
can promote users to Admin. Admins can only promote up to Mod level.
* Admins can no longer demote themselves or other admins.
These rules are being changed to account for the new Owner user level.
Also change it so that when a user upgrades their account, the promotion
is done by DanbooruBot. This means that the inviter and the mod action
will show DanbooruBot as the promoter instead of the user themselves.
