Fix searches like this:
https://danbooru.donmai.us/artist_urls?search[url_not_regex]=https://www/.artstation/.com/[a-zA-Z0-9-.]
returning a HTTP 500 error with a blank page.
Here the problem is that Postgres raises an error because `[a-zA-Z0-9-.]`
is an invalid character class (it should be `[a-ZA-Z0-9.-]`). This error
happens on the error page itself, when rendering the <link rel="next"> /
<link rel="prev"> links in the <head>, so it causes the error page to fail.
Fix mass updates of the form `mass update A -> B` not being allowed.
This was originally because after `rename` was introduced, we wanted to
prevent people from using mass updates to move tags. Now, `mass update A -> B`
adds B to all posts tagged A instead of moving A to B. So `mass update A -> B`
should no longer be disallowed.
This also makes it so that it's an error to create a mass update with a
syntax error in the search. Before searches couldn't have syntax errors,
but now with the new query parser it's possible.
Fix the Tags field in the BUR search form not finding all BURs
mentioning that tag. Specifically, tags that were part of a mass update,
and that were prefixed with `~` or `-` (OR tags and NOT tags), weren't
indexed as tags affected by the BUR.
This requires re-running script/fixes/064_initialize_bulk_update_request_tags.rb
to fix old BURs.
Add a new tag tag search parser that supports full boolean expressions, including `and`,
`or`, and `not` operators and parenthesized subexpressions.
This is only the parser itself, not the code for converting the search into SQL. The new
parser isn't used yet for actual searches. Searches still use the old parser.
Some example syntax:
* `1girl 1boy`
* `1girl and 1boy` (same as `1girl 1boy`)
* `1girl or 1boy`
* `~1girl ~1boy` (same as `1girl or 1boy`)
* `1girl and ((blonde_hair blue_eyes) or (red_hair green_eyes))`
* `1girl ~(blonde_hair blue_eyes) ~(red_hair green_eyes)` (same as above)
* `1girl -(blonde_hair blue_eyes)`
* `*_hair *_eyes`
* `*_hair or *_eyes`
* `user:evazion or fav:evazion`
* `~user:evazion ~fav:evazion`
Rules:
AND is implicit between terms, but may be written explicitly:
* `a b c` is `a and b and c`
AND has higher precedence (binds tighter) than OR:
* `a or b and c or d` is `a or (b and c) or d`
* `a or b c or d e` is `a or (b and c) or (d and e)`
All `~` operators in the same subexpression are combined into a single OR:
* `a b ~c ~d` is `a b (c or d)`
* `~a ~b and ~c ~d` is `(a or b) (c or d)`
* `(~a ~b) (~c ~d)` is `(a or b) (c or d)`
A single `~` operator in a subexpression by itself is ignored:
* `a ~b` is `a b`
* `~a and ~b` is `a and b`, which is `a b`
* `(~a) ~b` is `a ~b`, which is `a b`
The parser is written as a backtracking recursive descent parser built on top of
StringScanner and a handful of parser combinators. The parser generates an AST, which is
then simplified using Boolean algebra to remove redundant nodes and to convert the
expression to conjunctive normal form (that is, a product of sums, or an AND of ORs).
Change mass updates to not automatically remove the left-hand side tags
from the post. This won't work with full boolean searches in the future
and already doesn't work with complex searches involving metatags or OR-tags.
Fix the page_url method not to return URLs like this:
https://seiga.nicovideo.jp/image/source/8017978 (page: https://seiga.nicovideo.jp/watch/mg310193)
These are direct image URLs, not page URLs. It's not generally possible
to get to the page URL from an image URL like this.
This fixes it so that we don't incorrectly set the source of NicoSeiga
uploads to the image URL.
Refactor source strategies to remove the `canonical_url` method.
`canonical_url` returned the URL that should be used as the source of
the post after upload. Now we simply use `Source::URL#page_url` to
determine the source after upload. If the source is an image URL that is
convertible to a page URL, then the image URL is used as the source. If
the source is an image URL that is not convertible to a page URL, then
the page URL is used as the source.
This simplifies source strategies so that all they have to care about is
implementing the `Source::URL#page_url` and `Sources::Strategies#page_url`
methods, and the preferred source will be chosen for posts automatically.
Add partial support for fetching videos from ArtStation posts that
contain videos. Most of this code is disabled for now because actually
downloading these videos requires bypassing a Cloudflare captcha.
Fix regression in 1a4efbda3. Locking the comment before validation
failed when the comment had unsaved changes, as is the case when
clearing reports from a comment before it is deleted.
Fix a PublicSuffix::DomainNotAllowed exception raised with viewing or editing a post
with a source like `Blog.`.
This happened when parsing the post's source. `Danbooru::URL.parse("Blog.")` would
heuristically parse the source into `http://blog`. Calling any methods related to the
URL's hostname or domain would lead to calling `PublicSuffix.parse("blog")`, which
would fail with PublicSuffix::DomainNotAllowed.
Fix regression in 1ad0e8688. Caused by `relation.order_values` returning
an array of Arel nodes instead of an array of strings when doing a
`random:1` search.
Fix certain searches timing out when using sequential navigation (page=b1234).
The problem was that the so-called "small search optimization" (AKA: force Postgres
to use the tag index for small searches instead a sequential scan) wasn't triggering
because the ORDER BY clause for sequential navigation was `posts.id desc`, and we
were only checking for `posts.id DESC`.
Rationale:
* The spoilers tag is the most frequently removed tag from the default blacklist.
* It's frustrating for regular users to have posts randomly hidden because of trivial
spoilers from a series they don't care about.
* The spoilers tag is used way too liberally for things that aren't considered
spoilers on other sites.
* If you're looking up fanart on the internet, you should expect to see a certain
level of spoilers.
* The tag is used very inconsistently, with some characters like Nia_(blade)_(xenoblade)
getting the spoilers tag half the time and the rest of the time not.
