Commit Graph

20 Commits

Author SHA1 Message Date
evazion
4364516f2b posts: change safe mode to only allow rating:g posts. 2022-06-06 00:56:52 -05:00
evazion
05df3a194c posts: add more free metatags.
The following metatags no longer count against the tag search limit:

* is
* id
* date
* age
* filesize
* filetype
* parent
* child
* md5
* width
* height
* duration
* mpixels
* ratio
* score
* upvotes
* downvotes
* favcount
* embedded
* tagcount
* pixiv_id
* pixiv

These are mostly metatags that have to do with properties of the post
itself. Other metatags still count because they involve things like
subqueries or joins, or they're more tag-like in function.
2022-06-05 14:28:44 -05:00
evazion
81bd86d202 posts: add "general" rating; rename "safe" rating to "sensitive".
* Add "general" rating.
* Rename "safe" rating to "sensitive".
* Change safe mode to include both rating:s and rating:g.
* Treat rating:safe as a synonym for rating:sensitive.
* Link "howto:rate" in the post edit form.
2022-05-22 13:38:45 -05:00
evazion
ce18c866d9 Fix #4582: Safebooru should not block "censored" tag
* Remove the default list of blocked tags in safe mode.
* Change it so that tags that are blocked in safe mode are filtered out
  at the database level rather than at the html level.
2022-05-17 02:24:16 -05:00
evazion
1a89071f2d posts: fix error on post index page when search count times out in safe mode
Fix a nil deference error on the post index page. This happened when
performing a single tag search in safe mode and calculating the number
of search results timed out.
2022-05-02 22:03:33 -05:00
evazion
f117049750 users: remove 'hide deleted posts' account setting.
This setting automatically added the `-status:deleted` metatag to all searches. This meant deleted
posts were filtered out at the database level, rather than at the html level. This way searches
wouldn't have less-than-full pages.

The cost was that searches were slower, mainly because post counts weren't cached. Normally when you
search for a tag, we can get the post count from the tags table. If the search is actually like
`touhou -status:deleted`, then we don't know the count and we have to calculate it on demand.

This option is being removed because it did the opposite of what people thought it did. People
thought it made deleted posts visible, when actually it made them more hidden.
2022-05-01 00:47:46 -05:00
evazion
918f32c554 Fix #4461: Improve posts/index page titles. 2022-04-30 01:52:33 -05:00
evazion
219a0959a5 saved searches: fix to use new PostQuery class. 2022-04-17 23:20:22 -05:00
evazion
ca5dd61728 post queries: optimize zero tag and single tag searches.
Avoid going through the full post query parser for empty searches or
simple single-tag searches.
2022-04-17 23:20:22 -05:00
evazion
652db0cd9f post queries: show full error message on error page. 2022-04-17 23:20:22 -05:00
evazion
eca0ab04f7 post queries: raise error on invalid searches.
Raise an error if the search is invalid for one of the following reasons:

* It contains multiple conflicting order: metatags (e.g. `order:score order:favcount` or `ordfav:a ordfav:b`).
* It contains a metatag that can't be used more than once: (e.g. `limit:5 limit:10`, `random:5 random:10`).
* It contains a metatag that can't be negated (e.g. `-order:score`, `-limit:20`, or `-random:20`).
* It contains a metatag that can't be used in an OR clause (e.g. ` touhou or order:score`, `touhou or limit:20`, `touhou or random:20`).
2022-04-17 23:20:22 -05:00
evazion
3e8e33e663 post queries: fix handling of '~' operator.
Fix queries like `(~a ~b) (~c ~d)` being handled like `~a ~b ~c ~d`.
Caused by trimming AND nodes from the tree before rewriting the '~'
operator, which caused `~a` terms to be incorrectly lifted out of
subexpressions.
2022-04-17 23:20:22 -05:00
evazion
af183467b6 post queries: switch to new post search engine.
Switch to the post search engine using the new PostQuery parser. The new
engine fully supports AND, OR, and NOT operators and grouping expressions
with parentheses.

Highlights:

New OR operator:

* `skirt or dress` (same as `~skirt ~dress`)

Tags can be grouped with parentheses:

* `1girl (skirt or dress)`
* `(blonde_hair blue_eyes) or (red_hair green_eyes)`
* `~(blonde_hair blue_eyes) ~(red_hair green_eyes)` (same as above)
* `(pantyhose or thighhighs) (black_legwear or brown_legwear)`
* `(~pantyhose ~thighhighs) (~black_legwear ~brown_legwear)` (same as above)

Metatags can be OR'd together:

* `user:evazion or fav:evazion`
* `~user:evazion ~fav:evazion`

Wildcard tags can combined with either AND or OR:

* `black_* white_*` (find posts with at least one black_* tag AND one white_* tag)
* `black_* or white_*` (find posts with at least one black_* tag OR one white_* tag)
* `~black_* ~white_*` (same as above)

See 4c7cfc73 for more syntax examples.

Fixes #4949: And+or search?
Fixes #5056: Wildcard searches return unexpected results when combined with OR searches
2022-04-17 23:20:22 -05:00
evazion
7fe717506d post queries: add methods for normalizing queries. 2022-04-04 03:56:56 -05:00
evazion
1957cb354e post queries: add #replace_aliases method. 2022-04-04 03:56:54 -05:00
evazion
04551b8154 autocomplete: replace calls to PostQueryBuilder with PostQuery. 2022-03-30 02:12:25 -05:00
evazion
6edff247f2 search: replace calls to PostQueryBuilder#fast_count with PostQuery#fast_count.
Prepare a few more places for the new tag search parser.
2022-03-30 01:37:11 -05:00
evazion
fb0a7851bf BURs: fix mass update A -> B not being allowed.
Fix mass updates of the form `mass update A -> B` not being allowed.

This was originally because after `rename` was introduced, we wanted to
prevent people from using mass updates to move tags. Now, `mass update A -> B`
adds B to all posts tagged A instead of moving A to B. So `mass update A -> B`
should no longer be disallowed.

This also makes it so that it's an error to create a mass update with a
syntax error in the search. Before searches couldn't have syntax errors,
but now with the new query parser it's possible.
2022-03-29 21:31:28 -05:00
evazion
823fa5c6e9 search: switch to new tag search parser in a few places. 2022-03-29 18:21:47 -05:00
evazion
4c7cfc73c6 search: add new tag search parser.
Add a new tag tag search parser that supports full boolean expressions, including `and`,
`or`, and `not` operators and parenthesized subexpressions.

This is only the parser itself, not the code for converting the search into SQL. The new
parser isn't used yet for actual searches. Searches still use the old parser.

Some example syntax:

* `1girl 1boy`
* `1girl and 1boy` (same as `1girl 1boy`)
* `1girl or 1boy`
* `~1girl ~1boy` (same as `1girl or 1boy`)
* `1girl and ((blonde_hair blue_eyes) or (red_hair green_eyes))`
* `1girl ~(blonde_hair blue_eyes) ~(red_hair green_eyes)` (same as above)
* `1girl -(blonde_hair blue_eyes)`
* `*_hair *_eyes`
* `*_hair or *_eyes`
* `user:evazion or fav:evazion`
* `~user:evazion ~fav:evazion`

Rules:

AND is implicit between terms, but may be written explicitly:

* `a b c` is `a and b and c`

AND has higher precedence (binds tighter) than OR:

* `a or b and c or d` is `a or (b and c) or d`
* `a or b c or d e` is `a or (b and c) or (d and e)`

All `~` operators in the same subexpression are combined into a single OR:

* `a b ~c ~d` is `a b (c or d)`
* `~a ~b and ~c ~d` is `(a or b) (c or d)`
* `(~a ~b) (~c ~d)` is `(a or b) (c or d)`

A single `~` operator in a subexpression by itself is ignored:

* `a ~b` is `a b`
* `~a and ~b` is `a and b`, which is `a b`
* `(~a) ~b` is `a ~b`, which is `a b`

The parser is written as a backtracking recursive descent parser built on top of
StringScanner and a handful of parser combinators. The parser generates an AST, which is
then simplified using Boolean algebra to remove redundant nodes and to convert the
expression to conjunctive normal form (that is, a product of sums, or an AND of ORs).
2022-03-29 18:21:46 -05:00