Commit Graph

276 Commits

Author SHA1 Message Date
evazion
c452aa6c78 search: add exclusive range syntax (id:5...10).
Support exclusive ranges for numeric metatags. For example, `id:5...10`
is equivalent to `id:>=5 id:<10`. Useful for splitting searches into id
ranges without the endpoints overlapping: id:100...200, id:200...300,
id:300...400.
2020-04-20 04:13:01 -05:00
evazion
25159bb21c search: support backwards ranges.
Fix things like age:1y..2y or score:5..0 returning no results.
2020-04-20 02:44:09 -05:00
evazion
172095730c search: support repeated numeric-valued metatags.
Support using the same numeric-valued metatag twice in the same search.
Numeric-valued metatags are those taking an integer, float, filesize, or
date argument. Previously using the same metatag twice would cause the
second metatag to overwrite the first metatag.

Examples:

* "id:>5 id:<10"
* "width:>500 width:<1000"
* "date:>2019-01-01 date:<2020-01-01"
2020-04-20 02:44:09 -05:00
evazion
53e5d96bb0 search: support repeated parent: metatags.
Support using the parent: metatag twice in the same search ("parent:1 parent:2").
2020-04-20 02:08:22 -05:00
evazion
43067703b0 search: clean up date range parsing. 2020-04-19 13:48:07 -05:00
evazion
6ca42947bd search: support quoted values for all metatags.
Support using quoted values with all metatags. For example: user:"blah blah",
pool:"blah blah", commentary:"blah blah", etc. Things like rating:"safe",
id:"42" also work. Both single and double quotes are supported.

Also make the status: and rating: metatags fully free. Before only
status:deleted and rating:s were free.
2020-04-19 02:54:44 -05:00
evazion
7726563733 search: refactor scan_query callers to use split_query.
Refactor to use split_query instead of scan_query to split a query on
spaces. Preparation for refactoring scan_query into something smarter.
2020-04-19 02:54:44 -05:00
evazion
621b22111e search: clean up pixiv:<any|none> handling.
Refactor so that all metatags that use parse_helper (which is most
integer-valued metatags) support the special "any" and "none" keywords.

This is mainly for consistency, since it's really only useful for using
width:none and height:none to find for certain old unsupported filetypes
that have null width/height values.
2020-04-19 01:34:14 -05:00
evazion
824d0d8edb search: clean up mpixels:, filesize: tolerance handling.
Fold parse_helper_fudged into parse_helper. This is used to make
mpixels:N and filesize:N do an inexact match with a 5% error tolerance.
For example, filesize:1mb is really equivalent to filesize:0.95mb..1.05mb
because an exact filesize:1mb search is unlikely to match anything.
2020-04-19 01:34:14 -05:00
evazion
c1ce5024e4 search: fix bug with negated user metatags.
Bug: searching for "filetype:jpg -user:evazion" would negate the
filetype:jpg metatag too. This was because the -user:<name> metatag was
negating the entire cumulative relation instead of just the user:<name>
clause.
2020-04-17 00:29:09 -05:00
evazion
e848011080 Fix #4406: Add "commentary:" metasearch.
Add:

* commentary:true (posts with commentary)
* commentary:false (posts without commentary)
* commentary:translated (posts with translated commentary)
* commentary:untranslated (posts with untranslated commentary)
* commentary:"text" (posts where any commentary field matches "text")

Known issues:

* There's no way to escape the true, false, translated, or
  untranslated keywords to do a literal text search for commentaries
  containing one of these keywords.

* Negated searches may be slow. Using a left outer join instead of a
  subquery would be faster in most cases, but negating it is harder.
2020-04-17 00:11:00 -05:00
evazion
9901161bbf search: rename artcomm metatag to commentaryupdater.
* Rename the artcomm:<user> metatag to commentaryupdater:<user>.
* Add support for negated -commentaryupdater:<user> searches.
2020-04-16 21:06:51 -05:00
evazion
8009699cf7 search: fix searches for private favorites raising privilege errors.
* Fix fav:<user> searches to return no results instead of raising a
  UserPrivilege error when the user has private favorites.

* Fix fav:<nonexistent_user> raising a UserPrivilege error instead of
  returning no results.

* Fix -ordfav:<user> not being supported.
2020-04-16 21:06:15 -05:00
evazion
f8d420d6c0 search: fix searches for private favgroups raising privilege errors.
Change favgroup:<name> searches to return no results instead of raising
a UserPrivilege error when an unpermitted user searches for a private
favgroup.

Partial fix for #4389.
2020-04-16 21:05:50 -05:00
evazion
3691404a8d search: fix invalid rating searches returning wrong results.
Partial fix for #4389.

* Fix "rating:asdf" returning all posts.
* Fix "rating:s rating:q" returning results for rating:q.
2020-04-15 01:18:41 -05:00
evazion
be27423afd search: fix invalid username searches returning wrong results.
Partial fix for #4389.

* Fix invalid username searches returning all posts instead of no posts.
* Fix "user:A user:B" returning results for user:B instead of no results.
* Fix "approver:A approver:B" returning results for approver:B instead of no results.
* Add support for negated -commenter, -noter, -noteupdater, -upvote, -downvote metatags.
* Add support for "any" and "none" values for all username metatags,
  including negated metatags that didn't support "any" or "none" before.
* Change noter:any and commenter:any to include posts with deleted notes
  or comments. Note that commenter:<username> already included deleted
  comments before. This is so that commenter:any has the same behavior
  as commenter:<username>
2020-04-15 01:18:41 -05:00
evazion
0aa8e16113 posts: fix exception when searching for "-" or "~". 2020-03-31 21:57:34 -05:00
evazion
2445e8b82f favorites: convert user.hide_favorites? to pundit. 2020-03-21 23:06:42 -05:00
evazion
415d9591c5 pundit: convert post votes to pundit.
Side effects:

* The data-current-user-is-voter <body> attribute has been removed.
* {{upvote:self}} no longer works. {{upvote:<name>}} should be used instead.
2020-03-20 18:03:01 -05:00
evazion
33d81d0d1b pundit: convert post flags to pundit. 2020-03-20 18:03:01 -05:00
evazion
2c4c29b81a pundit: convert favorite groups to pundit. 2020-03-20 18:03:01 -05:00
evazion
d10925eb95 post query builder: fix deprecated raw sql warning. 2020-03-20 17:55:08 -05:00
evazion
1a2c082b86 search: support negated wildcards in post searches.
* Support negated wildcards in searches (e.g. "holding -holding_*")
* Raise wildcard limit to matching 25 tags regardless of user level.
* Fix wildcards potentially matching empty tags.
* Fix wildcard tags being sorted by post count only, and therefore not
  having a stable ordering when tags have equal post counts.
* Fix sidebar to calculate wildcards tags the same way the search does.
2020-03-06 23:23:38 -06:00
evazion
967d398c8e search: move query parsing code from tag model to post query builder. 2020-03-06 23:23:38 -06:00
evazion
04b69954eb modqueue: add order options; change default order to newest first.
* Add options for changing the order of the modqueue (newest first,
  oldest first, highest scoring first, lowest scoring first).

* Change the default order from oldest posts first to most recently
  flagged or uploaded posts first.

* Add an order:modqueue metatag to order by most recently flagged or
  uploaded in standard searches.
2020-03-03 03:25:51 -06:00
evazion
b232470248 search: change behavior of disapproval:<reason> metatag.
* Change the `disapproval:<reason>` metatag to `disapproved:<reason>`.

* Change `disapproved:<reason>` to show all posts disapproved for a
  given reason, not just those disapproved by the current user.

* Allow searching for your own disapprovals with `disapproved:<my_name>`.

* Drop the `disapproved:<any|none>` metatags. `disapproved:any` is
  equivalent to `disapproved:<my_name>` and `disapproved:none` is
  equivalent to `-disapproved:<my_name>`.

These changes are so that you can search e.g. disapproved:poor_quality
to find all posts disapproved for poor quality.
2020-02-29 17:46:57 -06:00
evazion
463b1b734e Add curated posts page.
Add a curated posts page at /explore/posts/curated. Curated posts are
the most favorited posts by contributor-level users (users with
unlimited upload permissions).

Also add an order:curated tag using for use in regular searches.
2020-02-23 17:52:38 -06:00
evazion
76259265de posts: fix noter:<name> metatag. 2020-02-23 17:26:08 -06:00
evazion
d3bbd82d8b application record: drop execute_sql, select_value_sql methods. 2020-02-16 05:56:23 -06:00
evazion
a30c546417 search: fix embedded metatag to use truthy/falsy. 2020-01-27 11:29:29 -06:00
BrokenEagle
d9e5b94838 Added search for posts with the embedded flag enabled 2020-01-23 03:30:38 +00:00
evazion
20f893161b favgroups: optimize favgroup:<name> search metatag.
The previous method resulted in a `WHERE posts.id IN (<list of ids>)`
clause, which turned into a huge list when the favgroup had a lot of
posts.
2020-01-17 13:38:51 -06:00
evazion
ab325c5d2b favgroups: convert post_ids from string to array. 2020-01-17 00:19:20 -06:00
evazion
aff3d3b18f Fix various rubocop issues. 2020-01-11 19:01:40 -06:00
evazion
309821bf73 rubocop: fix various style issues. 2019-12-22 21:23:37 -06:00
evazion
93b03c04ad post votes: let gold users use upvote:self, downvote:self metatags.
* Let gold users use upvote:self, downvote:self metatags to search for
  their own votes.
* Don't let mods use upvote:<user>, downvote:<user> metatags to see
  votes by other users. Only let admins see other users' votes.
* Add vote count to profile page.
2019-10-28 03:45:26 -05:00
evazion
2c6567b5d2 Remove uses of the read replica database.
https://danbooru.donmai.us/forum_topics/9127?page=283#forum_post_160508

There was a recent outage that was caused by the read replica
(yukinoshita.donmai.us) being temporarily unavailable. The pg driver in
rails got hardstuck trying to connect to the replica, which brought down
the whole site. The app servers stopped responding and could only be
brought down with SIGKILL. Even try to boot the rails console didn't
work.

We only really used this to calculate tag counts inside Post.fast_count,
which wasn't really beneficial since the read replica is slower than the
main database.
2019-10-22 12:15:46 -05:00
evazion
4e4c4d627e saved searches: remove option to disable saved searches.
Remove `SavedSearch.enabled?` checks. There's no need to make saved
searches optional, since Redis is now required to run Danbooru.
2019-09-22 23:04:22 -05:00
evazion
347aa4b835 Add error pages for tag limit and pagination errors. 2019-09-09 17:43:30 -05:00
evazion
dc4d2e54b2 pools: stop using the pool_string field (#4160).
Stop using the pool_string field internally, but keep maintaining it
until we can drop it later.

* Stop using the pool_string for `pool:<name>` metatag searches.
* Stop using the pool_string in the `Post#pools` method. This is used to
  get the list of pools on post show pages.
2019-09-08 00:11:57 -05:00
evazion
5df3b01ca2 Fix timeouts in source:<url> searches and bookmarklet.
* Change the source index on posts from `(lower(source) gin_trgm_ops) WHERE source != ''`
  to just `(source gin_trgm_ops)`. The WHERE clause prevented the index
  from being used in source:<url> searches because we didn't specify
  the `source != ''` clause in the search itself. Excluding blank
  sources only saved a marginal amount of space anyway. This fixes
  timeouts in source:<url> searches and in the bookmarklet (since we do
  a source dupe check on the upload page too).

* Also switch from indexing `lower(name)` to `name` on pools and users.
  We don't need to lowercase the column because GIN indexes can be used
  with both LIKE and ILIKE queries.
2019-09-02 18:53:27 -05:00
evazion
8756480500 search: drop special case for pixiv urls in source: metatag.
* Drop support for `source:pixiv/artist-name` searches. This was a hack
  that only worked on old pixiv urls that haven't been used for years.
* Replace the old SourcePattern(lower(source)) index with a trigram index.
2019-08-29 02:06:35 -05:00
evazion
4f024d2360 pools/show: fix N+1 query on pool show page.
Fix a N+1 query when fetching posts to render thumbnails. Also adds
support for the `limit` url param on the posts show page.
2019-08-09 00:05:50 -05:00
evazion
4a1f0523a4 search: add comment_count, note_count metatags (#4004).
Add these metatags:

* comment_count
* deleted_comment_count
* active_comment_count
* note_count
* deleted_note_count
* active_note_count
* order:comment_count
* order:deleted_comment_count
* order:active_comment_count
* order:note_count
* order:deleted_note_count
* order:active_note_count
2018-12-11 18:10:20 -06:00
evazion
37b2214472 post_tags_match: replace joins with subqueries.
Refactor various post_tag_match methods to use subqueries instead of joins.

This simplifies things inside PostQueryBuilder, since now we can assume
we're always dealing with a Post relation, rather than some other table
joined with the posts table.
2018-12-11 18:10:20 -06:00
evazion
d82418ed43 Fix #3988: ordpool:<name> fails for large pools.
Also fixes ordpool:<name> not returning all posts in the correct order
when searching for series pools that contain duplicate posts.
2018-11-16 22:48:17 -06:00
Albert Yi
0508b127fd continue refactoring savedsearch 2018-11-15 12:06:13 -08:00
evazion
115ed16a96 pools: store post_ids as array instead of string (fix #3979) 2018-11-08 15:09:31 -06:00
evazion
4f02c7f70a search: add status:modqueue, status:unmoderated metatags.
* status:modqueue = ~status:pending ~status:flagged
* status:unmoderated = status:modqueue -user:self -approver:self -disapproval:any
2018-08-23 14:48:39 -05:00
evazion
89c4fe150a search: add disapproval:<any|none|disinterest|poor_quality|breaks_rules> metatag. 2018-08-23 14:08:02 -05:00