Commit Graph

8137 Commits

Author SHA1 Message Date
evazion
6b8515811f tags: drop related_tags column. 2019-09-07 22:21:55 -05:00
evazion
4fc1e16731 pools: add more name validations.
These validations should never fail because normalize_name strips
leading/trailing underscores, consecutive underscores, and unicode
whitespace.
2019-09-07 22:10:25 -05:00
evazion
1852b821f8 autocomplete: don't return deleted pools. 2019-09-07 22:10:25 -05:00
evazion
886ee16911 pools: allow searching pools by post tags.
Find all pools containing at least one post tagged 'translated':

    https://danbooru.donmai.us/pools?search[post_tags_match]=translated
2019-09-06 17:54:07 -05:00
evazion
a9b7503aa7 pools: allow searching pools by post id or post count.
All pools containing post #1:

    https://danbooru.donmai.us/pools?search[post_ids_include]=1

All pools containing either post #1 or #2:

    https://danbooru.donmai.us/pools?search[post_ids_include]=1,2
    https://danbooru.donmai.us/pools?search[post_ids_include]=1+2

Pools with 1-100 posts:

    https://danbooru.donmai.us/pools?search[post_id_count]=1..100

Pools with no posts (empty pools):

    https://danbooru.donmai.us/pools?search[post_id_count]=0
2019-09-06 16:18:29 -05:00
evazion
7d07b5f289 artist: drop unused member_names method. 2019-09-06 16:18:29 -05:00
evazion
4fb24c812f Fix #4155: Fix profile/settings metadata discrepancy. 2019-09-05 00:14:11 -05:00
evazion
3af352c76a Fix #4154: Saved searches can be cached indefinitely. 2019-09-05 00:08:00 -05:00
evazion
fc3b822bdf artists: reduce queries in artist summaries.
Avoid a few queries when searching for single artist tags.
2019-09-05 00:00:15 -05:00
evazion
416f817f6d Post.fast_count: fix incorrect counts for aliased tags.
Revert optimization from a6163258b. Turns out that we have to resolve
aliases in fast_count, otherwise for aliased tags we'll return an empty
count.

Fixes #4156.
2019-09-04 23:56:12 -05:00
evazion
59d92b448e /artists: remove /artists/new from navbar for anon users.
Fixes the logs being flooded with 403 errors because of bots and logged
out users trying to access the new artist page.
2019-09-04 16:42:24 -05:00
evazion
0ec36785f0 uploads: don't clean up replaced files too early.
Don't try delete files belonging to completed uploads during pruning. If
an uploaded post was replaced shortly after upload, then the original
file could get deleted during pruning since it's no longer in use, but
this isn't supposed to happen until the replacement grace period (30
days) is over.
2019-09-04 14:49:52 -05:00
evazion
1b426fb23f Fix #4150: Nijie strategy fails for mp4 files. 2019-09-03 22:33:09 -05:00
evazion
6bbeb9621f Update gems. 2019-09-03 22:17:18 -05:00
evazion
3e2affb501 saved searches: fix refreshed_at, cached_size to use right key. 2019-09-03 11:48:00 -05:00
evazion
3290ac4185 ugoiras: fix ffmpeg/mkvmerge check.
The -p option is nonstandard and isn't supported by dash, the /bin/sh on
Debian, so the check failed on Debian.
2019-09-03 00:34:40 -05:00
evazion
d5a2ed65f8 saved searches: add 'created' and 'posts' columns to index page. 2019-09-02 22:27:25 -05:00
evazion
05d8a05106 saved searches: add 'last refreshed' column to index page. 2019-09-02 22:10:16 -05:00
evazion
2841f0742c saved searches: fix failure when search returns no results.
* Don't try to call `sadd` when a search returns no results (`sadd`
  fails in this case).
* Add a timeout when populating the search.
* Don't offload the search to read replica. The main db is fine.
* Disable synchronous population of searches. This was too slow.
2019-09-02 22:10:10 -05:00
evazion
4abffc7faa saved searches: paginate index page + add search form. 2019-09-02 22:10:03 -05:00
evazion
6ef79fe6ab Fix #4152: Pixiv source strategy is broken. 2019-09-02 20:01:52 -05:00
evazion
1e0aadb6e7 Fix /uploads?search[uploader_name] not filtering by uploader name.
Caused by `relation = self` in `search_post_id_attribute`.
2019-09-02 19:53:47 -05:00
evazion
5df3b01ca2 Fix timeouts in source:<url> searches and bookmarklet.
* Change the source index on posts from `(lower(source) gin_trgm_ops) WHERE source != ''`
  to just `(source gin_trgm_ops)`. The WHERE clause prevented the index
  from being used in source:<url> searches because we didn't specify
  the `source != ''` clause in the search itself. Excluding blank
  sources only saved a marginal amount of space anyway. This fixes
  timeouts in source:<url> searches and in the bookmarklet (since we do
  a source dupe check on the upload page too).

* Also switch from indexing `lower(name)` to `name` on pools and users.
  We don't need to lowercase the column because GIN indexes can be used
  with both LIKE and ILIKE queries.
2019-09-02 18:53:27 -05:00
evazion
ffc693ef37 profiles: temp disable saved searches on profile pages. 2019-09-02 13:43:05 -05:00
evazion
3e85ce8110 Fix searching models by id/created_at/updated_at attributes. 2019-09-02 13:38:00 -05:00
evazion
64eb6dbb2a pixiv: possible fix for #4152. 2019-09-02 13:13:58 -05:00
evazion
f040233a50 tests: fixup ugoira converter test. 2019-09-02 12:20:37 -05:00
evazion
777eea85dd related tags: disable safe mode while generating related tags.
Also bump timeouts slightly.
2019-09-02 12:17:45 -05:00
evazion
e781c6b608 pixiv: temp disable source strategy (#4152).
Disable use of the Pixiv API until we get it working again.
2019-09-02 11:13:00 -05:00
evazion
95b8420ba2 Drop unused common_searches code. 2019-09-01 15:54:00 -05:00
evazion
dacb21b085 Drop unused users/_login_header template. 2019-09-01 15:54:00 -05:00
evazion
5b468444e7 Tweak signin / signup pages.
* Link to the signup page on the signin page.
* Clarify that emails are optional on the signup page.
2019-09-01 15:54:00 -05:00
evazion
fe4d90c5e0 settings page: add help text, rearrange settings.
* Remove 'Change password' and 'Delete account' tabs.
* Put 'Change password' under the Basic section.
* Put 'Deactivate account' under the Advanced section.
* Add help text to various settings.
2019-09-01 15:53:59 -05:00
evazion
f673552743 users: add /settings page.
Add /settings page and link to it instead of /users/:id/edit.
2019-09-01 15:00:27 -05:00
evazion
ff92b32f02 users: add /profile page (fix #4151).
* Add /profile, /profile.json endpoints.
* Make "My Account" link to /profile.
* Add 'User ID' field to profile page.
2019-09-01 15:00:23 -05:00
evazion
a932b25608 Fix #4142: Missing images after upload. 2019-09-01 13:10:37 -05:00
evazion
67100f26eb Fix #4149: Add missing post search options on applicable models. 2019-09-01 13:10:37 -05:00
evazion
1bb1b147ea Fix autoloading deprecation warning in delayed job initializer.
DEPRECATION WARNING: Initialization autoloaded the constants
    Danbooru::Paginator and Danbooru::Paginator::ActiveRecordExtension.

    Being able to do this is deprecated. Autoloading during
    initialization is going to be an error condition in future versions
    of Rails.

    Reloading does not reboot the application, and therefore code
    executed during initialization does not run again. So, if you reload
    Danbooru::Paginator, for example, the expected changes won't be
    reflected in that stale Module object.

    `config.autoloader` is set to `classic`. These autoloaded constants
    would have been unloaded if `config.autoloader` had been set to
    `:zeitwerk`.

    Please, check the "Autoloading and Reloading Constants" guide for
    solutions.
2019-09-01 13:10:37 -05:00
evazion
648ba14021 gems: add rake to gemfile.
Fixes issue with newrelic failing to set up rake instrumentation:

    ERROR : Error while detecting rake_instrumentation:
    ERROR : NameError: uninitialized constant Rake::VERSION
2019-09-01 13:10:37 -05:00
evazion
6e7b882dda tests: skip ugoira tests if ffmpeg/mkvmerge not installed. 2019-09-01 13:10:37 -05:00
evazion
3216f83ad8 users: fix deprecation warning in current_user_first.
DEPRECATION WARNING: Dangerous query method (method whose arguments
    are used as raw SQL) called with non-attribute argument(s): "id =
    52664 desc". Non-attribute arguments will be disallowed in Rails
    6.1. This method should not be called with user-provided values,
    such as request parameters or model attributes. Known-safe values
    can be passed by wrapping them in Arel.sql().
2019-09-01 13:10:37 -05:00
evazion
80d881bfcb Fix #4148: Don't prune votes. 2019-08-31 20:09:41 -05:00
evazion
b77eebf136 /favorites: replace favorites view with ordfav: search.
Make /favorites redirect to a ordfav:<user> search instead of having a
separate view just for favorites. This duplicated a lot of code for no
good reason.
2019-08-31 19:58:38 -05:00
evazion
7f7730d249 /favorites: fixup leftover related tags calculator code.
Fixup for 6dd331745.
2019-08-31 19:58:38 -05:00
evazion
ee77b101af comments: change 'Show all comments' to 'Show N hidden comments'. 2019-08-31 16:32:15 -05:00
evazion
be36968b6d Fix #3351: Mod+: Treat deleted comments as below score threshold.
Comments have three states: visible, hidden, and invisible. Visible
comments are always shown. Hidden comments are not shown until the user
clicks 'Show all comments'. Invisible comments are never shown to the
user. Deleted comments are treated as hidden for moderators and
invisible for normal users. Thresholded comments are treated as hidden
for all users.
2019-08-31 16:24:44 -05:00
evazion
7e2eb7e5a7 Fix Rails 6 content_type deprecation warning.
DEPRECATION WARNING: Rails 6.1 will return Content-Type header without
    modification. If you want just the MIME type, please use `#media_type`
    instead.
2019-08-31 12:55:07 -05:00
evazion
836ec02265 Fix #4147: Remove non-relevant bookmarklet info. 2019-08-30 22:19:20 -05:00
evazion
6dd331745a Rewrite related tags implementation.
Rewrite the implementation of related tags to be simpler, faster, and
more accurate:

* The related tags are now calculated by taking a random sample of 1000
  posts, finding the top 250 most frequent tags among those posts, then
  ordering those tags by cosine similarity.

* Related tags can generally be calculated in 50-300ms at these sample
  sizes. Very high sample sizes (25000+ posts) are still relatively fast
  (1-3 seconds), but generally they don't improve accuracy much.

* Related tags are now cached in redis rather than in the tags table.
  The related_tags column in the tags table is no longer used.

* Only the related tags in the search taglist are cached. The related
  tags returned by the 'Related tags' button are not cached.

* The cache lifetime is a fixed 4 hours.

* The 'Related tags' button now works with metatags.

* The /related_tag page now works with metatags and multitag searches.

Fixes #4134, #4146.
2019-08-30 20:03:36 -05:00
evazion
7b8584e3b0 Model#search: refactor searching for attributes. 2019-08-29 20:44:33 -05:00