Commit Graph

341 Commits

Author SHA1 Message Date
evazion
0a6661d145 uploads: switch to active job.
* Switch upload processing from DelayedJob to ActiveJob.
* Remove remaining references to delayed job from tests.

Closes #4128.
2019-09-23 15:11:18 -05:00
evazion
c1bb88a52a tests: fix mock_saved_search_service! calls (fixup 4e4c4d627) 2019-09-23 03:22:32 -05:00
evazion
4e4c4d627e saved searches: remove option to disable saved searches.
Remove `SavedSearch.enabled?` checks. There's no need to make saved
searches optional, since Redis is now required to run Danbooru.
2019-09-22 23:04:22 -05:00
evazion
9839f97f9e Fix #4176: Ordering favorite groups errors out. 2019-09-22 15:24:23 -05:00
evazion
c85bbbf8c2 comments: fixup "show hidden comments" link.
Fixup for 896f144eb.
2019-09-17 00:28:40 -05:00
evazion
372fd7df1b /pool_versions: fix exception in pool diffs.
https://danbooru.donmai.us/forum_topics/9127?page=280#forum_post_159633
2019-09-10 15:13:06 -05:00
evazion
347aa4b835 Add error pages for tag limit and pagination errors. 2019-09-09 17:43:30 -05:00
evazion
059e061b15 tests: fixup comment vote, post controller tests.
Use git clean to delete stale files from an earlier checkout. These
files interfered with various things (specifically, a comment vote test
failed because there was an old comment_votes/create.json.erb template
hanging around that changed the behavior of the POST /comment_votes endpoint).
2019-09-09 15:06:24 -05:00
evazion
d812a19902 tests: fixup tests. 2019-09-09 11:51:01 -05:00
evazion
d0f060d8eb api: refactor api attribute declarations.
Replace the `method_attributes` and `hidden_attributes` methods with
`api_attributes`. `api_attributes` can be used as a class macro:

    # include only the given attributes.
    api_attributes :id, :created_at, :creator_name, ...

    # include all default attributes plus the `creator_name` method.
    api_attributes including: [:creator_name]

or as an instance method:

    def api_attributes
       [:id, :created_at, :creator_name, ...]
    end

By default, all attributes are included except for IP addresses and
tsvector columns.
2019-09-08 23:28:02 -05:00
evazion
19f2cc1e74 api: refactor only param. 2019-09-08 23:28:02 -05:00
evazion
24202d51f0 api: support the expires_in param on all GET requests.
`expires_in` is an alternative to `expiry` that supports durations other
than days:

* https://danbooru.donmai.us/posts.json?expires_in=30s
* https://danbooru.donmai.us/posts.json?expires_in=5min
* https://danbooru.donmai.us/posts.json?expires_in=1hour
* https://danbooru.donmai.us/posts.json?expires_in=1week
* https://danbooru.donmai.us/posts.json?expires_in=1month
2019-09-08 15:32:31 -05:00
evazion
22fd90eee9 api: support the expiry param on all GET requests. 2019-09-08 15:32:31 -05:00
evazion
3f7e05316d api: refactor default options for xml responses.
In xml responses, if the result is an empty array we want the response
to look like this:

   <posts type="array"/>

not like this (the default):

   <nil-classes type="array"/>

This refactors controllers so that this is done automatically instead of
having to manually call `@things.to_xml(root: "things")` everywhere. We
do this by overriding the behavior of `respond_with` in `ApplicationResponder`
to set the `root` option by default in xml responses.
2019-09-08 15:32:31 -05:00
evazion
32343303d2 forum posts: raise privilege error when viewing restricted topics.
Raise a privilege error when trying to view a restricted topic instead
of handling it in the controller. This way error handling is standardized.
2019-09-08 15:32:31 -05:00
evazion
dc4d2e54b2 pools: stop using the pool_string field (#4160).
Stop using the pool_string field internally, but keep maintaining it
until we can drop it later.

* Stop using the pool_string for `pool:<name>` metatag searches.
* Stop using the pool_string in the `Post#pools` method. This is used to
  get the list of pools on post show pages.
2019-09-08 00:11:57 -05:00
evazion
1e0aadb6e7 Fix /uploads?search[uploader_name] not filtering by uploader name.
Caused by `relation = self` in `search_post_id_attribute`.
2019-09-02 19:53:47 -05:00
evazion
3e85ce8110 Fix searching models by id/created_at/updated_at attributes. 2019-09-02 13:38:00 -05:00
evazion
f673552743 users: add /settings page.
Add /settings page and link to it instead of /users/:id/edit.
2019-09-01 15:00:27 -05:00
evazion
ff92b32f02 users: add /profile page (fix #4151).
* Add /profile, /profile.json endpoints.
* Make "My Account" link to /profile.
* Add 'User ID' field to profile page.
2019-09-01 15:00:23 -05:00
evazion
b77eebf136 /favorites: replace favorites view with ordfav: search.
Make /favorites redirect to a ordfav:<user> search instead of having a
separate view just for favorites. This duplicated a lot of code for no
good reason.
2019-08-31 19:58:38 -05:00
evazion
ee77b101af comments: change 'Show all comments' to 'Show N hidden comments'. 2019-08-31 16:32:15 -05:00
evazion
be36968b6d Fix #3351: Mod+: Treat deleted comments as below score threshold.
Comments have three states: visible, hidden, and invisible. Visible
comments are always shown. Hidden comments are not shown until the user
clicks 'Show all comments'. Invisible comments are never shown to the
user. Deleted comments are treated as hidden for moderators and
invisible for normal users. Thresholded comments are treated as hidden
for all users.
2019-08-31 16:24:44 -05:00
evazion
7e2eb7e5a7 Fix Rails 6 content_type deprecation warning.
DEPRECATION WARNING: Rails 6.1 will return Content-Type header without
    modification. If you want just the MIME type, please use `#media_type`
    instead.
2019-08-31 12:55:07 -05:00
evazion
6dd331745a Rewrite related tags implementation.
Rewrite the implementation of related tags to be simpler, faster, and
more accurate:

* The related tags are now calculated by taking a random sample of 1000
  posts, finding the top 250 most frequent tags among those posts, then
  ordering those tags by cosine similarity.

* Related tags can generally be calculated in 50-300ms at these sample
  sizes. Very high sample sizes (25000+ posts) are still relatively fast
  (1-3 seconds), but generally they don't improve accuracy much.

* Related tags are now cached in redis rather than in the tags table.
  The related_tags column in the tags table is no longer used.

* Only the related tags in the search taglist are cached. The related
  tags returned by the 'Related tags' button are not cached.

* The cache lifetime is a fixed 4 hours.

* The 'Related tags' button now works with metatags.

* The /related_tag page now works with metatags and multitag searches.

Fixes #4134, #4146.
2019-08-30 20:03:36 -05:00
evazion
a3d748e300 Fix #4145: Unable to view deleted comments from post page.
Bug: if all the comments on a post were deleted then the deleted
comments wouldn't be visible to moderators.

This was because we assumed that if `last_commented_at` was nil it meant
that the post had no comments, but this was wrong. `last_commented_at`
only counts undeleted comments. It's reset to nil if all the commnets
have been deleted.
2019-08-26 13:50:42 -05:00
evazion
edc0c9df32 Fix exception in /pools/:id/order/edit. 2019-08-25 21:06:16 -05:00
evazion
0df5c0fd2b Replace deprecated update_attributes with update.
https://rubyinrails.com/2019/04/09/rails-6-1-activerecord-deprecates-update-attributes-methods/

DEPRECATION WARNING: update_attributes! is deprecated and will be removed from Rails 6.1 (please, use update! instead)
2019-08-25 20:29:32 -05:00
evazion
62875eabb2 /posts.atom: fix banned posts being visible to Members.
Bug: /posts.atom only hid loli/shota, not banned posts and not unsafe
posts when safe mode was on.
2019-08-25 20:29:32 -05:00
evazion
8e39985d66 app controller: fix api responses on access denied errors.
Bug: A .json/.xml/.js request that resulted in an access denied error
returned a html response instead of a .json/.xml/.js response.
2019-08-25 20:29:32 -05:00
evazion
c7f8fbbec2 app controller: standardize access denied error handling.
Refactor to use `render_error_page` to handle User::PrivilegeError
exceptions. This way these exceptions are logged to New Relic.

Changes:

* Anonymous users aren't automatically redirected to the login page.
  Instead they're taken to the access denied page, which links to the
  login/signup pages.

* JSON/XML error responses return `message` instead of `reason`.
2019-08-24 23:10:22 -05:00
evazion
d4c43af1dd app controller: replace calls to access_denied with PrivilegeError.
Standardize controllers to raise User::PrivilegeError instead of calling
`access_denied` directly.
2019-08-24 23:10:22 -05:00
evazion
2d34e69737 api: disable csrf protection for api requests.
Fixes POST/PUT API requests failing with InvalidAuthenticityToken errors
due to missing CSRF tokens.

CSRF protection is only necessary for cookie-based authentication. For
non-cookie-based authentication we can safely disable it. That is, if
the user is already passing their login + api_key, then we don't need
to additionally verify the request with a CSRF token.

ref: 2e407fa476 (comments)
2019-08-24 22:55:35 -05:00
evazion
9c79f006f8 tests: move authentication tests out of post tests. 2019-08-24 22:55:35 -05:00
evazion
7c1552061b tests: fix authentication in controller tests.
Fix controller tests to login for real instead of faking it.
2019-08-24 22:55:35 -05:00
evazion
87c480e2ed tests: fix /users?name=<nonexistent> test. 2019-08-22 21:30:00 -05:00
evazion
0b14146f6a posts: hotfix bug in /posts.json?tags=blah+order:random. 2019-08-19 18:51:33 -05:00
evazion
782d9f3d61 comment votes: fix error handling.
Return the comment itself and a standard error response on failure.
2019-08-19 17:22:36 -05:00
evazion
59b277ead1 users: drop id_to_name, name_to_id caching.
Changes:

* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
  to return these fields in the api.

id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.

name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.

Avoiding caching also avoids the need to keep these caches consistent.
2019-08-18 11:24:42 -05:00
evazion
27a118dfc8 tests: drop timecop gem. 2019-08-18 11:24:41 -05:00
evazion
d657624a80 Revert "application controller: fix errors in normalize_search."
This reverts commit 28a88cfa85.
2019-08-14 01:46:43 -05:00
evazion
d0428da120 sessions: remove nonexistent GET /session route. 2019-08-14 01:46:43 -05:00
evazion
dbfd6185f1 posts: fix error on /posts?md5=<does_not_exist>. 2019-08-13 21:30:21 -05:00
evazion
0f98631908 wiki pages: fix error in /wiki_pages/does_not_exist.json 2019-08-13 21:30:21 -05:00
evazion
28a88cfa85 application controller: fix errors in normalize_search.
Fix exceptions in `normalize_search` on e.g. `https://danbooru.donmai.us/users?search=blah`.
Caused when the `search` param is not a hash.
2019-08-13 21:30:20 -05:00
evazion
a28a58c1f6 tests: add more posts/index controller tests. 2019-08-12 13:38:45 -05:00
evazion
d76be490f2 Remove tag alias corrections controller tests (fixup 18a216c67). 2019-08-10 23:49:32 -05:00
evazion
35dfc704bc application controller: fix bad file extension errors.
Fix requests with bad file extensions not always returning errors correctly:

* https://danbooru.donmai.us/posts.jpg
* https://danbooru.donmai.us/posts.blah
* https://danbooru.donmai.us/posts/bad.jpg
* https://danbooru.donmai.us/posts/bad.blah
2019-08-08 22:16:39 -05:00
evazion
39bd766b34 Fix #4053: Add disapproval index improvements.
Add search form to /moderator/post/disapprovals.
2019-08-02 22:03:03 -05:00
Albert Yi
0a058dd52b fixes #4080: Status code 500 when disapproving through API 2019-05-17 12:09:03 -07:00