Commit Graph
100 Commits
Author SHA1 Message Date
evazion 3f86a60457 Upgrade to Rails 6.0.0. 2019-08-27 18:13:30 -05:00
evazion 371182b102 sessions: fix session started_at tracking. 2019-08-27 14:22:50 -05:00
evazion 27be15ce7a related tags: disable csrf protection for update action.
This fixes InvalidAuthenticityToken errors caused by Reportbooru trying to
use this endpoint to update related tags. Reportbooru uses a secret key
to authenticate rather than using apikey-based authentication, which
makes it fail the CSRF protection.
2019-08-27 14:20:20 -05:00
evazion a3d748e300 Fix #4145: Unable to view deleted comments from post page.
Bug: if all the comments on a post were deleted then the deleted
comments wouldn't be visible to moderators.

This was because we assumed that if `last_commented_at` was nil it meant
that the post had no comments, but this was wrong. `last_commented_at`
only counts undeleted comments. It's reset to nil if all the commnets
have been deleted.
2019-08-26 13:50:42 -05:00
evazion fa37d7c156 pool orders: raise limit for order page to <1000 posts. 2019-08-26 13:50:02 -05:00
evazion e147a34664 pool orders: disable drag-and-drop ordering for pools with >100 posts.
Previously if a pool had >100 posts then the 'Order' link wouldn't
appear in the navbar, but it was still possible to visit the pool order
page directly. If a user did so, only the first 100 posts in the pool
would be shown in the drag-and-drop widget. If they tried to reorder the
pool anyway, then everything beyond the first 100 posts would be
silently removed from the pool.

Now we always show the 'Order' link, but we disable the drag-and-drop
reordering widget when the pool has >100 posts to prevent posts from
being silently removed.
2019-08-25 21:28:32 -05:00
evazion edc0c9df32 Fix exception in /pools/:id/order/edit. 2019-08-25 21:06:16 -05:00
evazion 86fa502c71 login page: remove 'sign in securely' link.
This link is unnecessary since Danbooru is now HTTPS-only.
2019-08-25 20:32:25 -05:00
evazion 11912e5cfd Update gems. 2019-08-25 20:29:32 -05:00
evazion 50df810e67 gems: drop foreman gem.
Foreman is unmaintained and locked to a really old version of thor,
which prevents upgrading to Rails 6.

The Procfile can still be used by manually installing Foreman with
`gem install foreman`. This is what Foreman's README recommends anyway.
2019-08-25 20:29:32 -05:00
evazion 0df5c0fd2b Replace deprecated update_attributes with update.
https://rubyinrails.com/2019/04/09/rails-6-1-activerecord-deprecates-update-attributes-methods/

DEPRECATION WARNING: update_attributes! is deprecated and will be removed from Rails 6.1 (please, use update! instead)
2019-08-25 20:29:32 -05:00
evazion 62875eabb2 /posts.atom: fix banned posts being visible to Members.
Bug: /posts.atom only hid loli/shota, not banned posts and not unsafe
posts when safe mode was on.
2019-08-25 20:29:32 -05:00
evazion 0101b5f5f4 danbooru default config: remove unused config settings. 2019-08-25 20:29:32 -05:00
evazion 8e39985d66 app controller: fix api responses on access denied errors.
Bug: A .json/.xml/.js request that resulted in an access denied error
returned a html response instead of a .json/.xml/.js response.
2019-08-25 20:29:32 -05:00
evazion c7f8fbbec2 app controller: standardize access denied error handling.
Refactor to use `render_error_page` to handle User::PrivilegeError
exceptions. This way these exceptions are logged to New Relic.

Changes:

* Anonymous users aren't automatically redirected to the login page.
  Instead they're taken to the access denied page, which links to the
  login/signup pages.

* JSON/XML error responses return `message` instead of `reason`.
2019-08-24 23:10:22 -05:00
evazion d4c43af1dd app controller: replace calls to access_denied with PrivilegeError.
Standardize controllers to raise User::PrivilegeError instead of calling
`access_denied` directly.
2019-08-24 23:10:22 -05:00
evazion fda9843a55 app controller: standardize auth failure error handling. 2019-08-24 22:55:36 -05:00
evazion bb7c993f4a app controller: clean up helpers.
* Move `show_moderation_notice?` out.
* Don't need to declare `helper :pagination` manually.
2019-08-24 22:55:36 -05:00
evazion 491cae6c60 app controller: standardize api limit error handling.
* Refactor api_check to use render_error_page so that api limit errors
  get logged to New Relic for analysis.

* Also standardize json error responses to return the error message in
  `message` instead of `reason`.
2019-08-24 22:55:36 -05:00
evazion a6ed7ca936 app controller: move session[:started_at] init to SessionLoader. 2019-08-24 22:55:36 -05:00
evazion 1a964f7a10 app controller: move safe mode from app controller to SessionLoader. 2019-08-24 22:55:36 -05:00
evazion f151285038 views: standardize page titles. 2019-08-24 22:55:35 -05:00
evazion 2d34e69737 api: disable csrf protection for api requests.
Fixes POST/PUT API requests failing with InvalidAuthenticityToken errors
due to missing CSRF tokens.

CSRF protection is only necessary for cookie-based authentication. For
non-cookie-based authentication we can safely disable it. That is, if
the user is already passing their login + api_key, then we don't need
to additionally verify the request with a CSRF token.

ref: https://github.com/r888888888/danbooru/commit/2e407fa47680eb05b91d20440d498b6a204bf307#comments
2019-08-24 22:55:35 -05:00
evazion 9c79f006f8 tests: move authentication tests out of post tests. 2019-08-24 22:55:35 -05:00
evazion 7c1552061b tests: fix authentication in controller tests.
Fix controller tests to login for real instead of faking it.
2019-08-24 22:55:35 -05:00
evazion a9b0362fc7 Fix #4125: Detect forum and comment spam. 2019-08-23 22:41:47 -05:00
evazion 06ff249530 dmails: factor out spam detector service. 2019-08-23 22:38:03 -05:00
evazion 7ab701c19a dmails: clean up sender/recipient logic. 2019-08-23 22:38:03 -05:00
evazion 0b78485227 dotenv: load .env.local before initializers. 2019-08-23 22:30:23 -05:00
evazion bb82011bb2 helpers: drop fast_link_to. 2019-08-22 23:33:58 -05:00
evazion 70e0d2fac2 tag change notices: use cache instead of using redis directly. 2019-08-22 23:28:49 -05:00
evazion dfe2b831a3 cache: replace memcache with redis.
Drop memcache. Use redis for all caching instead.
2019-08-22 23:15:34 -05:00
evazion 2e407fa476 rails: update defaults from 5.1 to 5.2.
protect_from_forgery removed from because it's now on by default.

ref: https://edgeguides.rubyonrails.org/configuring.html
2019-08-22 22:09:05 -05:00
evazion 87c480e2ed tests: fix /users?name=<nonexistent> test. 2019-08-22 21:30:00 -05:00
evazion e03f6b9bc0 tests: drop PostSets::Pool tests.
We no longer use this class for pool show pages. Only used by
inheritance by PostSets::FavoriteGroup.
2019-08-22 21:30:00 -05:00
evazion 6ba3d68792 dmails: fix feedback not being left when spammer is autobanned. 2019-08-22 21:29:58 -05:00
evazion 4f5d80bb60 forum post votes: fix conflicting scope name.
Fixes a conflict with `Enumerable#excluding` in Rails 6.

    Rename Array#without and Enumerable#without to Array#excluding and
    Enumerable#excluding. Old method names are retained as aliases.
2019-08-22 21:28:56 -05:00
evazion 84f4e076cc whenever: log cronjob output to log/whenever.log. 2019-08-22 17:28:53 -05:00
evazion 97cc873a3f maintenenace: fix db timeouts during vacuuming.
Setting the statement timeout at the beginning didn't work because
`PostPruner.new.prune!` clobbers the timeout (it calls `without_timeout`,
which doesn't restore the timeout properly if the timeout was zero).
2019-08-22 17:03:18 -05:00
evazion fd639e3c81 capistrano: lower downtime window for delayed jobs. 2019-08-22 16:56:42 -05:00
evazion 831aa57cad posts/show: fix thresholded comments not being hidden on pageload. 2019-08-22 16:40:45 -05:00
evazion d05ebfe116 dmails: fix bug preventing members from sending dmails.
Bug: sending dmails failed for members.

Cause: using lambdas with `rakismet_attrs` failed because unexpected
arguments are passed to the lambdas. Using procs works because the
arguments are ignored.

Also fix the tests to actually test akismet. We didn't catch this
because the tests mocked out the `spam?` call.
2019-08-22 00:42:49 -05:00
evazion 44653fb722 Fix errors on /artists/show_or_new, /wiki_pages/show_or_new pages.
Fix these pages to redirect to the new page instead of erroring out when
a name or title isn't given.
2019-08-22 00:09:50 -05:00
evazion 1292e73931 Fix errors on HEAD requests to various index pages.
Various search forms try to do `params[:search][:blah]`, which failed
for HEAD requests because we didn't normalize the `search` param.
2019-08-21 23:37:46 -05:00
evazion ac259859f9 search: don't error out on source:*pixiv* searches. 2019-08-21 23:33:49 -05:00
evazion d75d2266c9 users: fix error handling for /users?name=<does_not_exist>. 2019-08-21 23:32:34 -05:00
evazion 804ccd4fcc capistrano: fix cronjobs not updating.
Bug: capistrano didn't update the crontab during deployment. As a result
the crontab still referenced an old deployment. Eventually that
deployment was deleted and cronjobs started silently failing.

Broken in 223e5f1e8. Turns out that `whenever` defaults to running on
servers with the `db` role. Removing kagamihara from the db role meant
that whenever was no longer configured to run anywhere.
2019-08-21 18:02:55 -05:00
evazion 7cc31b97c9 pools/show: fix pagination to respect user's posts-per-page setting. 2019-08-21 14:31:58 -05:00
evazion b283281e5e comments: minimize sql queries.
Certain parts of comment rendering triggered sql queries that we didn't
really need to do. Rework things to avoid this.

* Preload comment creators in order to display commenter names with link_to_user.

* Preload comment votes in order to display "undo vote" links. Only preload
  votes for members since anonymous users can't vote and don't have "undo
  vote" links.

* Rework various conditionals to do the filtering in Ruby so that we
  avoid issuing any extra queries in sql.

* Avoid issuing any queries at all when the post doesn't have any
  comments (when last_commented_at is blank).
2019-08-20 21:55:25 -05:00
evazion 2cf929ad6e comments: show "undo vote" link by default on voted comments (fix #4143). 2019-08-20 21:17:02 -05:00
evazion 6e708dec18 /artists/new: fix artist name input box sizing.
In production the name input is really big because the artist name
column in the production db is `text` rather than `character varying`,
and simpleform treats text columns as textbox inputs.
2019-08-19 19:49:31 -05:00
evazion 0b14146f6a posts: hotfix bug in /posts.json?tags=blah+order:random. 2019-08-19 18:51:33 -05:00
evazion 782d9f3d61 comment votes: fix error handling.
Return the comment itself and a standard error response on failure.
2019-08-19 17:22:36 -05:00
evazion b8450671f2 /mod_actions: fix N+1 query. 2019-08-19 00:46:31 -05:00
evazion d4a243c8d1 footer: abbreviate git hash. 2019-08-19 00:46:31 -05:00
evazion 13f4dee59f footer: only show keyboard shortcuts link on desktop. 2019-08-19 00:46:31 -05:00
evazion dab43d96c9 jobs: migrate mass updates to ActiveJob.
Also fixes a bug where mod actions weren't logged on mass updates.
Creating the mod action silently failed because it was called when
CurrentUser wasn' set.
2019-08-19 00:46:31 -05:00
evazion 48488d04db jobs: migrate bulk reverts to ActiveJob. 2019-08-19 00:46:30 -05:00
evazion ddcc7eddfb jobs: migrate fixing post counts to ActiveJob. 2019-08-19 00:46:30 -05:00
evazion 2b033443db tests: fix user name change test (fixup for 59b277ead). 2019-08-19 00:46:30 -05:00
evazion a586916cb3 /wiki_page_versions: various usability improvements.
Changes to the /wiki_page_versions global listing:

* Add "diff" links that show you what changed in the given edit.
* Add "?" links that take you to the current version of the wiki.
* Add "»" links next to wiki page titles that take you to the wiki's full edit history.
* Add "»" links next to usernames that take you to the user's full edit history.
* Add a "Status" column that shows whether the wiki page was created,
  deleted, undeleted, or renamed.
* Link to /wiki_page_versions in sidebar, not /wiki_pages?order=time.
2019-08-18 17:08:35 -05:00
evazion 5b27726635 views: fixup regressions in new dmail, new feedback forms.
Fixup regression in 59b277ead. Simpleform expected `to_name`,
`user_name` methods to exist on these models.
2019-08-18 13:33:05 -05:00
evazion 576b4feb3b views: convert mod dashboard + admin user edit page to simple form.
Fixes bug with the user level select dropdown on the mod dashboard page
missing the builder level.
2019-08-18 11:24:42 -05:00
evazion d7d4a63e69 js: fixup bug in note hiding.
Fixup regression in e000bdb86.
2019-08-18 11:24:42 -05:00
evazion 6019a9fdc4 js: fix eslint warnings. 2019-08-18 11:24:42 -05:00
evazion f2937e3d27 js: update yarn packages. 2019-08-18 11:24:42 -05:00
evazion 59b277ead1 users: drop id_to_name, name_to_id caching.
Changes:

* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
  to return these fields in the api.

id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.

name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.

Avoiding caching also avoids the need to keep these caches consistent.
2019-08-18 11:24:42 -05:00
evazion 7871dced00 users: fix find_by_name, name_to_id to strip whitespace.
Fix find_by_name and name_to_id to use normalize_name properly, so that
they ignore leading/trailing whitespace. This fixes various search forms
failing to return results when the username field contains trailing
whitespace (inserted by autocomplete).
2019-08-18 11:24:42 -05:00
evazion 82fbb19e0f users: drop unused code. 2019-08-18 11:24:42 -05:00
evazion 17b2cd38cd dmails: drop unneeded methods. 2019-08-18 11:24:42 -05:00
evazion 27a118dfc8 tests: drop timecop gem. 2019-08-18 11:24:41 -05:00
evazion 51ca31ef8d Drop unused tag subscription model. 2019-08-17 02:47:49 -05:00
evazion 145894fe8b tests: fix alias/implication tests. 2019-08-17 02:41:07 -05:00
evazion 868a2256d1 jobs: migrate file deletion jobs to ActiveJob. 2019-08-16 20:49:35 -05:00
evazion 2b6361369e jobs: migrate aliases/implications to ActiveJob. 2019-08-16 20:49:35 -05:00
evazion 2bbdc5d143 jobs: migrate saved searches to ActiveJob.
* Fix tests to run the searches for real instead of mocking everything out.

* Fix SavedSearch.populate to only use the read only database in
  production because in breaks things in tests. Specifically:
  the posts get created in one db connection but searched for in
  another, but the second transaction doesn't see the uncommitted posts
  in the first transaction, so the search doesn't work.
2019-08-16 20:49:35 -05:00
evazion a68db501c2 jobs: migrate related tag updates to ActiveJob. 2019-08-16 20:49:35 -05:00
evazion 24eb1b155f jobs: migrate pool category updates to ActiveJob. 2019-08-16 20:49:34 -05:00
evazion 4e8b3d9310 jobs: migrate user deletions to ActiveJob. 2019-08-16 20:49:34 -05:00
evazion 822dce20ed jobs: add base ApplicationJob. 2019-08-16 20:49:34 -05:00
evazion 817f5ecf9c jobs: drop favgroup expunge job.
Make `Post#expunge!` remove favgroups synchronously.
2019-08-16 20:49:34 -05:00
evazion 798d524e60 Post#tag_match: clean up read_only param.
* Drop /posts?ro=true param (broken).
* Clean up tag_match (rescuing PG::ConnectionBad didn't do anything, we
  just build the query here, we don't run it).
2019-08-16 00:26:00 -05:00
evazion bb157f5d5b models: drop various unused #named methods. 2019-08-15 19:18:19 -05:00
evazion 0d7d2cac80 Drop unused AmazonBackup model. 2019-08-15 18:41:57 -05:00
evazion b9d35eaf2c Fix #3272: Unicode tags are still being allowed.
* Don't allow adding tags with invalid names when they already exist in
  the tags table.
* If an invalid tag is added, show an warning and ignore the tag instead
  of failing with a hard error.
* Move the _(cosplay) tag validation into the tag name validator.
2019-08-15 16:42:23 -05:00
evazion e85c6c3250 artists.js: replace 'check name' code with tag autocomplete.
Instead of having custom code check whether the artist name is already
in use, just enable autocomplete on the artist name field. This is an
easier and more robust way to indicate tag name conflicts.
2019-08-15 12:51:51 -05:00
evazion df8391aec0 Make /artists_versions and /artist_commentaries/search visible to all. 2019-08-15 12:51:51 -05:00
evazion a64cd50be4 profiles: add account upgrade link. 2019-08-15 12:51:51 -05:00
evazion 01c4395105 profiles: add 'refresh' link next to post changes count.
Remove the 'Refresh counts' link from the site map. Put it in the user's
profile instead.
2019-08-15 12:51:51 -05:00
evazion 5ceb71da8d sitemap: split 'users' section into 'profile' + 'users'.
Move everything related to the current user into it's own 'Profile'
section.
2019-08-15 12:51:51 -05:00
evazion f4db4b2eea sitemap: add missing pages to site map. 2019-08-15 12:51:51 -05:00
evazion 39d70ae5cf capistrano: simplify unicorn:terminate task.
Use the builtin task provided by capistrano3-unicorn. Ours failed with
invalid pidfile errors in some cases.
2019-08-15 12:51:51 -05:00
evazion 06b564cf76 Fix failure with did_you_mean in bundle install.
Deploying to production failed during `bundle install` because of an
incompatibility between did_you_mean-1.3.0 + ruby-2.6 + rubygems-3.0 +
bundler-2.0.

ref: `https://github.com/yuki24/did_you_mean/issues/117#issuecomment-482565387`
2019-08-15 01:23:10 -05:00
evazion 2b236a7111 unicorn: bump worker count (12 -> 16). 2019-08-15 00:36:25 -05:00
evazion 13dbdd5cd9 unicorn: clear environment variables during hot restarts.
Fixes an issue with zero downtime deployments (#4123). Hot restarting
didn't clear out BUNDLER_GEMFILE, which mean that new workers still used
the old Gemfile from previous deployments. This led to mysterious errors
with autoloading libraries (#4138).
2019-08-15 00:36:25 -05:00
evazion 5f1226ca92 Rename maintenance.rb -> danbooru_maintenance.rb.
Fixes random test failures caused by ambiguous constant lookup issues
(the `Maintenance` module name was used in multiple conflicting places).
2019-08-15 00:36:25 -05:00
evazion 247d825618 tests: fixup Post.fast_count test (a6163258b). 2019-08-15 00:36:25 -05:00
evazion 4833b8a63c Fix the circleci build. 2019-08-14 16:42:37 -05:00
evazion e29e5fe984 Fix #4131: Upgrade to Ruby 2.6.
Upgrade procedure:

    $ echo 2.6.3 > .ruby-version

    # upgrade ruby-build to know about the latest ruby versions.
    # see also: github.com/rkh/rbenv-update
    $ cd ~/.rbenv/plugins/ruby-build && git pull

    # install the version of ruby specified in .ruby-version.
    # see also: github.com/capistrano/rbenv/issues/83
    $ rbenv install --skip-existing

    # update the shell to use the latest version of ruby by default.
    $ rbenv global 2.6.3

    # update rubygems and bundler
    $ gem update --system

    # update the lockfile to use the latest version of bundler
    $ bundle update --bundler
2019-08-14 12:25:50 -05:00
evazion 38ab13f0b7 Remove unused sign up banner.
No longer used, probably broken in 98b31aafa6.
2019-08-14 01:46:44 -05:00