Commit Graph

300 Commits

Author SHA1 Message Date
evazion
0a0a85ee70 Fix #4568: Send appealed posts back to the mod queue
* Include appealed posts in the modqueue.

* Add `status` field to appeals. Appeals start out as `pending`, then
  become `rejected` if the post isn't approved within three days. If the
  post is approved, the appeal's status becomes `succeeded`.

* Add `status` field to flags. Flags start out as `pending` then become
  `rejected` if the post is approved within three days. If the post
  isn't approved, the flag's status becomes `succeeded`.

* Leave behind a "Unapproved in three days" dummy flag when an appeal
  goes unapproved, just like when a pending post is unapproved.

* Only allow deleted posts to be appealed. Don't allow flagged posts to be appealed.

* Add `status:appealed` metatag. `status:appealed` is separate from `status:pending`.

* Include appealed posts in `status:modqueue`. Search `status:modqueue order:modqueue`
  to view the modqueue as a normal search.

* Retroactively set old flags and appeals as succeeded or rejected. This
  may not be correct for posts that were appealed or flagged multiple
  times. This is difficult to set correctly because we don't have
  approval records for old posts, so we can't tell the actual outcome of
  old flags and appeals.

* Deprecate the `is_resolved` field on post flags. A resolved flag is a
  flag that isn't pending.

* Known bug: appealed posts have a black border instead of a blue
  border. Checking whether a post has been appealed would require either
  an extra query on the posts/index page, or an is_appealed flag on
  posts, neither of which are very desirable.

* Known bug: you can't use `status:appealed` in blacklists, for the same
  reason as above.
2020-08-06 20:55:45 -05:00
evazion
9f53c459c8 wiki: fix titles with invalid space characters.
Fix wiki pages that have titles containing space characters.
2020-07-16 11:42:59 -05:00
evazion
209350de5b scripts: remove unused danbooru-specific config.
Danbooru-specific configuration now lives in https://github.com/danbooru/danbooru-ansible.
2020-06-21 18:56:42 -05:00
evazion
5c7843bd3d scripts: refactor mocked services.
Replace the mocked services in scripts/mocked_services with Rails-level
mocked services.

The scripts in scripts/mocked_services were a set of stub Sinatra
servers used to mock the Reportbooru, Recommender, and IQDBs services
during development. They return fake data so you can test pages that use
these services.

Implementing these services in Rails makes it easier to run them. It
also lets us drop a dependency on Sinatra and drop a use of HTTParty.

To use these services, set the following configuration in danbooru_local_config.rb
or .env.local:

* reportbooru_server: http://localhost:3000/mock/reportbooru
* recommender_server: http://localhost:3000/mock/recommender
* iqdbs_server: http://localhost:3000/mock/iqdb

where `http://localhost:300` is the url for your local Danbooru server
(may need to be changed depending on your configuration).
2020-06-21 15:11:56 -05:00
evazion
49383d393a BURs: rename AliasAndImplicationImporter to BulkUpdateRequestProcessor.
The name AliasAndImplicationImporter is a holdover from the time before
bulk update requests existed. This was a bad name because it doesn't do
any actual importing, instead it's used for parsing and executing bulk
update requests.
2020-05-10 22:18:47 -05:00
evazion
d5a7fafca1 posts/index: fix several "This tag is under discussion" issues.
Several fixes for the "This tag is under discussion" notice on the post
index page:

* Fix the notice appearing for BURs that aren't pending.
* Fix the notice never going away because of the cache never expiring.
* List all topics when a tag is involved in multiple BURs.
* Link to the forum post instead of the forum topic (fix #4421).
* Optimization: don't check for BURs when the search isn't a simple
  single tag search.
* Add a `tags` field to the bulk update requests table for tracking all
  tags involved in the request (excluding tags in mass updates that are
  negated/optional/wildcards). Known issue: doesn't handle tag type
  prefixes in mass updates correctly (e.g. `mass update foo -> artist:bar`
  doesn't detect the tag `bar`).
* Allow searching the /bulk_update_requests page by tags.

We don't really need to cache the notice here, but we do it anyway to
reduce queries on the post index page.
2020-04-27 19:11:47 -05:00
evazion
f7175d3b12 emails: fix migrate emails fix script. 2020-03-18 15:11:32 -05:00
evazion
258f4a8b95 users: move emails to separate table.
* Move emails from users table to email_addresses table.
* Validate that addresses are formatted correctly and are unique across
  users. Existing invalid emails are grandfathered in.
* Add is_verified flag (the address has been confirmed by the user).
* Add is_deliverable flag (an undeliverable address is an address that bounces).
* Normalize addresses to prevent registering multiple accounts with the
  same email address (using tricks like Gmail's plus addressing).
2020-03-12 21:18:53 -05:00
evazion
3c2a379d6f uploads: replace old upload limits with new upload limits. 2020-02-03 22:05:58 -06:00
evazion
18affeb4e9 Add new upload limit system (fix #4234). 2020-01-27 00:47:35 -06:00
evazion
4a7322b197 users: rework privacy mode into private favorites (fix #4257).
* Rename 'privacy mode' to 'private favorites'.
* Make the private favorites setting only hide favorites, not favgroups
  and not the user's uploads on their profile page.
* Make the favgroup is_public flag default to true instead of false and
  fix existing favgroups to be public if the user didn't have privacy mode
  enabled before.
* List _all_ public favgroups on the /favorite_groups index, not just
  favgroups belonging to the current user.
* Add a /users/<id>/favorite_groups endpoint.
2020-01-17 22:24:29 -06:00
evazion
b03173d61e favgroups: fix favgroups with invalid post ids.
Fix favgroups containing post ids that don't exist. Most of these are from
typos or from old expunged posts that weren't properly removed from favgroups.
2020-01-15 23:32:51 -06:00
evazion
ec8c427ef3 reports/uploads: remove upload report page. 2020-01-13 12:58:20 -06:00
evazion
309821bf73 rubocop: fix various style issues. 2019-12-22 21:23:37 -06:00
evazion
43d0955b61 Fix #4065: Sometimes normalization brakes twitter intent links.
Fix incorrectly normalized Twitter intent URLs:

    url: https://twitter.com/intent/user?user_id=881080941443239936
    normalized_url: http://twitter.com/intent/

to

    url: https://twitter.com/intent/user?user_id=881080941443239936
    normalized_url: http://twitter.com/intent/user?user_id=881080941443239936/

Fixes about ~1300 incorrectly normalized urls in the production db.
2019-11-17 19:08:07 -06:00
evazion
13f26c81d8 Remove unused files. 2019-11-15 23:59:56 -06:00
evazion
d617b20b49 dtext links: allow searching for forum posts linking to specific tag. 2019-10-27 01:05:54 -05:00
evazion
6dcf0484f0 dtext links: add fix script to populate dtext_links table. 2019-10-24 19:48:30 -05:00
evazion
2c6567b5d2 Remove uses of the read replica database.
https://danbooru.donmai.us/forum_topics/9127?page=283#forum_post_160508

There was a recent outage that was caused by the read replica
(yukinoshita.donmai.us) being temporarily unavailable. The pg driver in
rails got hardstuck trying to connect to the replica, which brought down
the whole site. The app servers stopped responding and could only be
brought down with SIGKILL. Even try to boot the rails console didn't
work.

We only really used this to calculate tag counts inside Post.fast_count,
which wasn't really beneficial since the read replica is slower than the
main database.
2019-10-22 12:15:46 -05:00
evazion
22d4625071 Fix #4173: Fix inconsistent max upload size. 2019-09-22 22:59:33 -05:00
evazion
1d6d816b23 nginx: add production nginx config files. 2019-09-22 22:59:33 -05:00
evazion
0df5c0fd2b Replace deprecated update_attributes with update.
https://rubyinrails.com/2019/04/09/rails-6-1-activerecord-deprecates-update-attributes-methods/

DEPRECATION WARNING: update_attributes! is deprecated and will be removed from Rails 6.1 (please, use update! instead)
2019-08-25 20:29:32 -05:00
evazion
dfe2b831a3 cache: replace memcache with redis.
Drop memcache. Use redis for all caching instead.
2019-08-22 23:15:34 -05:00
evazion
e29e5fe984 Fix #4131: Upgrade to Ruby 2.6.
Upgrade procedure:

    $ echo 2.6.3 > .ruby-version

    # upgrade ruby-build to know about the latest ruby versions.
    # see also: github.com/rkh/rbenv-update
    $ cd ~/.rbenv/plugins/ruby-build && git pull

    # install the version of ruby specified in .ruby-version.
    # see also: github.com/capistrano/rbenv/issues/83
    $ rbenv install --skip-existing

    # update the shell to use the latest version of ruby by default.
    $ rbenv global 2.6.3

    # update rubygems and bundler
    $ gem update --system

    # update the lockfile to use the latest version of bundler
    $ bundle update --bundler
2019-08-14 12:25:50 -05:00
evazion
b0da649b09 Remove obsolete scripts.
These are mostly old files mostly related to maintenance of Danbooru 1
or the upgrade to Danbooru 2.
2019-08-06 10:42:45 -05:00
nothink
14b0b3f000 Fix typo in nginx.danbooru.conf 2019-08-06 07:48:08 +09:00
nothink
72ceb484ce Update INSTALL.debian, for Debian 9(stretch) 2019-08-05 23:29:12 +09:00
Albert Yi
5e5e86c383 limit number of synchronous saved search updates 2019-07-02 17:16:33 -07:00
Albert Yi
0508b127fd continue refactoring savedsearch 2018-11-15 12:06:13 -08:00
Albert Yi
ea405bdc47 update nginx config 2018-10-16 14:58:30 -07:00
Albert Yi
66ce07e993 add additional install scripts, fix staging deploy 2018-10-16 09:47:58 -07:00
evazion
60ea46e1e7 Add sample newrelic config. 2018-10-09 20:11:44 -05:00
Albert Yi
1a4949368c add server 2018-10-09 14:40:43 -07:00
Albert Yi
897d3d3a99 add production2 env for testing new servers 2018-10-03 11:23:26 -07:00
Albert Yi
e58172483b update iqdb after post replacements (#3885) 2018-09-11 12:24:15 -07:00
evazion
6ff70c5002 mod actions: separate account upgrades from promotions (fix #3841) 2018-08-26 11:12:04 -05:00
r888888888
344c46ed00 add mock recommender service for development, add user-context recommended posts 2018-07-21 14:02:57 -07:00
Albert Yi
2ec9aad320 fixes #3735 2018-06-05 13:23:30 -07:00
evazion
c50ad41792 nginx.danbooru.conf: fix X-Forwarded-Proto.
Sending `http` is wrong, it causes a redirect loop if HSTS is enabled.
2018-04-19 00:45:55 -05:00
r888888888
abce4d2551 Raise error on unpermitted params.
Fail loudly if we forget to whitelist a param instead of silently
ignoring it.

misc models: convert to strong params.

artist commentaries: convert to strong params.

* Disallow changing or setting post_id to a nonexistent post.

artists: convert to strong params.

* Disallow setting `is_banned` in create/update actions. Changing it
  this way instead of with the ban/unban actions would leave the artist in
  a partially banned state.

bans: convert to strong params.

* Disallow changing the user_id after the ban has been created.

comments: convert to strong params.

favorite groups: convert to strong params.

news updates: convert to strong params.

post appeals: convert to strong params.

post flags: convert to strong params.

* Disallow users from setting the `is_deleted` / `is_resolved` flags.

ip bans: convert to strong params.

user feedbacks: convert to strong params.

* Disallow users from setting `disable_dmail_notification` when creating feedbacks.
* Disallow changing the user_id after the feedback has been created.

notes: convert to strong params.

wiki pages: convert to strong params.

* Also fix non-Builders being able to delete wiki pages.

saved searches: convert to strong params.

pools: convert to strong params.

* Disallow setting `post_count` or `is_deleted` in create/update actions.

janitor trials: convert to strong params.

post disapprovals: convert to strong params.

* Factor out quick-mod bar to shared partial.
* Fix quick-mod bar to use `Post#is_approvable?` to determine visibility
  of Approve button.

dmail filters: convert to strong params.

password resets: convert to strong params.

user name change requests: convert to strong params.

posts: convert to strong params.

users: convert to strong params.

* Disallow setting password_hash, last_logged_in_at, last_forum_read_at,
  has_mail, and dmail_filter_attributes[user_id].

* Remove initialize_default_image_size (dead code).

uploads: convert to strong params.

* Remove `initialize_status` because status already defaults to pending
  in the database.

tag aliases/implications: convert to strong params.

tags: convert to strong params.

forum posts: convert to strong params.

* Disallow changing the topic_id after creating the post.
* Disallow setting is_deleted (destroy/undelete actions should be used instead).
* Remove is_sticky / is_locked (nonexistent attributes).

forum topics: convert to strong params.

* merges https://github.com/evazion/danbooru/tree/wip-rails-5.1
* lock pg gem to 0.21 (1.0.0 is incompatible with rails 5.1.4)
* switch to factorybot and change all references

Co-authored-by: r888888888 <r888888888@gmail.com>
Co-authored-by: evazion <noizave@gmail.com>

add diffs
2018-04-06 18:09:57 -07:00
evazion
feaad0bbd0 nginx.conf: fix inconsistent client_max_body_size. 2018-02-27 18:22:37 -06:00
evazion
89b523bdae Fix #3480: NoMethodError error when adding artist tag to post. 2017-12-30 16:07:52 -06:00
evazion
45cecff1a6 Move with_timeout / without_timeout to ApplicationRecord. 2017-11-26 13:57:52 -06:00
evazion
7034a96048 Purge invalid gentags from tags table (#3390).
Purges gentags with invalid names from the tags table. Addresses #3390
(Searching certain metatags results in an empty paginator) by removing
metatags from the tags table.
2017-11-20 12:47:30 -06:00
r888888888
5f004b6138 add script for vacuuming 2017-11-15 18:56:19 -08:00
BrokenEagle
e312ae3d92 Added fix script to update the correct tagcount for all posts 2017-11-14 17:11:18 -08:00
r888888888
13f7d7e66d add targeted post down voting report, add delayed job plugin to disable statement timeout 2017-10-10 12:56:57 -07:00
r888888888
507aa7c641 Merge branch 'master' of github.com:r888888888/danbooru 2017-09-23 13:15:05 -07:00
r888888888
5ad18aef22 implement gestures 2017-09-23 13:14:49 -07:00
r888888888
b9bc536a79 force s3 backup on post replacements if md5 doesn't change (ref #3290) 2017-09-18 11:11:48 -07:00