Commit Graph

10529 Commits

Author SHA1 Message Date
evazion
247934ad83 db: add non-null constraints to all non-optional columns.
Add non-null constraints to all columns that are non-optional. Now the
only columns that are nullable are optional columns.
2021-03-30 04:52:01 -05:00
evazion
803efe8501 Don't use secure cookes on non-HTTPS deployments.
Fixes not being able to login or signup when running in production mode
on a non-HTTPS site.
2021-03-30 03:58:34 -05:00
evazion
172265b4ef Merge pull request #4772 from nonamethanks/fix-alias-implication
Tag Aliases: automatically remove redundant implications
2021-03-30 00:40:11 -05:00
evazion
857776db9e Merge pull request #4779 from nonamethanks/fix-typo
Tag implications: fix typo in validate error
2021-03-30 00:33:45 -05:00
evazion
b3c1c753b3 comments: allow admins to remove comment votes (fix #4640)
Allow admins to remove comment votes by other users. This is done by
clicking the comment score to get to the comment vote list, then
clicking the Remove button on every vote.
2021-03-30 00:10:25 -05:00
evazion
6b91e55283 comments: allow votes to be soft deleted.
Make it so that when a user removes their own vote, the vote is soft
deleted (the is_deleted flag is set) instead of hard deleted.

Changes:

* Add is_deleted flag to comment votes.
* Relax uniqueness constraint so you can have multiple deleted votes on
  the same comment. You can still only have one active vote on the comment.
* Add `soft_delete` method to Deletable concern.
2021-03-30 00:10:22 -05:00
evazion
55129b1819 Rework Dockerfile.
* Optimize Dockerfile to minimize size of the Docker image.
* Specify exact versions of important dependencies (Ruby, Node, Vips) to
  ensure our dependencies are up to date and locked to known versions.
* Install Vips from source because the version that ships with Ubuntu is too old.
* Install FFmpeg from source because otherwise using the Ubuntu package
  pulls in tons of video libraries we don't need, bloating the image.
2021-03-29 06:00:37 -05:00
evazion
6f6a73edd2 Fix "Blocked host" error in development environment.
Fix custom hostnames being blocked in the development environment.
2021-03-29 03:01:06 -05:00
evazion
6404aa9aa9 Log to stdout in development and production.
Always log to stdout instead of logging to files in `log/{development,production}.log`.

For development, logging to files wasn't really useful, and could
generate multi-gigabyte log files if you weren't paying attention. For
production, most systems these days (such as Docker and Systemd) prefer
that you write your logs to stdout so they can manage them.

Fixes the Docker image writing logs inside the container, which never
got rotated and could fill up the container.
2021-03-29 03:01:02 -05:00
evazion
12436c4aa9 Fix IpAddressType autoload warning.
Fix Rails complaining about IpAddressType not being reloaded by hot
reloading:

    DEPRECATION WARNING: Initialization autoloaded the constant IpAddressType.

    Being able to do this is deprecated. Autoloading during initialization is going
    to be an error condition in future versions of Rails.

    Reloading does not reboot the application, and therefore code executed during
    initialization does not run again. So, if you reload IpAddressType, for example,
    the expected changes won't be reflected in that stale Class object.

    This autoloaded constant has been unloaded.

    In order to autoload safely at boot time, please wrap your code in a reloader
    callback this way:

        Rails.application.reloader.to_prepare do
        # Autoload classes and modules needed at boot time here.
        end

    That block runs when the application boots, and every time there is a reload.
    For historical reasons, it may run twice, so it has to be idempotent.

    Check the "Autoloading and Reloading Constants" guide to learn more about how
    Rails autoloads and reloads.
2021-03-29 03:01:02 -05:00
evazion
e2704f6a7b Danbooru::Http: redirect POST to GET on 302.
When a POST request returns a 302 redirect, follow the redirect with a
GET request instead of with a POST request.

HTTP standards leave it unspecified whether a POST request that returns
a 302 redirect should be followed with a GET or with a POST. A GET is
what most browsers use, which means it's what most servers expect.

Fixes the /tagme Discord command not working because when we uploaded
the image to DeepDanbooru, the POST request returned a 302 redirect,
which the server expected us to follow with a GET, not with a POST.

Ref:

* https://stackoverflow.com/questions/17605915/what-is-the-correct-behavior-expected-of-an-http-post-302-redirect-to-get
2021-03-29 03:01:02 -05:00
evazion
07720b04a5 Update Ruby gems and Yarn packages. 2021-03-29 03:01:02 -05:00
evazion
b8fa15cf50 gems: temp disable meta_request.
Revert 442d0f8dd. Fix was merged upstream but a new gem hasn't been
released yet.
2021-03-29 03:01:02 -05:00
nonamethanks
cda1f64d89 Tag implications: fix typo in validate error 2021-03-28 14:21:18 +02:00
evazion
7984575210 mimemagic: update to 0.3.8 (#4776). 2021-03-25 14:50:42 -05:00
evazion
442d0f8dd3 gems: remove meta_request fork.
Remove a workaround added in 2c06766c9. meta_request had a bug that
caused Rails to fail to launch under Rails 6.1. The fix was finally
merged upstream.

hxxps://github.com/dejan/rails_panel/pull/177.
2021-03-24 17:37:48 -05:00
evazion
c11f13050b Fix #4776: docker-compose.simple.yaml - fails to find mimemagic gem 2021-03-24 13:31:22 -05:00
evazion
9baac1fec0 autocomplete: increase contrast of selected tag in light mode. 2021-03-24 13:29:27 -05:00
evazion
178cf07690 discord: increase timeout of /count command. 2021-03-24 13:29:05 -05:00
evazion
08270973f1 Add scripts for a single-command Danbooru deployment.
Add a Docker Compose file that launches a minimal Danbooru instance in a
Docker container with a single command. This is suitable as a quick demo
or for personal use, not for public-facing sites.

To use it, just run `bin/danbooru`. This is a wrapper script that
installs Docker Compose then uses it to start Danbooru.

This will generate a lot of debug output and take several minutes while
it builds the Docker containers. Be patient. When it's done, you should
have an empty booru accessible at http://localhost.
2021-03-23 21:35:23 -05:00
evazion
9c07d710f4 config: fix bug in default backup storage manager config.
Fix uploads failing in the default configuration because the backup
storage manager didn't initialize the null storage manager with the
required `base_url` and `base_dir` options.
2021-03-23 19:53:31 -05:00
evazion
c60b9f2804 robots.txt: disallow /iqdb_queries.
Block Google from accidentally DoS'ing us by crawling /iqdb_queries en
masse.
2021-03-23 11:33:42 -05:00
evazion
1a8c70f5ff config: auto generate secret key if none given.
Automatically generate a random secret key for `Danbooru.config.secret_key_base`
if no key is specified.

This so that you can run Danbooru in a Docker container with zero
configuration.

This removes support for the ~/.danbooru/secret_token file and the
SECRET_TOKEN environment variable. If you used either one of these, you
must copy the value either to DANBOORU_SECRET_KEY_BASE in .env.local, or to
`secret_key_base` in config/danbooru_local_config.rb.

   # .env.local
   DANBOORU_SECRET_KEY_BASE=<value>

   # config/danbooru_local_config.rb
   def secret_key_base
      # <value>
   end
2021-03-23 03:11:41 -05:00
evazion
189adc683f config: set default database connection settings.
Set sensible defaults for connecting to the database. By default, we try
to connect to the `danbooru2` database running on localhost as the
`danbooru` user. These are the defaults recommended by the install
guide.

If you need to change the database settings, set DATABASE_URL in
.env.local or on the command line:

   # .env.local
   DATABASE_URL=postgresql://danbooru:password@localhost/danbooru2

   # command line
   $ DATABASE_URL=postgresql://danbooru:password@localhost/danbooru2 bin/rails server

This eliminates the need to copy script/install/database.yml.templ to
config/database.yml during installation and during deployment. This is
so that Danbooru works out of the box without extra configuration. In
particular, this is so that we can run Danbooru in a Docker container
without having to set DATABASE_URL.
2021-03-23 02:29:45 -05:00
evazion
6a84d33409 Fix #4770: Allow flaggers to update flag reason. 2021-03-23 01:27:16 -05:00
evazion
41e0cad458 ip bans: allow full bans to overlap partial bans.
Allow full banning an IP that is part of a subnet that has already been
partially banned.
2021-03-23 01:27:16 -05:00
evazion
d906de8192 Merge pull request #4773 from nonamethanks/delete-notice
Add embed wiki to delete dialog
2021-03-23 00:42:39 -05:00
evazion
c9ca5c0598 rake: fix images:manifest failing on file permission errors.
Fix images:manifest failing with an exception when a file can't be
opened because we don't have permission to access the file.
2021-03-20 16:15:16 -05:00
evazion
048171fe29 rake: fix images:manifest task to handle corrupted files. 2021-03-20 05:30:06 -05:00
evazion
6879fe73db rake: add task for checking image md5s.
Add a rake task for checking the md5s and size/width/height of all
uploaded files. Generates a JSON manifest file.

Usage: bin/rake images:manifest
2021-03-20 02:46:28 -05:00
evazion
fd09cc5e96 posts: fix Download link not respecting tagged filenames option.
Fix bug reported in forum #182766:

    The Download button on the posts page does not respect the Disable
    tagged filenames user setting. Tags are included in the filename when
    clicking the Download button even when the Disable tagged filenames
    setting is set to Yes. Right click -> Save As on the image still
    respects the setting.
2021-03-20 02:14:23 -05:00
nonamethanks
e7e0c2314d Add embed wiki to delete dialog 2021-03-20 05:45:34 +01:00
evazion
92225177a8 Update oauth2 gem.
Fixes a critical bug introduced in oauth2 1.4.5 that caused the Pawoo
source strategy to fail.
2021-03-19 16:49:14 -05:00
evazion
737f4ffcf7 Update CHANGELOG.md. 2021-03-19 15:51:25 -05:00
nonamethanks
917b08639f Tag Aliases: automatically remove redundant implications 2021-03-19 15:38:47 +01:00
evazion
1a7a108d47 discord: add /tagme command. 2021-03-19 04:44:22 -05:00
evazion
cebfe3308e discord: show favcount in post embeds. 2021-03-18 23:00:13 -05:00
evazion
d5903b61c4 discord: add function to register all commands.
* Add a `DiscordSlashCommand.register_slash_commands!` method to register
  all slash commands with the Discord API.
* Allow registering global commands.
* Refactor slash commands to use class attributes for the command
  name, description, and options.
2021-03-18 22:59:43 -05:00
evazion
f75b1ddb4a discord: add /time command. 2021-03-18 22:30:22 -05:00
evazion
1fd23c344a Update ruby gems and yarn packages. 2021-03-18 21:35:17 -05:00
evazion
fb7ffafe0a Merge pull request #4763 from BrokenEagle/embedded-notes-stack-order
Add ability to determine stack order of embedded notes
2021-03-18 21:34:36 -05:00
evazion
6267426013 Merge pull request #4765 from nonamethanks/skeb-artist
Artist finder: add skeb.jp root to blacklist
2021-03-18 21:32:13 -05:00
evazion
b8f7c3795c Merge pull request #4767 from nonamethanks/pixiv-other-names
Pixiv: don't add auto-generated usernames to the other names field
2021-03-18 21:29:21 -05:00
evazion
29d2e7fed2 storage manager: remove hierarchical option.
Remove the `hierarchical` file storage option. This means that image
files are always stored in MD5-based subdirectories, like this:

   https://danbooru.donmai.us/data/original/f3/a7/f3a70a89c350b5ed4db22dbb25b934bb.jpg
   https://danbooru.donmai.us/data/sample/f3/a7/sample-f3a70a89c350b5ed4db22dbb25b934bb.jpg
   https://danbooru.donmai.us/data/preview/f3/a7/f3a70a89c350b5ed4db22dbb25b934bb.jpg

instead of in a single flat directory, like this:

   https://danbooru.donmai.us/data/original/f3a70a89c350b5ed4db22dbb25b934bb.jpg

This option is removed because storing files in a single directory is a
bad idea for large installations, and migrating from a single directory
to subdirectories later is a pain.

Downstream boorus who still have files in the old layout can migrate by
running this script:

   `./script/fixes/077_symlink_subdirectories.rb`

This will create symlinks that redirect the 00-ff subdirectories back to
the current directory, so that you can still store files in a single
directory, but use URLs containing subdirectories.

You should also make sure to remove the `hierarchical` option from
`storage_manager` in `config/danbooru_local_config.rb` if you set it
there.
2021-03-18 01:33:56 -05:00
evazion
a620a71b59 storage manager: remove original_subdir option.
Always store original files in `public/data/original` instead of directly in
`public/data`. Previously this was optional and defaulted to off.

Downstream boorus will need to either move all images in the
`public/data` directory to `public/data/original`, or symlink the
`public/data/original` directory to the toplevel `public/data` directory:

    ln -s . /path/to/danbooru/public/data/original

This to simplify file layout. This option existed because in the past we
stored original files in different locations on different servers (for
no particular reason).
2021-03-16 23:37:07 -05:00
evazion
0f90ae0fed storage manager: use canonical URL for image URLs.
Generate image URLs relative to the site's canonical URL instead of
relative to the domain of the current request.

This means that all subdomains of Danbooru - safebooru.donmai.us,
shima.donmai.us, saitou.donmai.us, and kagamihara.donmai.us - will use
image URLs from https://danbooru.donmai.us, instead of from the current
domain.

The main reason we did this before was so that we could generate either
http:// or https:// image URLs, depending on whether the current request
was HTTP or HTTPS, back when we tried to support both at the same time.
Now we support only HTTPS in production, so there's no need for this. It
was also pretty hacky, since it required storing the URL of the current
request in a per-request global variable in `CurrentUser`.

This also improves caching slightly, since users of safebooru.donmai.us
will receive cached images from danbooru.donmai.us.

Downstream boorus should make sure that the `canonical_url` and
`storage_manager` config options are set correctly. If you don't support
https:// in development, you should make sure to set the canonical_url
option to http:// instead of https://.
2021-03-16 23:30:29 -05:00
evazion
f93b1fe478 custom css: don't add !important to every line.
Fixes #4398.
2021-03-16 20:04:09 -05:00
BrokenEagle
79a70083ef Add support for determining stack order of embedded notes 2021-03-16 04:02:47 +00:00
nonamethanks
073f63cfa7 Pixiv: don't add auto-generated usernames to the other names field 2021-03-16 02:44:49 +01:00
evazion
28c0a48117 discord: fix tag search commands being limited to 2 tags. 2021-03-14 16:42:07 -05:00