Commit Graph

2379 Commits

Author SHA1 Message Date
evazion
5b208ddb78 discord: fix /posts command inside DMs
Fix a bug where the /posts command failed when used in a DM channel,
because we were trying to fetch a `nsfw` key that didn't exist.
2021-06-02 23:32:08 -05:00
evazion
4003ddd788 Merge pull request #4808 from nonamethanks/fix-weibo
Weibo: fix new lengthier image urls breaking the batch bookmarklet
2021-06-02 15:17:54 -05:00
nonamethanks
ac4b4a151c Weibo: fix new lengthier image urls breaking the batch bookmarklet 2021-05-31 00:36:53 +02:00
nonamethanks
1234d93292 Nijie: get correct image when using bookmarklet 2021-05-25 12:20:39 +02:00
nonamethanks
ac071b147f Lofter: fix strategy for additional type of html theme 2021-05-23 18:05:16 +02:00
nonamethanks
bc4fd16c2c Add Lofter support 2021-05-19 16:40:33 +02:00
evazion
12eacbe76f Merge pull request #4800 from nonamethanks/fix-skeb
Skeb: add support for multi-image posts and fix tests
2021-05-17 12:16:21 -05:00
nonamethanks
1bc266a3db Skeb: add support for multi-image posts and fix tests 2021-05-16 12:50:39 +02:00
evazion
2537145b02 users: disallow banned users from changing email or deleting account.
This is to prevent people from wiping their email address after they've
been banned and reusing it to verify a new account.
2021-05-15 04:36:22 -05:00
evazion
e36fb6fee3 /status: show HTTP request headers and client IP.
Show the HTTP request headers and the client IP on the /status page.
This is for debugging request headers added by reverse proxies such as
Cloudflare and Nginx, and for making sure the client IP is correctly set
by the X-Forwarded-For header.
2021-05-06 00:32:24 -05:00
evazion
ba5a2dcd79 Merge pull request #4794 from nonamethanks/fix-skeb
Skeb: fix broken strategy
2021-05-05 05:08:52 -05:00
nonamethanks
e95731d58f Skeb: fix broken strategy 2021-05-04 21:18:12 +02:00
nonamethanks
cb6196c259 Nicoseiga: auto-add spoiler tags to commentary 2021-04-06 14:08:49 +02:00
evazion
6b91e55283 comments: allow votes to be soft deleted.
Make it so that when a user removes their own vote, the vote is soft
deleted (the is_deleted flag is set) instead of hard deleted.

Changes:

* Add is_deleted flag to comment votes.
* Relax uniqueness constraint so you can have multiple deleted votes on
  the same comment. You can still only have one active vote on the comment.
* Add `soft_delete` method to Deletable concern.
2021-03-30 00:10:22 -05:00
evazion
e2704f6a7b Danbooru::Http: redirect POST to GET on 302.
When a POST request returns a 302 redirect, follow the redirect with a
GET request instead of with a POST request.

HTTP standards leave it unspecified whether a POST request that returns
a 302 redirect should be followed with a GET or with a POST. A GET is
what most browsers use, which means it's what most servers expect.

Fixes the /tagme Discord command not working because when we uploaded
the image to DeepDanbooru, the POST request returned a 302 redirect,
which the server expected us to follow with a GET, not with a POST.

Ref:

* https://stackoverflow.com/questions/17605915/what-is-the-correct-behavior-expected-of-an-http-post-302-redirect-to-get
2021-03-29 03:01:02 -05:00
evazion
178cf07690 discord: increase timeout of /count command. 2021-03-24 13:29:05 -05:00
evazion
9c07d710f4 config: fix bug in default backup storage manager config.
Fix uploads failing in the default configuration because the backup
storage manager didn't initialize the null storage manager with the
required `base_url` and `base_dir` options.
2021-03-23 19:53:31 -05:00
evazion
1a7a108d47 discord: add /tagme command. 2021-03-19 04:44:22 -05:00
evazion
cebfe3308e discord: show favcount in post embeds. 2021-03-18 23:00:13 -05:00
evazion
d5903b61c4 discord: add function to register all commands.
* Add a `DiscordSlashCommand.register_slash_commands!` method to register
  all slash commands with the Discord API.
* Allow registering global commands.
* Refactor slash commands to use class attributes for the command
  name, description, and options.
2021-03-18 22:59:43 -05:00
evazion
f75b1ddb4a discord: add /time command. 2021-03-18 22:30:22 -05:00
evazion
6267426013 Merge pull request #4765 from nonamethanks/skeb-artist
Artist finder: add skeb.jp root to blacklist
2021-03-18 21:32:13 -05:00
evazion
b8f7c3795c Merge pull request #4767 from nonamethanks/pixiv-other-names
Pixiv: don't add auto-generated usernames to the other names field
2021-03-18 21:29:21 -05:00
evazion
29d2e7fed2 storage manager: remove hierarchical option.
Remove the `hierarchical` file storage option. This means that image
files are always stored in MD5-based subdirectories, like this:

   https://danbooru.donmai.us/data/original/f3/a7/f3a70a89c350b5ed4db22dbb25b934bb.jpg
   https://danbooru.donmai.us/data/sample/f3/a7/sample-f3a70a89c350b5ed4db22dbb25b934bb.jpg
   https://danbooru.donmai.us/data/preview/f3/a7/f3a70a89c350b5ed4db22dbb25b934bb.jpg

instead of in a single flat directory, like this:

   https://danbooru.donmai.us/data/original/f3a70a89c350b5ed4db22dbb25b934bb.jpg

This option is removed because storing files in a single directory is a
bad idea for large installations, and migrating from a single directory
to subdirectories later is a pain.

Downstream boorus who still have files in the old layout can migrate by
running this script:

   `./script/fixes/077_symlink_subdirectories.rb`

This will create symlinks that redirect the 00-ff subdirectories back to
the current directory, so that you can still store files in a single
directory, but use URLs containing subdirectories.

You should also make sure to remove the `hierarchical` option from
`storage_manager` in `config/danbooru_local_config.rb` if you set it
there.
2021-03-18 01:33:56 -05:00
evazion
a620a71b59 storage manager: remove original_subdir option.
Always store original files in `public/data/original` instead of directly in
`public/data`. Previously this was optional and defaulted to off.

Downstream boorus will need to either move all images in the
`public/data` directory to `public/data/original`, or symlink the
`public/data/original` directory to the toplevel `public/data` directory:

    ln -s . /path/to/danbooru/public/data/original

This to simplify file layout. This option existed because in the past we
stored original files in different locations on different servers (for
no particular reason).
2021-03-16 23:37:07 -05:00
evazion
0f90ae0fed storage manager: use canonical URL for image URLs.
Generate image URLs relative to the site's canonical URL instead of
relative to the domain of the current request.

This means that all subdomains of Danbooru - safebooru.donmai.us,
shima.donmai.us, saitou.donmai.us, and kagamihara.donmai.us - will use
image URLs from https://danbooru.donmai.us, instead of from the current
domain.

The main reason we did this before was so that we could generate either
http:// or https:// image URLs, depending on whether the current request
was HTTP or HTTPS, back when we tried to support both at the same time.
Now we support only HTTPS in production, so there's no need for this. It
was also pretty hacky, since it required storing the URL of the current
request in a per-request global variable in `CurrentUser`.

This also improves caching slightly, since users of safebooru.donmai.us
will receive cached images from danbooru.donmai.us.

Downstream boorus should make sure that the `canonical_url` and
`storage_manager` config options are set correctly. If you don't support
https:// in development, you should make sure to set the canonical_url
option to http:// instead of https://.
2021-03-16 23:30:29 -05:00
evazion
f93b1fe478 custom css: don't add !important to every line.
Fixes #4398.
2021-03-16 20:04:09 -05:00
nonamethanks
073f63cfa7 Pixiv: don't add auto-generated usernames to the other names field 2021-03-16 02:44:49 +01:00
evazion
28c0a48117 discord: fix tag search commands being limited to 2 tags. 2021-03-14 16:42:07 -05:00
nonamethanks
0cae2e75b9 Artist finder: add skeb.jp root to blacklist 2021-03-14 04:13:55 +01:00
evazion
0f39ea1ff4 Merge pull request #4758 from nonamethanks/fix-mastodon
Mastodon: fix strategy raising an exception for direct links
2021-03-12 22:48:12 -06:00
evazion
f219fc09ec discord: add /wiki command. 2021-03-12 22:44:57 -06:00
evazion
698be2d0e4 discord: add /random command. 2021-03-11 21:23:20 -06:00
evazion
2c8c7ff80a discord: add initial slash command integration.
Add initial support for the `/count <tags>` and `/posts <tags>` slash commands.

Slash commands are basically like webhooks; we register a command, and
when anybody types that command in Discord, Discord sends us a HTTP
request that we respond to.

* https://discord.com/developers/docs/interactions/slash-commands
* https://support.discord.com/hc/en-us/articles/1500000368501-Slash-Commands-FAQ
2021-03-11 03:04:10 -06:00
evazion
1c2f3abe56 discord: rename DiscordApiClient to DiscordWebhookService. 2021-03-11 03:02:50 -06:00
evazion
81fe68d392 bans: change expires_at field to duration.
Changes:

* Change the `expires_at` field to `duration`.
* Make moderators choose from a fixed set of standard ban lengths,
  instead of allowing arbitrary ban lengths.
* List `duration` in seconds in the /bans.json API.
* Dump bans to BigQuery.

Note that some old bans have a negative duration. This is because their
expiration date was before their creation date, which is because in 2013
bans were migrated to Danbooru 2 and the original ban creation dates
were lost.
2021-03-11 02:59:58 -06:00
evazion
f235b72b3f Export public database dumps to BigQuery.
* Export daily public database dumps to BigQuery and Google Cloud Storage.
* Only data visible to anonymous users is exported. Some tables have
  null or missing fields because of this.
* The bans table is excluded because some bans have an expires_at
  timestamp set beyond year 9999, which BigQuery doesn't support.
* The favorites table is excluded because it's too slow to dump (it
  doesn't have an id index, which is needed by find_each).
* Version tables are excluded because dumping them every day is
  inefficient, streaming insertions should be used instead.

Links:

* https://console.cloud.google.com/bigquery?project=danbooru1
* https://console.cloud.google.com/storage/browser/danbooru_public
* https://storage.googleapis.com/danbooru_public/data/posts.json
2021-03-10 02:52:16 -06:00
nonamethanks
a123bf0ee8 Mastodon: fix strategy not working for direct or dead links 2021-03-09 22:35:36 +01:00
evazion
0249c290fd skeb: remove skeb from site_name in base strategy.
Fixup a mistake with the way the merge conflict was resolved in 9dd903d21.
2021-03-08 03:56:44 -06:00
evazion
9dd903d212 Merge branch 'master' into skeb 2021-03-08 03:43:15 -06:00
evazion
66af4986c6 Merge pull request #4754 from nonamethanks/lazymap
Optimize Sources::Strategies.find()
2021-03-08 03:40:28 -06:00
evazion
921dde6522 users: don't set inviter field; clear inviter field for most users.
* Don't set the inviter field for newly promoted users, or for Gold/Plat
  upgrades.

* Clear the inviter field for paid Gold/Plat upgrades, and for users who
  have a feedback or a modaction listing who invited them. This leaves
  about 600 remaining users with an inviter field with no other record
  of who invited them.

See #4750.
2021-03-08 03:16:34 -06:00
evazion
64440757c1 Merge pull request #4737 from nonamethanks/fix-mastodon-page-url
Sources: get correct mastodon page url
2021-03-08 01:49:56 -06:00
evazion
224d4932bf Merge pull request #4745 from nonamethanks/twitter-common-regexes
Twitter: update common hashtag regexes
2021-03-08 01:49:30 -06:00
evazion
e17cefe3ab Merge pull request #4753 from nonamethanks/fix-baraag-regex
Baraag: fix image regex
2021-03-08 01:49:15 -06:00
evazion
1716cc5bf9 artists: add more artist url icons. 2021-03-08 01:30:02 -06:00
evazion
869a99d9a3 nijie: clear session cookie if it's expired (#4665).
If we detect that the session cookie has expired (by the presence of the
`#login_illust` element on the page), then clear the cached session
cookie. The current source fetch will still fail, but the next fetch
will try to login again and hopefully succeed.
2021-03-08 01:30:02 -06:00
evazion
5e4865701c nijie: reduce login cookie cache duration to 60 minutes (#4665). 2021-03-08 01:30:02 -06:00
evazion
52d76a4f22 artists: fix error messages for invalid artist names.
* When trying to create an artist entry for a non-artist tag, set the
  error on the name attribute so that the artist name gets marked
  as incorrect in the artist edit form.

* Fix a bad `Name '' cannot be blank` error message when the artist name
  is blank.

* Fix showing wiki pages of non-artist tags in the artist edit form when
  the artist name conflicts with a non-artist tag (e.g. if you try to
  create an artist named '1girl', don't show the wiki for 1girl in the
  artist edit form).
2021-03-07 21:07:44 -06:00
nonamethanks
a4ae07416a Optimize Sources::Strategies.find()
Use lazy.map instead of map, so that we do not run every single
strategy's match method for every source
2021-03-07 18:20:10 +01:00