Commit Graph
10540 Commits
Author SHA1 Message Date
evazion d03b150180 BURs: fix tag nukes not removing antecedent implications.
Fix a bug where if A implied B, and A was nuked, then the A->B
implication wasn't removed.
2021-09-06 03:25:03 -05:00
evazion 4dcfd1d141 aliases/implications: log manual deletions by admins.
Log when an admin manually deletes an alias or implication outside of a
BUR. This is usually only necessary when a BUR is bugged.
2021-09-06 03:25:02 -05:00
evazion 28edd5a22a emails: hardcode nondisposable email list.
Hardcode the list of nondisposable email providers instead of making it
a config option. Also add a few new providers.

This was previously a config option to keep it secret, but there's not
much need for secrecy here.

A Restricted user's email must be on this list to unrestrict their
account. If a user is Restricted and their email is not in this list,
then it's assumed to be disposable and can't be used to unrestrict their
account even if they verify their email address.
2021-09-06 03:24:53 -05:00
evazion 19f01d4554 emails: update canonical domains list. 2021-09-05 17:56:24 -05:00
evazion d7d3439d79 ffmpeg: generate smart previews as .png instead of .jpg
Generate smart previews as .png so we don't suffer recompression losses
when we convert the preview frame from a full size image down to a
150x150 .jpg thumbnail.
2021-09-05 07:53:12 -05:00
evazion 13f98c02e3 media file: fix overly large thumbnails for animated GIFs.
Fix regression in ef2857667 that caused animated GIFs and PNGs to
generate thumbnails that were larger than 150x150.

Also fix a bug with cropped previews not being generated for animated
GIFs and PNGs.
2021-09-05 07:53:00 -05:00
evazion 540a3e111a Replace streamio-ffmpeg library.
Replace the streamio-ffmpeg library with our own very thin FFmpeg wrapper.
2021-09-05 06:54:56 -05:00
evazion ef28576673 Fix #3400: Smarter thumbnail generation for videos 2021-09-05 06:10:18 -05:00
evazion 52847e4ce9 Update README.md.
Remove some outdated information and link to the install guide in the
wiki.
2021-09-04 07:06:58 -05:00
evazion e240c92d6f github: update docker build workflow to use cache.
Update the Docker build workflow to use premade actions so we can cache
the build.

https://github.com/docker/setup-buildx-action
https://github.com/docker/login-action
https://github.com/docker/metadata-action
https://github.com/docker/build-push-action
2021-09-04 04:07:53 -05:00
evazion 207a1b7688 db/structure.sql: fix syntax error. 2021-09-04 03:59:35 -05:00
evazion 8aa7608816 github: add docker build workflow action.
Add an action to build a Docker image and push it to the Github Packages
registry.
2021-09-03 23:23:00 -05:00
evazion 8b85bbe8ea storage manager: remove 'hybrid' and 'match' manager.
Remove StorageManager::Hybrid and StorageManager::Match. These were used
to store uploads on different servers based on the post ID or file
sample type. This is no longer used in production because in hindsight
it's a lot more difficult to manage uploads when they're fragmented
across different servers.

If you need this, you can do tricks with network filesystems to get the
same effect. For example, if you want to store some files on server A
and others on server B, then mount servers A and B as network
filesystems (with e.g. sshfs, Samba, NFS, etc), and use symlinks to
point subdirectories at either server A or B.
2021-09-03 22:45:27 -05:00
evazion 440d2c807f posts: fix JS errors on non-flash posts when using Ruffle extension.
Fix https://github.com/danbooru/danbooru/issues/4783#issuecomment-912426739:

    Spoke too soon. It works in flash posts, but not in non-flash posts.

    Here's the full trace that happens by trying the Shift-E shortcut
    (tested on https://danbooru.donmai.us/posts/4749304 with the latest
    ruffle for chrome at https://ruffle.rs/#downloads):

    4749304:7 Uncaught TypeError: Cannot read properties of undefined (reading 'appendChild')
        at Object.initialize_ruffle_player (application-237cebbe071cb34ce095.js:1)
        at Object.initialize_all (application-237cebbe071cb34ce095.js:1)
        at HTMLDocument.<anonymous> (application-237cebbe071cb34ce095.js:1)
        at f (857-ff5115a2d6ceb23b4291.js:2)
        at p (857-ff5115a2d6ceb23b4291.js:2)
        at nrWrapper (4749304?q=status%3Aany:7)

Caused by trying to initialize the Ruffle player on non-Flash posts,
because we checked for the existence of the `window.RufflePlayer` object
to tell if we were on a Flash post, but when using the Ruffle browser
extension the RufflePlayer object will always exist.
2021-09-03 05:33:11 -05:00
evazion d441739ab2 Fix #4802: Ruffle conflicts with danbooru JS
Caused by Webpack clobbering the `Danbooru` variable for unknown reasons
when the app/javascript/packs/flash.js pack is loaded. Disabling the
output.library option seems to fix it.
2021-09-03 03:50:52 -05:00
evazion b068c113a8 Add MediaAsset model.
A MediaAsset represents an image or video file uploaded to Danbooru. It
stores the metadata associated with the image or video. This is to work
on decoupling files from posts so that images can be uploaded separately
from posts.
2021-09-02 06:07:52 -05:00
evazion 2d976cf557 posts: remove has_dimensions? method.
All posts have non-null width and height now that unsupported filetypes
have been purged.

https://danbooru.donmai.us/forum_topics/18027
2021-09-02 05:58:45 -05:00
evazion c29539d44e /posts/random: fix reorder(nil).first deprecation warning
DEPRECATION WARNING: `.reorder(nil)` with `.first` / `.first!` no
    longer takes non-deterministic result in Rails 6.2. To continue
    taking non-deterministic result, use `.take` / `.take!` instead.
    (called from random at /home/user/src/danbooru/app/controllers/posts_controller.rb:91)
2021-09-02 04:09:03 -05:00
evazion 1bb383703d iqdb: allow searching images by iqdb hash.
Example:

    https://danbooru.donmai.us/iqdb_queries?search[hash]=iqdb_3fe4c0bd5a88fab53fbe5104efdc43ba3fa094416e6e039cf880f9f1fafafafffc80fd7bfd7dfd80fe72fe79fe80fefafefdfeffff75ff79ff7aff7dfff1fffb000100020003000400080082008a00940184018802000287028c028f030004000600078308001080ef80f800f8fdf900fafdfc00fc7dfcfafe00fe72fefdff7effdffff0fff8fffcfffd0007000f001f003e0080008f0106018d02800282038003810403040806800683070b078007870d800d830f801f00f800f96cfa76fb00fb7efbf4fc00fcfdfd00fd74fdfbfe78fe7efef5fef9fefbfefeff6cff76ff7efffcfffe00070080008300860087008b01030106018002820283028503800402040a0502058b0d80

The hash may be obtained from a regular IQDB search, or by calculating
it yourself (an exercise for the reader).
2021-09-02 03:37:36 -05:00
evazion ed600f4829 iqdb: fix non-jpeg files not working with direct file upload.
Convert non-JPEGs to JPEG before sending them to IQDB.
2021-09-02 03:16:04 -05:00
evazion 80bf54205c Fix post order is no longer randomized when sorting by random (#4870).
Random posts are now returned in descending order by ID when sorting by
    random, they used to be returned in a random order.
2021-09-02 02:04:08 -05:00
evazion f198a52f9e nokogiri: use bundled libxml2.
Make nokogiri use the bundled version of libxml2 instead of the system
version. In the past installing nokogiri was slow because it had to
compile the bundled version of libxml2, which is partly why we switched
to the system library. Now it's faster because the bundled version comes
pre-compiled with the nokogiri gem.

https://nokogiri.org/#native-gems-faster-more-reliable-installation

Reverts 440bbbb28.
2021-09-02 01:58:11 -05:00
evazion 19c0027d1f hentai foundry: fix 'Document tree depth exceeded' when parsing commentaries.
Fix a regression in 38c9559fe that caused #4657 to fail again.
2021-09-01 01:40:01 -05:00
evazion d00aa847ae search: allow mods to search disapproved:<user> for other users.
Allow moderators to search `disapproved:<username>` with any user.
Before mods could only search for their own disapprovals, even though
they could see disapprovals by others.
2021-09-01 01:39:14 -05:00
evazion c41e3f4590 saved searches: fix exploit allowing flaggers to be determined.
Fix an exploit that let you determine the flagger of a post using
`flagger:<username>` saved searches. Saved searches were performed as
DanbooruBot, but since DanbooruBot is a moderator, it let unprivileged
users do `flagger:<username>` searches. Saved searches were done as a
moderator to avoid tag limits, but this is no longer necessary since the
last PostQueryBuilder refactor.

fred get out
2021-09-01 00:55:19 -05:00
evazion 88e379f9cc Update DText gem. 2021-08-31 21:48:53 -05:00
evazion 374298a743 Fix #4853: Users should not be able to search by disapprover 2021-08-31 21:11:07 -05:00
evazion 38c9559fe8 nokogiri: switch to the nokogumbo-based html5 parser.
https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md#1120--2021-08-02
2021-08-30 21:21:27 -05:00
evazion 49d18e64e8 Fix #4869: "Random" button raises exception when viewing ordfav.
Fix exception during https://danbooru.donmai.us/posts/random?tags=ordfav:nonamethanks

Before we were doing a query like this:

    SELECT
      "posts".*
    FROM
      "posts"
    INNER JOIN
      "favorites" ON "favorites"."post_id" = "posts"."id"
    WHERE
      (favorites.user_id % 100 = 64 AND favorites.user_id = 52664)
      AND "posts"."id" = 343894
    ORDER BY
      favorites.id DESC,
      posts.id DESC,
      ID=343894 DESC

but `ID=? DESC` is ambiguous during an ordfav: search because of the
join on the favorites table. The fix is to qualify the reference as
`posts.id`.
2021-08-30 16:46:03 -05:00
evazion 1e5c7d6f0f Fix #4867: random=true in api only returns one post.
Pundit 2.1.1 changed it so that if the first argument to `authorize` is
an Array, then the `authorize` call returns the last element of the
array. This broke order:random, because in that case we returned an
Array of posts. The fix is to return an ActiveRecord::Relation of posts,
which is more correct anyway.
2021-08-29 22:37:16 -05:00
evazion 46c1b2c37d Fix #4868: undefined method `to_string' errors in /user_events search
Fix a regression introduced in rails/rails@4b1122c with the upgrade to Rails
6.1.4.1.

Triggered by a call to `SELECT * FROM ip_geolocations WHERE ip_addr in ...`.
A Rails refactoring changed the way that `WHERE ... IN ...` statements
worked, which had the side effect of passing a string value to our
IpAddressType serializer where before we expected a Danbooru::IpAddress
object.
2021-08-29 17:46:04 -05:00
evazion d7cc844bd9 rake: add task for reindexing posts in iqdb. 2021-08-29 03:36:16 -05:00
evazion 34861678cb Fix #4661: Change shortcut for submitting uploads from Enter to Ctrl+Enter
Also fix #4734.
2021-08-28 04:53:33 -05:00
evazion 38edbb0512 Update Yarn packages. 2021-08-28 04:53:33 -05:00
evazion 3348e1000c Update Ruby gems. 2021-08-28 04:53:33 -05:00
evazion a3587c30b2 Fix broken tests. 2021-08-28 04:53:33 -05:00
evazion bb7f24d279 Add HTTP proxy support.
Add support for using a proxy for HTTP requests. Only used for external
requests, such as downloading files or talking to source sites such as
Pixiv or Twitter, not for internal requests, such as talking to IQDB or
Reportbooru.
2021-08-28 04:53:33 -05:00
evazion 349bf14764 Merge pull request #4864 from nottalulah/patch-1
Fix ugoira regenerations
2021-08-28 02:04:52 -05:00
evazion 1f02e1b0e7 Merge pull request #4865 from nonamethanks/fix-url-form-length
Fix url fields in forms not having the same length as other text inputs
2021-08-28 02:04:39 -05:00
nonamethanks 08234c496d Fix url fields in forms not having the same length as other text inputs 2021-08-26 13:32:16 +02:00
Lily 54ce73145d Fix ugoira regenerations
Currently ugoira with broken samples can't be regenerated, since the PixivUgoiraFrameData object is passed rather than the actual data itself (.data).
2021-08-24 20:27:40 -03:00
evazion ec0fce34b7 Fix #4863: Retire deletion appeal thread. 2021-08-21 23:48:20 -05:00
evazion c8d7f94d7e Update Ruffle version. 2021-08-21 04:01:24 -05:00
evazion 826736caaf replacements: fix updater in replacement comments.
Fix the "<User> replaced this post with a new file" comment saying the
comment was edited by the replacer.
2021-08-21 03:49:06 -05:00
evazion 38ca30d6fc Merge pull request #4862 from GlassedSilver/patch-1
Reflect that SQS is not needed anymore for IQDB
2021-08-21 02:44:31 -05:00
GlassedSilver 09eeb60337 Reflect that SQS is not needed anymore for IQDB
As discussed here: https://github.com/danbooru/danbooru/issues/4198
2021-08-19 05:16:49 +02:00
evazion b4dc7487ee BURs: reduce autorejection timeout from 60 days to 45 days. 2021-08-15 04:51:48 -05:00
evazion 8f24e789b6 newrelic: fix crash during bootup caused by Rails.logger.
Using `Rails.logger` here causes server boot to fail with a `Undefined
method 'tagged'` error, possibly because `Rails.logger` isn't ready yet
during early initialization.
2021-08-15 02:16:57 -05:00
nonamethanks 0ddac45160 Mass updates: make the destination search clickable 2021-08-15 02:16:57 -05:00
nonamethanks f60fce614b Fix lofter strategy due to changes in their image urls 2021-08-15 02:16:57 -05:00