Commit Graph

10534 Commits

Author SHA1 Message Date
evazion
540a3e111a Replace streamio-ffmpeg library.
Replace the streamio-ffmpeg library with our own very thin FFmpeg wrapper.
2021-09-05 06:54:56 -05:00
evazion
ef28576673 Fix #3400: Smarter thumbnail generation for videos 2021-09-05 06:10:18 -05:00
evazion
52847e4ce9 Update README.md.
Remove some outdated information and link to the install guide in the
wiki.
2021-09-04 07:06:58 -05:00
evazion
e240c92d6f github: update docker build workflow to use cache.
Update the Docker build workflow to use premade actions so we can cache
the build.

https://github.com/docker/setup-buildx-action
https://github.com/docker/login-action
https://github.com/docker/metadata-action
https://github.com/docker/build-push-action
2021-09-04 04:07:53 -05:00
evazion
207a1b7688 db/structure.sql: fix syntax error. 2021-09-04 03:59:35 -05:00
evazion
8aa7608816 github: add docker build workflow action.
Add an action to build a Docker image and push it to the Github Packages
registry.
2021-09-03 23:23:00 -05:00
evazion
8b85bbe8ea storage manager: remove 'hybrid' and 'match' manager.
Remove StorageManager::Hybrid and StorageManager::Match. These were used
to store uploads on different servers based on the post ID or file
sample type. This is no longer used in production because in hindsight
it's a lot more difficult to manage uploads when they're fragmented
across different servers.

If you need this, you can do tricks with network filesystems to get the
same effect. For example, if you want to store some files on server A
and others on server B, then mount servers A and B as network
filesystems (with e.g. sshfs, Samba, NFS, etc), and use symlinks to
point subdirectories at either server A or B.
2021-09-03 22:45:27 -05:00
evazion
440d2c807f posts: fix JS errors on non-flash posts when using Ruffle extension.
Fix https://github.com/danbooru/danbooru/issues/4783#issuecomment-912426739:

    Spoke too soon. It works in flash posts, but not in non-flash posts.

    Here's the full trace that happens by trying the Shift-E shortcut
    (tested on https://danbooru.donmai.us/posts/4749304 with the latest
    ruffle for chrome at https://ruffle.rs/#downloads):

    4749304:7 Uncaught TypeError: Cannot read properties of undefined (reading 'appendChild')
        at Object.initialize_ruffle_player (application-237cebbe071cb34ce095.js:1)
        at Object.initialize_all (application-237cebbe071cb34ce095.js:1)
        at HTMLDocument.<anonymous> (application-237cebbe071cb34ce095.js:1)
        at f (857-ff5115a2d6ceb23b4291.js:2)
        at p (857-ff5115a2d6ceb23b4291.js:2)
        at nrWrapper (4749304?q=status%3Aany:7)

Caused by trying to initialize the Ruffle player on non-Flash posts,
because we checked for the existence of the `window.RufflePlayer` object
to tell if we were on a Flash post, but when using the Ruffle browser
extension the RufflePlayer object will always exist.
2021-09-03 05:33:11 -05:00
evazion
d441739ab2 Fix #4802: Ruffle conflicts with danbooru JS
Caused by Webpack clobbering the `Danbooru` variable for unknown reasons
when the app/javascript/packs/flash.js pack is loaded. Disabling the
output.library option seems to fix it.
2021-09-03 03:50:52 -05:00
evazion
b068c113a8 Add MediaAsset model.
A MediaAsset represents an image or video file uploaded to Danbooru. It
stores the metadata associated with the image or video. This is to work
on decoupling files from posts so that images can be uploaded separately
from posts.
2021-09-02 06:07:52 -05:00
evazion
2d976cf557 posts: remove has_dimensions? method.
All posts have non-null width and height now that unsupported filetypes
have been purged.

https://danbooru.donmai.us/forum_topics/18027
2021-09-02 05:58:45 -05:00
evazion
c29539d44e /posts/random: fix reorder(nil).first deprecation warning
DEPRECATION WARNING: `.reorder(nil)` with `.first` / `.first!` no
    longer takes non-deterministic result in Rails 6.2. To continue
    taking non-deterministic result, use `.take` / `.take!` instead.
    (called from random at /home/user/src/danbooru/app/controllers/posts_controller.rb:91)
2021-09-02 04:09:03 -05:00
evazion
1bb383703d iqdb: allow searching images by iqdb hash.
Example:

    https://danbooru.donmai.us/iqdb_queries?search[hash]=iqdb_3fe4c0bd5a88fab53fbe5104efdc43ba3fa094416e6e039cf880f9f1fafafafffc80fd7bfd7dfd80fe72fe79fe80fefafefdfeffff75ff79ff7aff7dfff1fffb000100020003000400080082008a00940184018802000287028c028f030004000600078308001080ef80f800f8fdf900fafdfc00fc7dfcfafe00fe72fefdff7effdffff0fff8fffcfffd0007000f001f003e0080008f0106018d02800282038003810403040806800683070b078007870d800d830f801f00f800f96cfa76fb00fb7efbf4fc00fcfdfd00fd74fdfbfe78fe7efef5fef9fefbfefeff6cff76ff7efffcfffe00070080008300860087008b01030106018002820283028503800402040a0502058b0d80

The hash may be obtained from a regular IQDB search, or by calculating
it yourself (an exercise for the reader).
2021-09-02 03:37:36 -05:00
evazion
ed600f4829 iqdb: fix non-jpeg files not working with direct file upload.
Convert non-JPEGs to JPEG before sending them to IQDB.
2021-09-02 03:16:04 -05:00
evazion
80bf54205c Fix post order is no longer randomized when sorting by random (#4870).
Random posts are now returned in descending order by ID when sorting by
    random, they used to be returned in a random order.
2021-09-02 02:04:08 -05:00
evazion
f198a52f9e nokogiri: use bundled libxml2.
Make nokogiri use the bundled version of libxml2 instead of the system
version. In the past installing nokogiri was slow because it had to
compile the bundled version of libxml2, which is partly why we switched
to the system library. Now it's faster because the bundled version comes
pre-compiled with the nokogiri gem.

https://nokogiri.org/#native-gems-faster-more-reliable-installation

Reverts 440bbbb28.
2021-09-02 01:58:11 -05:00
evazion
19c0027d1f hentai foundry: fix 'Document tree depth exceeded' when parsing commentaries.
Fix a regression in 38c9559fe that caused #4657 to fail again.
2021-09-01 01:40:01 -05:00
evazion
d00aa847ae search: allow mods to search disapproved:<user> for other users.
Allow moderators to search `disapproved:<username>` with any user.
Before mods could only search for their own disapprovals, even though
they could see disapprovals by others.
2021-09-01 01:39:14 -05:00
evazion
c41e3f4590 saved searches: fix exploit allowing flaggers to be determined.
Fix an exploit that let you determine the flagger of a post using
`flagger:<username>` saved searches. Saved searches were performed as
DanbooruBot, but since DanbooruBot is a moderator, it let unprivileged
users do `flagger:<username>` searches. Saved searches were done as a
moderator to avoid tag limits, but this is no longer necessary since the
last PostQueryBuilder refactor.

fred get out
2021-09-01 00:55:19 -05:00
evazion
88e379f9cc Update DText gem. 2021-08-31 21:48:53 -05:00
evazion
374298a743 Fix #4853: Users should not be able to search by disapprover 2021-08-31 21:11:07 -05:00
evazion
38c9559fe8 nokogiri: switch to the nokogumbo-based html5 parser.
https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md#1120--2021-08-02
2021-08-30 21:21:27 -05:00
evazion
49d18e64e8 Fix #4869: "Random" button raises exception when viewing ordfav.
Fix exception during https://danbooru.donmai.us/posts/random?tags=ordfav:nonamethanks

Before we were doing a query like this:

    SELECT
      "posts".*
    FROM
      "posts"
    INNER JOIN
      "favorites" ON "favorites"."post_id" = "posts"."id"
    WHERE
      (favorites.user_id % 100 = 64 AND favorites.user_id = 52664)
      AND "posts"."id" = 343894
    ORDER BY
      favorites.id DESC,
      posts.id DESC,
      ID=343894 DESC

but `ID=? DESC` is ambiguous during an ordfav: search because of the
join on the favorites table. The fix is to qualify the reference as
`posts.id`.
2021-08-30 16:46:03 -05:00
evazion
1e5c7d6f0f Fix #4867: random=true in api only returns one post.
Pundit 2.1.1 changed it so that if the first argument to `authorize` is
an Array, then the `authorize` call returns the last element of the
array. This broke order:random, because in that case we returned an
Array of posts. The fix is to return an ActiveRecord::Relation of posts,
which is more correct anyway.
2021-08-29 22:37:16 -05:00
evazion
46c1b2c37d Fix #4868: undefined method `to_string' errors in /user_events search
Fix a regression introduced in rails/rails@4b1122c with the upgrade to Rails
6.1.4.1.

Triggered by a call to `SELECT * FROM ip_geolocations WHERE ip_addr in ...`.
A Rails refactoring changed the way that `WHERE ... IN ...` statements
worked, which had the side effect of passing a string value to our
IpAddressType serializer where before we expected a Danbooru::IpAddress
object.
2021-08-29 17:46:04 -05:00
evazion
d7cc844bd9 rake: add task for reindexing posts in iqdb. 2021-08-29 03:36:16 -05:00
evazion
34861678cb Fix #4661: Change shortcut for submitting uploads from Enter to Ctrl+Enter
Also fix #4734.
2021-08-28 04:53:33 -05:00
evazion
38edbb0512 Update Yarn packages. 2021-08-28 04:53:33 -05:00
evazion
3348e1000c Update Ruby gems. 2021-08-28 04:53:33 -05:00
evazion
a3587c30b2 Fix broken tests. 2021-08-28 04:53:33 -05:00
evazion
bb7f24d279 Add HTTP proxy support.
Add support for using a proxy for HTTP requests. Only used for external
requests, such as downloading files or talking to source sites such as
Pixiv or Twitter, not for internal requests, such as talking to IQDB or
Reportbooru.
2021-08-28 04:53:33 -05:00
evazion
349bf14764 Merge pull request #4864 from nottalulah/patch-1
Fix ugoira regenerations
2021-08-28 02:04:52 -05:00
evazion
1f02e1b0e7 Merge pull request #4865 from nonamethanks/fix-url-form-length
Fix url fields in forms not having the same length as other text inputs
2021-08-28 02:04:39 -05:00
nonamethanks
08234c496d Fix url fields in forms not having the same length as other text inputs 2021-08-26 13:32:16 +02:00
Lily
54ce73145d Fix ugoira regenerations
Currently ugoira with broken samples can't be regenerated, since the PixivUgoiraFrameData object is passed rather than the actual data itself (.data).
2021-08-24 20:27:40 -03:00
evazion
ec0fce34b7 Fix #4863: Retire deletion appeal thread. 2021-08-21 23:48:20 -05:00
evazion
c8d7f94d7e Update Ruffle version. 2021-08-21 04:01:24 -05:00
evazion
826736caaf replacements: fix updater in replacement comments.
Fix the "<User> replaced this post with a new file" comment saying the
comment was edited by the replacer.
2021-08-21 03:49:06 -05:00
evazion
38ca30d6fc Merge pull request #4862 from GlassedSilver/patch-1
Reflect that SQS is not needed anymore for IQDB
2021-08-21 02:44:31 -05:00
GlassedSilver
09eeb60337 Reflect that SQS is not needed anymore for IQDB
As discussed here: https://github.com/danbooru/danbooru/issues/4198
2021-08-19 05:16:49 +02:00
evazion
b4dc7487ee BURs: reduce autorejection timeout from 60 days to 45 days. 2021-08-15 04:51:48 -05:00
evazion
8f24e789b6 newrelic: fix crash during bootup caused by Rails.logger.
Using `Rails.logger` here causes server boot to fail with a `Undefined
method 'tagged'` error, possibly because `Rails.logger` isn't ready yet
during early initialization.
2021-08-15 02:16:57 -05:00
nonamethanks
0ddac45160 Mass updates: make the destination search clickable 2021-08-15 02:16:57 -05:00
nonamethanks
f60fce614b Fix lofter strategy due to changes in their image urls 2021-08-15 02:16:57 -05:00
Seedgou
35c9eff5bc fix ruffle-mirror version 2021-08-15 02:16:57 -05:00
nonamethanks
2a13667422 Tag nuke: remove implications to a tag before nuking it 2021-08-15 02:16:57 -05:00
nonamethanks
539af4421d IQDB direct url lookup: send the thumbnail instead of the full size 2021-08-15 02:16:57 -05:00
nonamethanks
d469b87ae2 Fix link to rest of user's post disapprovals 2021-08-15 02:16:57 -05:00
nonamethanks
bb4bdefc23 Lofter: add support for another theme 2021-08-15 02:16:56 -05:00
nonamethanks
716b9a5b88 Add utility links to user events from user page 2021-08-15 02:16:56 -05:00