Calculate the dimensions of thumbnails ourselves instead of letting
libvips calculate them for us. This way we know the exact size of
thumbnails, so we can set the right width and height for <img> tags. If
we let libvips calculate thumbnail sizes for us, then we can't predict
the exact size of thumbnails, because sometimes libvips rounds numbers
differently than us.
The problem was that we were stripping color profiles from thumbnails,
but we weren't setting `export_profile: "srgb"` to convert images to
sRGB first. This resulted in wrong colors for images with non-sRGB color
profiles, such as Adobe RGB.
The fix is to convert images to sRGB when possible, while leaving CMYK
and greyscale images alone. We leave CMYK images alone because we can't
convert CMYK to sRGB without losing color. We leave greyscale images
alone if they don't have a color profile, that way they stay as
one-channel greyscale (or two-channel greyscale, in case of alpha)
instead of being converted to three-channel sRGB. However, if a
greyscale image has a color profile, then we have to convert to sRGB,
otherwise the colors would be wrong when we strip the profile.
We also have to set the import profile, otherwise images with broken
embedded color profiles won't have a fallback profile and may get
incorrect colors. In this case we also have to be careful, because we
can't specify an sRGB fallback for greyscale or CMYK images.
Upgrade bootsnap to 1.9.3 too because Ruby 3.0.3 has a bug that causes
Rails to fail to boot when bootsnap is enabled. Bootsnap 1.9.3 works
around this bug.
Also add libgmp to build with bignum support.
Fix an issue where if you tried to remove the same vote multiple times,
nothing would happen.
Specifically, if you upvoted a post in one tab, then opened it in a
second tab, then removed your vote in the first tab, then upvoted it
again in the first tab, then tried to remove your vote again in the
second tab, nothing would happen when removing it in the second tab.
This was because we were removing votes by vote ID, which meant that if
a vote had already been removed, removing it again would do nothing
since it was already deleted.
Fix a division by zero when calculating the upvote ratio in the votes tooltip.
This could happen if the post had votes according to the post_votes
table, but didn't have any votes according to the up_score and
down_score. This should never happen, yet it did happen for post 2248039
and post 2959269.
Make the `order:random` metatag truly randomize the search. Add a
`random:N` metatag that returns up to N random posts, like what
`order:random` did before.
`order:random` now returns the entire search in random order. Before it
just returned a pageful of pseudorandom posts. This will be more
accurate for small searches, but slower for large searches. If
`order:random` times out, try `random:N` instead.
The `random:N` metatag returns up to N pseudorandom posts. This is
faster than `order:random` for large searches, but for small searches,
it may return less than N posts, and the randomness may be biased. Some
posts may be more likely than others to appear. N must be between 0 and
200.
Also, `/posts?tags=touhou&random=1` now redirects to `/posts?tags=touhou+random:N`.
Before the `random=1` param acted like a free `order:random` tag; now it
redirects to a `random:N` search, so it counts against your tag limit.
Optimize metatag searches involving usernames, including user:,
approver:, appealer:, commenter:, upvoter:, etc.
Do `User.find_by_name` instead of `User.name_matches` because this
fetches the user upfront instead of doing it inside a subquery. Using a
subquery makes the SQL more complicated and leads to worse query plans.
This especially helps searches involving multiple username metatags.
Allow admins to remove votes on posts. This is for fixing vote abuse.
Votes can be removed by going to the vote list on the /post_votes page,
or by clicking on a post's score, then using the "Remove" option in the
"..." dropdown menu next to the vote.
Votes are soft-deleted - they're marked as deleted in the database, but
not fully deleted. Removed votes are only visible to admins, not to
regular users. When a vote is removed by an admin, it leaves a mod
action.
Technically it's possible to undelete votes, but there's no UI for it.
Optimize searches using the `search[user_name]=...` URL parameter. If
we're not doing a wildcard search, then do a regular user lookup, which
generates better SQL.
* Add a gap between thumbnails on mobile.
* Adjust CSS for scores and vote buttons.
* Include "Private favorites" as an incentive on the user upgrade page.
* Fix vote buttons not being visible beneath thumbnails on mobile.
* Fix the "Show scores" link not preserving the current page number.
* Fix vote buttons being unintentionally enabled for all thumbnails by default.
* Fix banned and restricted users being able to favorite posts by
tagging them with `fav:self`.
* Fix search engines being able to crawl /posts?view=score pages.
* Fix broken tests.
Changes:
* Make it so you can click or hover over a post's favorite count to see
the list of public favorites.
* Remove the "Show »" button next to the favorite count.
* Make the favorites list visible to all users. Before favorites were
only visible to Gold users.
* Make the /favorites page show the list of all public favorites,
instead of redirecting to the current user's favorites.
* Add /posts/:id/favorites endpoint.
* Add /users/:id/favorites endpoint.
This is for several reasons:
* To make viewing favorites work the same way as viewing upvotes.
* To make posts load faster for Gold users. Before, we loaded all the
favorites when viewing a post, even when the user didn't look at them.
This made pageloads slower for posts that had hundreds or thousands of
favorites. Now we only load the favlist if the user hovers over the favcount.
* To make the favorite list visible to all users. Before, it wasn't
visible to non-Gold users, because of the performance issue listed above.
* To make it more obvious that favorites are public by default. Before,
since regular users could only see the favcount, they may have
mistakenly believed other users couldn't see their favorites.
* only show the [+] and [-] feedback buttons to gold users
* only show the [promote] button to moderators, and only when targetting users
builder or below
* don't show either button if the target is the current user
* only show the [+] and [-] feedback buttons to gold users
* only show the [promote] button to moderators, and only when targetting users
builder or below
* don't show either button if the target is the current user
Add upvote and downvote buttons beneath thumbnails on the post index page.
This is disabled by default. To enable it, click the "..." menu in the top
right of the page, then click "Show scores".
This is currently a per-search setting, not an account setting. If you
enable it in one tab, it won't be enabled in other tabs.
Make private favgroups a Gold-only option. This is for consistency with
private favorites and upvotes being Gold-only options.
Existing Members with private favgroups are allowed to keep them, as
long as they don't disable privacy. If they disable it, then they can't
re-enable it again without upgrading to Gold first.
Make private favorites and upvotes a Gold-only account option.
Existing Members with private favorites enabled are allowed to keep it
enabled, as long as they don't disable it. If they disable it, then they
can't re-enable it again without upgrading to Gold first.
This is a Gold-only option to prevent uploaders from creating multiple
accounts to upvote their own posts. If private upvotes were allowed for
Members, then it would be too easy to use fake accounts and private
upvotes to upvote your own posts.
* Allow Member-level users to vote.
* Don't allow Banned or Restricted users to create favorites any more.
Banned and Restricted users aren't allowed to upvote or favorite any
more to prevent sockpuppet accounts from upvoting even after they're
banned.
Make upvotes public the same way favorites are public:
* Rename the "Private favorites" account setting to "Private favorites and upvotes".
* Make upvotes public, unless the user has private upvotes enabled. Note
that private upvotes are still visible to admins. Downvotes are still
hidden to everyone except for admins.
* Make https://danbooru.donmai.us/post_votes visible to all users. This
page shows all public upvotes. Private upvotes and downvotes are only
visible on the page to admins and to the voter themselves.
* Make votes searchable with the `upvote:username` and `downvote:username`
metatags. These already existed before, but they were only usable by
admins and by people searching for their own votes.
Upvotes are public to discourage users from upvoting with multiple
accounts. Upvote abuse is obvious to everyone when upvotes are public.
The other reason is to make upvotes consistent with favorites, which are
already public.
Add `upvotes:N`, `downvotes:N`, `order:upvotes`, `order:downvotes`,
`order:upvotes_asc`, `order:downvotes_asc` metatags.
In the API, the field is called up_score / down_score. Here it's called
`upvotes` and `downvotes` because this should be easier to understand
for end users.
Note that internally, `down_score` is negative. A post that matches
`downvotes:>5` will have down_score < -5 internally.
Fix certain artist commentaries for foundation.app containing scrambled
characters. Apparently caused by the Nokogiri HTML5 parser not handling
UTF-8 input correctly when the encoding isn't explicitly set to UTF-8.
* Fix /proc not being mounted read-only.
* Fix other read-only directories not actually being mounted read-only.
This was because the mount(2) system call ignores mount flags when
creating a bind mount. The solution is to bind mount the directory
first, then remount it as read-only second.
Known bug: submounts still don't get mounted as read-only. That is, if
we're mounting /usr as read-only, and /usr has a submount of /usr/local,
then /usr/local won't get mounted as read-only.
Fix thumbnails incorrectly showing a duration of "0:00" when a video had
a duration between 59.5 seconds and 60.0 seconds. This happened because
of incorrect rounding - the seconds value was rounded up, but the
minutes value wasn't.
Examples: https://danbooru.donmai.us/posts?tags=duration:59.5...60.0+status:any
Add a Sandbox class for running untrusted external programs like ffmpeg
or exiftool inside a sandbox. This uses Linux namespaces to run the
process in an isolated container, much like a Docker container. Unlike a
Docker container, we can use it to sandbox programs when Danbooru itself
is already running inside a Docker container.
This is also more restrictive than Docker in several ways:
* It has a system call filter that is more restrictive and more
customizable than Docker's filter by default. Even if the process
breaks out of the container, the syscall filter will limit what it can
do, even if it escalates to root.
* It blocks the use of setuid binaries, so the process can't use things
like sudo to escalate to root inside the sandbox.
* It blocks all network access inside the sandbox by default.
* All files in the container are read-only by default. The sandboxed
process can only communicate by writing to stdout.
See app/logical/sandbox.rb for more details.
This isn't actually enabled yet. It will be rolled out progressively to
ensure it doesn't break things.
Block Googlebot from crawling certain slow useless URLs. Sometimes
Googlebot tries to crawl old source:<url>, approver:<name>, and
ordfav:<name> searches in bulk, which tends to slow down the site because
things like source:<url> are inherently slow, and because Google spends
hours at a time crawling them in parallel. This is despite the fact that
these links are already marked as nofollow and noindex, and source:<url>
links were removed from posts a long time ago to try to stop Google from
crawling them.
Add a Ruby wrapper library around the libseccomp library. Seccomp is
used to restrict the syscalls a program can make. See comments in
app/logical/seccomp.rb for further details.
This is not used for anything yet. It's simply adding part of the
sandboxing infrastructure for later use.
Send all logs to stderr by default instead of stdout. Fixes a problem
where parsing the output of sandboxed commands could fail, because they
could contain Rails log messages in their stdout.
When we run a command in a sandbox, we call fork+exec to run the command
in the background so we can capture its output. If Rails prints
anything to stdout between the fork and exec calls, then it will be
inadvertently captured along with the command's output. This will break
parsing of the command's output. This can happen if warning messages are
printed by Rails while setting up the sandbox between the fork and exec
calls.
Writing to stderr is also more correct, since stdout is buffered by
default, which means logs could potentially be lost if the process dies
unexpectedly before the buffers are flushed. Stderr is unbuffered by
default, which means logs will always be output immediately.
