Commit Graph

3250 Commits

Author SHA1 Message Date
evazion
f11c46b4f8 uploads: stop pruning uploads. 2022-01-28 04:13:22 -06:00
evazion
c4775d96a9 uploads: add upload_media_assets table.
Add a join table that allows multiple media assets (images or videos) to
be attached to uploads. This is for a future ability to upload multiple
files at once.
2022-01-25 16:10:53 -06:00
evazion
56722df753 forum: delete posts when topic is deleted.
Fix it so that when a forum topic is deleted, all posts in the topic are
deleted too. Also make it so that when a forum topic is undeleted, all
posts in it are undeleted too.

Before when a topic was deleted, only the topic itself was marked as
deleted, not the posts inside the topic. This meant that when a spam
topic was deleted, the OP wouldn't be marked as deleted, so any
modreports against it wouldn't be marked as handled.

Also change it so that it's not possible to undelete a post in a deleted
topic, or to delete the OP of a topic without deleting the topic itself.

Finally, add a fix script to delete all active posts in deleted topics,
and to undelete all deleted OPs in active topics.
2022-01-21 22:35:20 -06:00
evazion
5fd0d498a4 modreports: log modaction when report is handled or rejected. 2022-01-20 21:28:29 -06:00
evazion
c8d27c2719 Fix #4669: Track moderation report status.
* Add ability to mark moderation reports as 'handled' or 'rejected'.
* Automatically mark reports as handled when the comment or forum post
  is deleted.
* Send a dmail to the reporter when their report is handled.
* Don't show the report notice on comments or forum posts when all
  reports against it have been handled or rejected.
* Add a fix script to mark all existing reports for deleted comments,
  forum posts, or dmails as handled.
2022-01-20 20:50:23 -06:00
evazion
02c9498860 artists: normalize group names.
Normalize artist group names following the same rules as artist other names.

This means artist group names now use underscores instead of spaces.
It also means extra space characters at the beginning and end of names
is stripped, and Unicode characters are normalized.

Fixes #4647, which was caused by users accidentally replacing group
names with a single space character when trying to remove a group.
2022-01-20 00:17:06 -06:00
evazion
b9d4be16c8 Fix #4609: Searching for users failed flags fails. 2022-01-19 22:56:51 -06:00
evazion
acf565be7b Fix #4678: Validate custom CSS.
* Make it an error to add invalid custom CSS to your account.
* Add a fix script to remove custom CSS from all accounts with invalid CSS.
2022-01-15 23:20:49 -06:00
evazion
8a4faf7741 Fix #4960: Cosplay wikis link to char:x instead of just x. 2022-01-15 21:30:32 -06:00
evazion
5bec0aa147 Fix #4629: Tag renames/aliases should also modify pool descriptions. 2022-01-15 20:34:08 -06:00
evazion
33103f6dc4 pools: add ability to search for pools linking to given tag.
Add ability to search for pools linking to a given tag in the pool
description. Example:

    https://danbooru.donmai.us/pools?search[linked_to]=touhou

(This isn't actually exposed in the UI to avoid cluttering the pool
search form with rarely used options.)

Pools with broken links can be found here:

    https://danbooru.donmai.us/dtext_links?search[has_linked_tag]=No&search[has_linked_wiki]=No&search[model_type]=Pool

Lays the groundwork for fixing #4629.
2022-01-15 20:26:30 -06:00
evazion
c3c4f5a2a7 Fix #4957: Autotag non-web_source.
Autotag non-web_source on posts that have a non-http:// or https:// URL.
Add a fix script to backfill old posts.

Syntactically invalid URLs are still considered web sources. For
example, `https://google,com` technically isn't a valid URL, but it's
not considered a non-web source.
2022-01-14 22:58:27 -06:00
evazion
40d6351249 Fix #4940: DanbooruBot retiring banned_artist implications. 2022-01-11 11:53:35 -06:00
evazion
33828ec8a4 posts: remove set_tag_string method. 2022-01-11 10:06:46 -06:00
evazion
2e1c7ce6d3 Fix #4951: chartags:0 returning posts with chartags.
* Add fix script to fix posts with incorrect tag_count_* fields.
* Simplify the code for updating tag_count_* fields (no functional change).
2022-01-10 13:33:56 -06:00
evazion
c09cd9e9fd users: fix incorrect count columns on users table.
Fix incorrect post_upload_count, note_update_count, and
unread_dmail_count columns on the users table.
2022-01-09 12:51:10 -06:00
evazion
04d3a04836 favorites: fix users with incorrect favorite counts.
Fix users that have a non-zero favorite count, but no favorites.
Follow-up to 69facb22b.
2022-01-08 14:18:29 -06:00
evazion
69facb22b1 Fix #4265: Negative favorite counts.
There were 2176 users with incorrect favorite counts, and it stopped
after user id 522393, so this must have been an old bug.
2022-01-08 13:42:49 -06:00
evazion
72ea78e697 searchable: replace find_ordered with in_order_of.
Rails 7 added an `in_order_of` method that does what our `find_ordered`
method did before.
2022-01-07 14:24:57 -06:00
evazion
a26ed9d389 rails: fix #to_s(:human_size) deprecation warnings.
Fix this deprecation warning:

  DEPRECATION WARNING: Integer#to_s(:human_size) is deprecated. Please use Integer#to_formatted_s(:human_size) instead.
2022-01-07 14:24:54 -06:00
evazion
ea069c7b0d models: remove hack to ignore tsvector columns.
Previously we overrode `ActiveRecord::Base#columns` to make it ignore
tsvector columns so they wouldn't show up in API responses. These
columns have been removed, so this is no longer necessary.
2022-01-07 11:23:53 -06:00
evazion
53527b9b29 posts: remove pool_string, fav_string from ignored columns.
These columns have been removed from the database.
2022-01-07 11:22:10 -06:00
evazion
b254d6d9d9 jobs: fix searching by name not finding certain jobs.
Fix a bug where /jobs?search[name]=Prune+Posts didn't find jobs named PrunePostsJob.
`"Prune Posts".tr(" ", "_").classify` was wrong because it returned `"PrunePost"`.
2022-01-05 11:14:56 -06:00
evazion
1ccc74adca jobs: add ability to order jobs in /jobs listing. 2022-01-04 17:53:35 -06:00
evazion
dc159ed135 jobs: show retried jobs in /jobs listing.
Fix the /jobs listing to show retried jobs. GoodJob::ActiveJobJob has a
default scope that filters out retried jobs; remove it in the index
controller so we can see retried jobs.

Also fix mail delivery jobs not showing up in the dropdown in the /jobs
search form.
2022-01-04 17:46:54 -06:00
evazion
82211ba935 jobs: add ability to search jobs on /jobs page.
Add ability to search jobs on the /jobs page by job type or by status.

Fixes #2577 (Search filters for delayed jobs). This wasn't possible
before with DelayedJobs because it stored the job data in a YAML string,
which made it difficult to search jobs by type. GoodJobs stores job data
in a JSON object, which is easier to search in Postgres.
2022-01-04 17:18:36 -06:00
evazion
12601e49fd users: fix exception in can_receive_email?. 2022-01-04 16:04:41 -06:00
evazion
32613f9bb1 emails: fix sending emails to invalid addresses.
Fix mailers to not attempt deliveries to invalid or nonexistent email
addresses. This usually happened when someone changed their email, and
we tried to send a confirmation email to a nonexistent address.
2022-01-02 16:07:57 -06:00
evazion
18c08688df Merge pull request #4947 from nonamethanks/fix-duration-twitter
Fix duration check for uploads from twitter
2021-12-29 22:33:59 -06:00
nonamethanks
15bd5f73b3 Fix duration check for uploads from twitter
Some twitter videos near the max duration had some stray
milliseconds that made the check fail.

For example
https://twitter.com/kivo_some_18/status/1152167154059321344?s=20 (nsfw)
has 140.053333 duration.
2021-12-29 14:25:14 +01:00
evazion
f2b5c9f54d winter sale: fix error with promotion codes.
Fix an exception when checking out on Stripe caused by using the
`allow_promotion_codes` and `discounts` parameters at the same time.

   Stripe::InvalidRequestError - You may only specify one of these parameters: allow_promotion_codes, discounts
2021-12-28 16:05:45 -06:00
evazion
b8356c97c1 Add Danbooru Winter Sale. 2021-12-28 14:41:37 -06:00
evazion
993965b654 posts: reduce string allocations during thumbnail generation.
Further micro-optimize thumbnails to reduce string allocations.

`Post#levelblocked?` gets called once per thumbnail. Before it split the
tag string, which meant one string allocation for each tag on each post.
This added up to thousands of string allocations per pageload.
2021-12-16 17:17:06 -06:00
evazion
1c5786d20f posts: remove cropped thumbnails. 2021-12-16 15:58:29 -06:00
evazion
163ba8e7da posts: micro-optimize allocations during thumbnail generation.
Do a few micro-optimizations to reduce the number of memory allocations
during thumbnail generation.

This commit, combined with freezing string literals in a7dc05 and
67b961, reduces the number of allocations on the front page from 180,000
to 150,000, and the number of retained objects from 8,000 to 4,000.
2021-12-16 00:53:48 -06:00
evazion
a7dc05ce63 Enable frozen string literals.
Make all string literals immutable by default.
2021-12-14 21:33:27 -06:00
evazion
208b618918 tags: remove tag category locks.
Remove the ability to lock a tag's category. Before a moderator could
lock a tag such that only an admin could change the tag's category.

Nowadays the ability to change a tag's category is based on the tag's
size. Members can change tag categories for tags with up to 50 posts,
and Builders can change categories for tags with up to 1000 posts.
Manually locking tags is not necessary.

We only had a few dozen locked tags, mostly random *_(cosplay) tags or
company name tags. Most of these are holdovers from moderators randomly
locking tags like ten years ago.

The `is_locked` field is still in the database, so it is still returned
by the /tags.json API, even though it is unused.
2021-12-09 13:20:26 -06:00
evazion
c22f7b799b media assets: fix error when generating thumbnails for corrupt files.
Fix an error being raised when trying to generate thumbnails for corrupt
files. If the original image is corrupt, then ignore any errors and let
libvips try to generate a thumbnail as best it can. This will usually
result in an incomplete thumbnail.
2021-12-05 21:46:14 -06:00
evazion
ad49a10147 media assets: fix bug in thumbnail generation.
Fix thumbnail generation throwing a NoMatchingPatternError.
2021-12-05 19:04:17 -06:00
evazion
9cb70fa632 posts: add 720x720 thumbnail size.
This is used to provide higher resolution thumbnails for high pixel
density displays, such as phones or laptops. If your screen has a 2x
pixel density ratio, then 360x360 thumbnails will be rendered at 720x720
resolution.

We use WebP here because it's about 15% smaller than the equivalent
JPEG, and because if a device has a high enough pixel density to use
this, then it probably supports WebP.

720x720 thumbnails average about 36kb in size, compared to 20.35kb for
360x360 thumbnails and 7.55kb for 180x180 thumbnails.
2021-12-05 09:19:29 -06:00
evazion
17537084fe posts: generate 180x180px and 360x360px thumbnails (#4932).
Add two new thumbnail sizes. These new thumbnail sizes are generated on
upload, but not used yet.
2021-12-02 23:42:44 -06:00
evazion
e5ba6d4afc MediaFile: fix thumbnail dimension calculation.
Calculate the dimensions of thumbnails ourselves instead of letting
libvips calculate them for us. This way we know the exact size of
thumbnails, so we can set the right width and height for <img> tags. If
we let libvips calculate thumbnail sizes for us, then we can't predict
the exact size of thumbnails, because sometimes libvips rounds numbers
differently than us.
2021-12-01 04:45:26 -06:00
evazion
353e708538 votes: allow admins to remove post votes.
Allow admins to remove votes on posts. This is for fixing vote abuse.

Votes can be removed by going to the vote list on the /post_votes page,
or by clicking on a post's score, then using the "Remove" option in the
"..." dropdown menu next to the vote.

Votes are soft-deleted - they're marked as deleted in the database, but
not fully deleted. Removed votes are only visible to admins, not to
regular users. When a vote is removed by an admin, it leaves a mod
action.

Technically it's possible to undelete votes, but there's no UI for it.
2021-11-23 23:18:54 -06:00
evazion
be5173c8dd votes: add is_deleted flag to post_votes table.
Add an is_deleted flag to post_votes so they can be soft-deleted in the future.
2021-11-21 02:36:30 -06:00
evazion
eda23c719a votes: fixup various minor issues.
* Add a gap between thumbnails on mobile.
* Adjust CSS for scores and vote buttons.
* Include "Private favorites" as an incentive on the user upgrade page.
* Fix vote buttons not being visible beneath thumbnails on mobile.
* Fix the "Show scores" link not preserving the current page number.
* Fix vote buttons being unintentionally enabled for all thumbnails by default.
* Fix banned and restricted users being able to favorite posts by
  tagging them with `fav:self`.
* Fix search engines being able to crawl /posts?view=score pages.
* Fix broken tests.
2021-11-20 02:40:18 -06:00
evazion
3ae62d08eb favorites: show favlist when hovering over favcount.
Changes:

* Make it so you can click or hover over a post's favorite count to see
  the list of public favorites.
* Remove the "Show »" button next to the favorite count.
* Make the favorites list visible to all users. Before favorites were
  only visible to Gold users.
* Make the /favorites page show the list of all public favorites,
  instead of redirecting to the current user's favorites.
* Add /posts/:id/favorites endpoint.
* Add /users/:id/favorites endpoint.

This is for several reasons:

* To make viewing favorites work the same way as viewing upvotes.
* To make posts load faster for Gold users. Before, we loaded all the
  favorites when viewing a post, even when the user didn't look at them.
  This made pageloads slower for posts that had hundreds or thousands of
  favorites. Now we only load the favlist if the user hovers over the favcount.
* To make the favorite list visible to all users. Before, it wasn't
  visible to non-Gold users, because of the performance issue listed above.
* To make it more obvious that favorites are public by default. Before,
  since regular users could only see the favcount, they may have
  mistakenly believed other users couldn't see their favorites.
2021-11-20 02:40:18 -06:00
evazion
bd8672681f votes: add vote buttons beneath thumbnails.
Add upvote and downvote buttons beneath thumbnails on the post index page.

This is disabled by default. To enable it, click the "..." menu in the top
right of the page, then click "Show scores".

This is currently a per-search setting, not an account setting. If you
enable it in one tab, it won't be enabled in other tabs.
2021-11-18 04:15:20 -06:00
evazion
a9997d0d2b favgroups: make private favgroups a Gold-only option.
Make private favgroups a Gold-only option. This is for consistency with
private favorites and upvotes being Gold-only options.

Existing Members with private favgroups are allowed to keep them, as
long as they don't disable privacy. If they disable it, then they can't
re-enable it again without upgrading to Gold first.
2021-11-18 04:15:20 -06:00
evazion
bc96eb864b votes: make private favorites and upvotes a Gold-only option.
Make private favorites and upvotes a Gold-only account option.

Existing Members with private favorites enabled are allowed to keep it
enabled, as long as they don't disable it. If they disable it, then they
can't re-enable it again without upgrading to Gold first.

This is a Gold-only option to prevent uploaders from creating multiple
accounts to upvote their own posts. If private upvotes were allowed for
Members, then it would be too easy to use fake accounts and private
upvotes to upvote your own posts.
2021-11-18 04:11:51 -06:00
evazion
1a27b1d5eb votes: make upvotes visible to everyone by default.
Make upvotes public the same way favorites are public:

* Rename the "Private favorites" account setting to "Private favorites and upvotes".
* Make upvotes public, unless the user has private upvotes enabled. Note
  that private upvotes are still visible to admins. Downvotes are still
  hidden to everyone except for admins.
* Make https://danbooru.donmai.us/post_votes visible to all users. This
  page shows all public upvotes. Private upvotes and downvotes are only
  visible on the page to admins and to the voter themselves.
* Make votes searchable with the `upvote:username` and `downvote:username`
  metatags. These already existed before, but they were only usable by
  admins and by people searching for their own votes.

Upvotes are public to discourage users from upvoting with multiple
accounts. Upvote abuse is obvious to everyone when upvotes are public.
The other reason is to make upvotes consistent with favorites, which are
already public.
2021-11-16 05:23:54 -06:00