Commit Graph

1709 Commits

Author SHA1 Message Date
evazion
acf565be7b Fix #4678: Validate custom CSS.
* Make it an error to add invalid custom CSS to your account.
* Add a fix script to remove custom CSS from all accounts with invalid CSS.
2022-01-15 23:20:49 -06:00
evazion
c455c08b2c tests: fix broken tests.
Fixups for c3c4f5a2a.
2022-01-15 22:02:47 -06:00
evazion
5bec0aa147 Fix #4629: Tag renames/aliases should also modify pool descriptions. 2022-01-15 20:34:08 -06:00
evazion
c3c4f5a2a7 Fix #4957: Autotag non-web_source.
Autotag non-web_source on posts that have a non-http:// or https:// URL.
Add a fix script to backfill old posts.

Syntactically invalid URLs are still considered web sources. For
example, `https://google,com` technically isn't a valid URL, but it's
not considered a non-web source.
2022-01-14 22:58:27 -06:00
evazion
00ebd2e13c Merge pull request #4956 from nonamethanks/fix-skeb
Skeb: fix several issues with the strategy
2022-01-14 22:04:44 -06:00
nonamethanks
33db1a2761 Skeb: fix several issues with the strategy
* Fix fetching of videos
* Fix fetching of original commentary
* Fix images being returned out of order in bookmarklet
2022-01-14 21:24:48 +01:00
evazion
dc724ee912 BURs: add test for aliasing together two tags with an implication (#4871).
Add a test for when A and B imply the same tag, and A is aliased to B.
2022-01-11 16:38:40 -06:00
evazion
df09bb239b Fix #4906: Trying to replace a sourceless post that matches md5 returns error and doesn't fix source.
Allow replacing a post with itself. Skip uploading the file and just
update the post's source with the given source.
2022-01-11 16:09:14 -06:00
evazion
17fb34922b nijie: fix failure to fetch source data due to change in login system.
Nijie changed their login system so that now there are two cookies that
need to be remembered: NIJIEIJIEID, and nijie_tok.
2022-01-11 15:14:54 -06:00
evazion
f8768fd6b7 BURs: move the is_banned flag when aliasing artists.
When a banned artist tag is aliased into a nonbanned artist tag, move
the is_banned flag from the old artist entry to the new artist_entry.

Related to #4940. Fixes a case where a banned artist could lose the
banned status when it was moved.
2022-01-11 12:17:56 -06:00
evazion
33828ec8a4 posts: remove set_tag_string method. 2022-01-11 10:06:46 -06:00
evazion
2e1c7ce6d3 Fix #4951: chartags:0 returning posts with chartags.
* Add fix script to fix posts with incorrect tag_count_* fields.
* Simplify the code for updating tag_count_* fields (no functional change).
2022-01-10 13:33:56 -06:00
evazion
450594b803 tests: fix broken tests. 2022-01-07 14:44:24 -06:00
evazion
edd0656b73 tests: fix broken tests. 2022-01-06 00:41:18 -06:00
evazion
7ce81ceccb emails: fix exception when user signs up without an email.
Fix an `ActionView::Template::Error: undefined method 'verification_key'
for nil` error in the welcome_user mailer when a user signs up without
an email address.

Caused by the fact that we now render mail templates regardless of
whether the user has an email address, and then skip sending the email
only after the mail template is rendered.
2022-01-04 17:57:16 -06:00
evazion
12601e49fd users: fix exception in can_receive_email?. 2022-01-04 16:04:41 -06:00
evazion
e40218d1c8 Fix #4515: Set List-Unsubscribe header on notification emails. 2022-01-02 17:10:27 -06:00
evazion
751835745b emails: fix email validation regex.
Fix the email validation regex allowing certain invalid emails like `foo@gmail..com`.
2022-01-02 16:08:35 -06:00
evazion
32613f9bb1 emails: fix sending emails to invalid addresses.
Fix mailers to not attempt deliveries to invalid or nonexistent email
addresses. This usually happened when someone changed their email, and
we tried to send a confirmation email to a nonexistent address.
2022-01-02 16:07:57 -06:00
evazion
6d7a139ef1 Fix #4946: Don't Add Comments to Posts When Doing Post Replacements. 2021-12-28 11:16:15 -06:00
evazion
0ba6dc9ee5 Fix #4945: Search for an artist by URL throws an exception. 2021-12-18 01:55:29 -06:00
evazion
1c5786d20f posts: remove cropped thumbnails. 2021-12-16 15:58:29 -06:00
evazion
aeb2b2b7ae tests: fix session_loader_test.
Fix regression in 52013eac1.
2021-12-10 01:46:01 -06:00
evazion
7dbde7bc14 Fix #4933: Don't allow mass update requests if both the antecedent and consequent are single tags 2021-12-09 18:05:14 -06:00
evazion
208b618918 tags: remove tag category locks.
Remove the ability to lock a tag's category. Before a moderator could
lock a tag such that only an admin could change the tag's category.

Nowadays the ability to change a tag's category is based on the tag's
size. Members can change tag categories for tags with up to 50 posts,
and Builders can change categories for tags with up to 1000 posts.
Manually locking tags is not necessary.

We only had a few dozen locked tags, mostly random *_(cosplay) tags or
company name tags. Most of these are holdovers from moderators randomly
locking tags like ten years ago.

The `is_locked` field is still in the database, so it is still returned
by the /tags.json API, even though it is unused.
2021-12-09 13:20:26 -06:00
evazion
e57e38b35f tests: fix broken tests. 2021-12-08 03:01:54 -06:00
evazion
c2e6202da6 Fix #4920: Wrong color for certain samples.
The problem was that we were stripping color profiles from thumbnails,
but we weren't setting `export_profile: "srgb"` to convert images to
sRGB first. This resulted in wrong colors for images with non-sRGB color
profiles, such as Adobe RGB.

The fix is to convert images to sRGB when possible, while leaving CMYK
and greyscale images alone. We leave CMYK images alone because we can't
convert CMYK to sRGB without losing color. We leave greyscale images
alone if they don't have a color profile, that way they stay as
one-channel greyscale (or two-channel greyscale, in case of alpha)
instead of being converted to three-channel sRGB. However, if a
greyscale image has a color profile, then we have to convert to sRGB,
otherwise the colors would be wrong when we strip the profile.

We also have to set the import profile, otherwise images with broken
embedded color profiles won't have a fallback profile and may get
incorrect colors. In this case we also have to be careful, because we
can't specify an sRGB fallback for greyscale or CMYK images.
2021-12-01 04:45:26 -06:00
evazion
0baca68a37 search: make order:random truly random; add random:N metatag.
Make the `order:random` metatag truly randomize the search. Add a
`random:N` metatag that returns up to N random posts, like what
`order:random` did before.

`order:random` now returns the entire search in random order. Before it
just returned a pageful of pseudorandom posts. This will be more
accurate for small searches, but slower for large searches. If
`order:random` times out, try `random:N` instead.

The `random:N` metatag returns up to N pseudorandom posts. This is
faster than `order:random` for large searches, but for small searches,
it may return less than N posts, and the randomness may be biased. Some
posts may be more likely than others to appear. N must be between 0 and
200.

Also, `/posts?tags=touhou&random=1` now redirects to `/posts?tags=touhou+random:N`.
Before the `random=1` param acted like a free `order:random` tag; now it
redirects to a `random:N` search, so it counts against your tag limit.
2021-11-25 18:14:34 -06:00
evazion
594b46a85d tests: fix broken tests. 2021-11-23 23:18:54 -06:00
evazion
353e708538 votes: allow admins to remove post votes.
Allow admins to remove votes on posts. This is for fixing vote abuse.

Votes can be removed by going to the vote list on the /post_votes page,
or by clicking on a post's score, then using the "Remove" option in the
"..." dropdown menu next to the vote.

Votes are soft-deleted - they're marked as deleted in the database, but
not fully deleted. Removed votes are only visible to admins, not to
regular users. When a vote is removed by an admin, it leaves a mod
action.

Technically it's possible to undelete votes, but there's no UI for it.
2021-11-23 23:18:54 -06:00
evazion
eda23c719a votes: fixup various minor issues.
* Add a gap between thumbnails on mobile.
* Adjust CSS for scores and vote buttons.
* Include "Private favorites" as an incentive on the user upgrade page.
* Fix vote buttons not being visible beneath thumbnails on mobile.
* Fix the "Show scores" link not preserving the current page number.
* Fix vote buttons being unintentionally enabled for all thumbnails by default.
* Fix banned and restricted users being able to favorite posts by
  tagging them with `fav:self`.
* Fix search engines being able to crawl /posts?view=score pages.
* Fix broken tests.
2021-11-20 02:40:18 -06:00
evazion
1a27b1d5eb votes: make upvotes visible to everyone by default.
Make upvotes public the same way favorites are public:

* Rename the "Private favorites" account setting to "Private favorites and upvotes".
* Make upvotes public, unless the user has private upvotes enabled. Note
  that private upvotes are still visible to admins. Downvotes are still
  hidden to everyone except for admins.
* Make https://danbooru.donmai.us/post_votes visible to all users. This
  page shows all public upvotes. Private upvotes and downvotes are only
  visible on the page to admins and to the voter themselves.
* Make votes searchable with the `upvote:username` and `downvote:username`
  metatags. These already existed before, but they were only usable by
  admins and by people searching for their own votes.

Upvotes are public to discourage users from upvoting with multiple
accounts. Upvote abuse is obvious to everyone when upvotes are public.
The other reason is to make upvotes consistent with favorites, which are
already public.
2021-11-16 05:23:54 -06:00
evazion
43c2870664 Fix #4917: Add down_score/up_score orders and metasearches.
Add `upvotes:N`, `downvotes:N`, `order:upvotes`, `order:downvotes`,
`order:upvotes_asc`, `order:downvotes_asc` metatags.

In the API, the field is called up_score / down_score. Here it's called
`upvotes` and `downvotes` because this should be easier to understand
for end users.

Note that internally, `down_score` is negative. A post that matches
`downvotes:>5` will have down_score < -5 internally.
2021-11-16 03:52:38 -06:00
evazion
b561ca49f2 foundation: fix mojibake in artist commentaries.
Fix certain artist commentaries for foundation.app containing scrambled
characters. Apparently caused by the Nokogiri HTML5 parser not handling
UTF-8 input correctly when the encoding isn't explicitly set to UTF-8.
2021-11-15 04:55:48 -06:00
nonamethanks
49e232f2ae Foundation: add support for unconventional account names 2021-11-09 13:35:52 +01:00
evazion
2225c9b472 Merge pull request #4912 from nonamethanks/feat-foundation-video
Foundation: add support for videos
2021-11-05 06:03:45 -05:00
evazion
a5f589f9e0 aliases/implications: change automatic retirement rules.
Change the rules for automatically retiring aliases and implications:

* Retire aliases to tags that are empty, or that are for a general or
  artist tag that hasn't received any new posts in the last two years.
* Retire implications from tags that are empty.
* Don't retire aliases or implications for character, copyright, or
  meta tags any more, unless the tags are empty.
2021-11-05 05:46:50 -05:00
nonamethanks
6c9b49c194 Foundation: add support for videos 2021-11-05 09:43:49 +01:00
evazion
7709e84502 BURs: allow reapproving failed BURs containing alias or implication removals.
Make it possible to reapprove failed BURs that removed aliases or
implications.

Before if a BUR failed midway through, and we tried to reapprove it,
then it would fail when it got to a `remove alias` line because the
alias had already been removed. Now we keep going if we try to remove an
alias or implication that has already been removed.
2021-11-03 19:45:28 -05:00
evazion
dccc2edb75 tests: fix broken tests.
* Fix a Twitter test broken by a privated tweet.
* Fix an IP geolocation test broken by the ipregistry.co API returning new data.
2021-11-02 04:42:07 -05:00
evazion
4b94cac757 Merge pull request #4911 from nonamethanks/feat-plurk
Add Plurk support
2021-11-02 04:07:07 -05:00
evazion
a5ed8c72c9 search: fix parsing of invalid metatag values.
* Change `age:` metatag to require time units. This means e.g.
  `age:<600` no longer works; instead you have to say `age:<600sec`.

* Allow time units in the `age:` metatag to be abbreviated as long as
  they're unambiguous. This means `age:<60sec`, `age:<5min`, and
  `age:<5mon` now work, in addition to `age:<60s` and `age:<60seconds`.

* Allow the `ratio:` metatag to be written like `ratio:16/9` in addition
  to `ratio:16:9`.

* Fix invalid date searches like `date:foo` or `date:05-15-2021`
  to return nothing instead of raising an "undefined method
  'beginning_of_day' for nil" exception. (`date:05-15-2021` is invalid
  because it's parsed as DD-MM-YYYY).

* Fix invalid searches like `score:foo`, `ratio:foo`, and `mpixels:foo`
  to return nothing instead of being treated like `score:0`, `ratio:0`,
  `mpixels:0`.

* Fix `age:<60m` to return nothing instead of silently being treated
  like `age:<60seconds`.

* Fix `age:foo` to return nothing instead of silently being treated like
  `age:0d` (return all uploads from today).

Fixes #4389.
2021-11-02 01:54:05 -05:00
nonamethanks
060223c9e2 Add Plurk support 2021-11-01 16:21:27 +01:00
evazion
5177a28f2c Merge pull request #4910 from nonamethanks/feat-foundation
Add Foundation support
2021-11-01 05:07:44 -05:00
nonamethanks
043f2fb124 Add Foundation support 2021-11-01 01:39:56 +01:00
evazion
1614b301e3 storage managers: add mirror storage manager.
Add a storage manager that allows mirroring files to multiple storage
backends.
2021-10-29 07:14:21 -05:00
evazion
e697d1886d Fix #4899: Alias fails when implication already exists. 2021-10-27 01:18:54 -05:00
evazion
eddff747d6 Fix certain IPs not being recognized as proxies.
Fix certain IPs (namely Digital Ocean IPs) no longer being recognized as
proxy IPs by the Ipregistry.co API. Caused by some sudden change in the
API.
2021-10-27 00:05:44 -05:00
evazion
082544ab03 StorageManager: remove Post-specific code.
Refactor StorageManager to remove all image URL generation code. Instead
the image URL generation code lives in MediaAsset.

Now StorageManager is only concerned with how to read and write files to
remote storage backends like S3 or SFTP, not with how image URLs should
be generated. This way the file storage code isn't tightly coupled to
posts, so it can be used to store any kind of file, not just images
belonging to posts.
2021-10-27 00:05:30 -05:00
evazion
afe5095ee6 posts: mark media asset as expunged when post is expunged.
Fix it so that when a post is expunged, the media asset is also marked
as expunged. This way the files will be deleted, but the media asset
will still remain as a record of what was expunged. The media asset will
have the md5, width, height, file ext, and file size of the deleted file.
2021-10-26 02:53:32 -05:00