Commit Graph

764 Commits

Author SHA1 Message Date
evazion
638c928c8a Add 18 USC 2257 disclaimer.
Add a 2257 disclaimer and link to it in the site footer.
2022-05-09 02:26:19 -05:00
evazion
449fd6c49c upgrades: factor out Stripe integration.
Factor out the Stripe code from the UserUpgrade class. Introduce a new
PaymentTransaction abstract class that represents a payment with some
payment processor, and a PaymentTransaction::Stripe class that
implements transactions with Stripe.

Note that we can't completely eliminate Stripe even though we no longer
accept payments with it because we still need to be able to look up old
payments in Stripe.
2022-05-06 22:52:33 -05:00
evazion
2219a64f47 posts: show takedown page for banned artists and posts.
Show a "This page has been removed because of a takedown request" error when
an unauthorized user searches for a banned tag, or tries to view a banned post.
2022-05-03 05:56:14 -05:00
evazion
1a89071f2d posts: fix error on post index page when search count times out in safe mode
Fix a nil deference error on the post index page. This happened when
performing a single tag search in safe mode and calculating the number
of search results timed out.
2022-05-02 22:03:33 -05:00
evazion
17ffe3590a Fix #4982: Add route to remove a post from a favorite group 2022-05-02 15:56:16 -05:00
evazion
93352b318e Fix #5146: Adding an existing favorite to favorite groups leads to an error.
Show "Favgroup already contains post XXX" error when trying to add a
post to a favgroup that already contains that post.
2022-05-02 15:56:16 -05:00
evazion
f117049750 users: remove 'hide deleted posts' account setting.
This setting automatically added the `-status:deleted` metatag to all searches. This meant deleted
posts were filtered out at the database level, rather than at the html level. This way searches
wouldn't have less-than-full pages.

The cost was that searches were slower, mainly because post counts weren't cached. Normally when you
search for a tag, we can get the post count from the tags table. If the search is actually like
`touhou -status:deleted`, then we don't know the count and we have to calculate it on demand.

This option is being removed because it did the opposite of what people thought it did. People
thought it made deleted posts visible, when actually it made them more hidden.
2022-05-01 00:47:46 -05:00
evazion
ccd0dde081 Fix #5013: BUR model doesn't validate tags.
Don't allow users to request aliases, implications, or renames for invalid tag names.

As a side effect, it's no longer possible to request shortcut aliases like
`/hr -> hakurei_reimu` (slash abbreviations still exist, but they can't
be overridden with aliases). Tests involving these types of aliases are
removed.
2022-04-30 20:03:04 -05:00
evazion
918f32c554 Fix #4461: Improve posts/index page titles. 2022-04-30 01:52:33 -05:00
evazion
bbe748bd2b posts: factor out post edit logic.
Factor out most of the tag edit logic from the Post class to a new
PostEdit class. The PostEdit class contains the logic for parsing tags
and metatags from the tag edit string, and for determining which tags
were added or removed by the edit.

Fixes various bugs caused by not calculating the set of added or removed
tags correctly, for example when tag category prefixes were used (e.g.
`copy:touhou`) or when the same tag was added and removed in the same
edit (e.g. `touhou -touhou`).

Fixes #5123: Tag categorization prefixes bypass deprecation check
Fixes #5126: Negating a deprecated tag will still cause the warning to show
Fixes #3477: Remove tag validator triggering on tag category changes
Fixes #4848: newpool: metatag doesn't parse correctly
2022-04-29 17:13:33 -05:00
evazion
6c5dd5ffed tests: fix broken tests. 2022-04-18 00:31:31 -05:00
nonamethanks
1a990d5ab9 Allow post disapprovals to be edited 2022-04-11 21:05:44 +02:00
nonamethanks
11281d6f58 Tags: don't allow deprecation of tags without wiki 2022-04-09 20:16:55 +02:00
nonamethanks
ea76a889db Add ability to mark tags as deprecated
* Deprecated tags can't be added to posts, but existing deprecated tags
  in a post won't be removed
* Only empty tags can be marked as deprecated manually
* No tags can be manually undeprecated
** These limits don't apply to admins
* Deprecating or undeprecating a tag will create a new mod action to
  prevent people from going rogue
* Added deprecate/undeprecate commands for BURs
* Deprecating a tag via BUR removes all implications to and from it as well
2022-04-08 09:07:14 +02:00
evazion
0d480eb832 artist urls: stop using normalized_url.
Stop the last remaining uses of the `artist_urls.normalized_url` column.
It's already no longer used by the artist finder. The only remaining
uses were by API users. Those users should use the `url` column instead.
2022-04-02 23:58:01 -05:00
evazion
a272c19b98 Fix #5078: Pixiv booth upload broken.
Allow image URLs from https://booth.pximg.net to be uploaded. Fix bug
where Booth.pm URLs were incorrectly caught by the Pixiv extractor.
2022-03-30 03:25:42 -05:00
evazion
d9d3c1dfe4 sources: rename Sources::Strategies to Source::Extractor.
Rename Sources::Strategies to Source::Extractor. A Source::Extractor
represents a thing that extracts information from a given URL.
2022-03-24 03:49:44 -05:00
evazion
56f47c60e1 posts: fix exception when viewing post with source Blog..
Fix a PublicSuffix::DomainNotAllowed exception raised with viewing or editing a post
with a source like `Blog.`.

This happened when parsing the post's source. `Danbooru::URL.parse("Blog.")` would
heuristically parse the source into `http://blog`. Calling any methods related to the
URL's hostname or domain would lead to calling `PublicSuffix.parse("blog")`, which
would fail with PublicSuffix::DomainNotAllowed.
2022-03-21 03:24:50 -05:00
evazion
defea08084 posts: fix exception in random:1 searches.
Fix regression in 1ad0e8688. Caused by `relation.order_values` returning
an array of Arel nodes instead of an array of strings when doing a
`random:1` search.
2022-03-21 01:29:10 -05:00
evazion
01b683798e sources: add Tinami support. 2022-03-19 00:50:36 -05:00
evazion
912e996027 Fix #4470: Check URLs for duplicates when creating artists
Show a warning when creating a duplicate artist; that is, when adding a
URL that already belongs to another artist.

This is a soft warning rather than a hard error because there are some
cases where multiple artists legitimately share the same site or account.
2022-03-18 17:10:23 -05:00
evazion
42144eaa4b Fix #5012: Fc2 image link paste not uploading.
Fix referer spoofing not working for certain fc2.com image URLs.

Spoofing the referer like this redirects to an HTML error page:

* curl -H "Referer: http://wwwew.web.fc2.com" http://wwwew.web.fc2.com/e/405.jpg

Spoofing it like this works:

* curl -H "Referer: http://wwwew.web.fc2.com/e/405.jpg" http://wwwew.web.fc2.com/e/405.jpg
2022-03-18 04:39:13 -05:00
evazion
10dac3ee51 artists: normalize urls added to artist entries.
When a URL is added to an artist entry, normalize it to a standard form.

Artist URLs have both a `url` column and a `normalized_url` column. The
`normalized_url` is used for artist finding and the `url` is the raw URL
entered by the user. Previously only the `normalized_url` field was
normalized; now the URL entered by the user is also converted to a
normalized form.

This means that if an URL like this is added to an artist entry:

* http://www.pixiv.net/member.php?id=1234
* http://www.pixiv.net/en/users/1234
* http://www.twitter.com/DanbooruBot/
* http://mobile.twitter.com/DanbooruBot/

It will get normalized to this:

* https://www.pixiv.net/users/1234
* https://twitter.com/DanbooruBot

This fixes problems with duplicate URLs being added to artist entries
because URLs weren't normalized to a single form.
2022-03-18 02:06:50 -05:00
nonamethanks
a6549bc6fe Add Fantia support
Also fixes a regression in 74fdeef10c
that stopped mastodon urls from being given the right priority.
2022-03-10 17:43:32 +01:00
evazion
80939f5e22 users: fix name changes not being validated.
Fix names not being validated when a user changes their name. Regression in 5ce724f1e.
2022-03-09 01:14:09 -06:00
evazion
df0bb70486 sources: factor out Source::URL::PixivSketch.
Add upload support for Pixiv Sketch. Fetch tags, commentary, and artist,
and rewrite sample images to full images.

Authentication isn't required. R18 images are hidden in the browser but
visible in the API.
2022-03-08 18:24:12 -06:00
evazion
5ce724f1e2 users: remove confirmation when changing username. 2022-03-08 03:29:15 -06:00
evazion
525acd17a5 users: lock out users with invalid names until they change their name.
Forcibly redirect users to the name change page if their name is
invalid. This means user with invalid names can't do anything or view
any pages until they change their name. API requests are still allowed.
2022-03-07 04:36:12 -06:00
evazion
a160a3acce users: add stricter username rules.
Add stricter username rules:

* Only allow usernames to contain basic letters, numbers, CJK characters, underscores, dashes and periods.
* Don't allow names to start or end with punctuation.
* Don't allow names to have multiple underscores in a row.
* Don't allow active users to have names that look like deleted users (e.g. "user_1234").
* Don't allow emoji or any other Unicode characters except for Chinese, Japanese, and Korean
  characters. CJK characters are currently grandfathered in but will be disallowed in the future.

Users with an invalid name will be shown a permanent sitewide banner until they change their name.
2022-03-05 01:08:53 -06:00
evazion
74fdeef10c sources: factor out Source::URL::Mastodon. 2022-02-26 15:08:27 -06:00
evazion
86d8e2d13d sources: factor out Source::URL::Lofter. 2022-02-25 23:43:10 -06:00
evazion
e6ded89f85 sources: factor out Source::URL::Plurk.
Also fix it so that for adult works, we get the images posted by the
artist in the replies. Example: https://www.plurk.com/p/omc64y (nsfw).
2022-02-25 02:06:57 -06:00
evazion
26f4cf1ebd sources: factor out Source::URL::Skeb. 2022-02-25 02:06:57 -06:00
evazion
112b323f01 foundation: fix exception when uploading new Foundation url format.
Fix 'null value in column "source_url"' exception when uploading urls like this:

* https://foundation.app/@KILLERGF/kgfgen/4
* https://foundation.app/@mochiiimo/foundation/97376
2022-02-22 13:29:28 -06:00
evazion
60a26af6e3 rails: add 'URL' inflection.
Make it so we can write `ArtistURL` instead of `ArtistUrl`.
2022-02-22 00:17:53 -06:00
evazion
68ba447494 uploads: remove batch upload page.
* Make /uploads/batch redirect to /uploads/new.
* Remove /uploads/image_proxy.
2022-02-21 00:03:43 -06:00
evazion
a916fd9e7b Fix #5015: 'Rating not selected' error leads to an empty page. 2022-02-19 17:25:14 -06:00
evazion
202dfe5d87 uploads: allow uploading multiple files from your computer at once.
Allow uploading multiple files from your computer at once.

The maximum limit is 100 files at once. There is still a 50MB size limit
that applies to the whole upload. This limit is at the Nginx level.

The upload widget no longer shows a thumbnail preview of the uploaded
file. This is because there isn't room for it in a multi-file upload,
and because the next page will show a preview anyway after the files are
uploaded.

Direct file uploads are processed synchronously, so they may be slow.

API change: the `POST /uploads` endpoint now expects the param to be
`upload[files][]`, not `upload[file]`.
2022-02-19 00:00:56 -06:00
evazion
9a5a04d74e nijie: fix uploads not working for new image URL format.
Fix uploads not working for image URLs like this:

    https://pic.nijie.net/07/nijie/17/95/728995/illust/0_0_403fdd541191110c_c25585.jpg
2022-02-15 20:45:28 -06:00
evazion
6b56b6a122 uploads: fix error when source doesn't have any images.
Fix an error when trying to upload a source that doesn't have any
images, for example a Twitter post with no images.
2022-02-15 18:55:12 -06:00
evazion
e4d7453180 uploads: improve error messages.
Improve upload error messages when downloading an URL fails, or it isn't
an image or video file.
2022-02-15 18:54:55 -06:00
evazion
87a00a1182 uploads: fix "ArgumentError: string contains null byte" error
Fix an error when trying to upload a file larger than the file size
limit. In this case we tried to dump the whole HTTP response into the
error message, which included the binary file itself, which caused this
exception because it contained null bytes.
2022-02-15 18:16:47 -06:00
evazion
16b8d4b607 uploads: consider uploads as failed when all assets fail.
Make the "completed" status for an upload mean "at least one file in the
upload successfully completed". The "error" status means "all files in
the upload failed".

This means that when an upload has multiple assets and some succeed and
some fail, the whole upload is considered completed. This can happen
when uploading multiple files and some files are over the size limit,
for example. The upload is considered failed only if all files in the
upload fail.

This fixes an issue where, if uploading a single file and that file
failed because it was over the size limit, then the upload wouldn't be
marked as failed.
2022-02-15 17:12:02 -06:00
evazion
58fc00e549 uploads: allow uploading iso5 .mp4 files.
This is an MP4 ftyp sometimes used by Twitter.
2022-02-09 16:48:11 -06:00
evazion
26d54c1e37 artists: show "Artist" tab in searches for artists without artist entry.
On the post index page, show the "Artist" tab instead of the "Wiki" tab when searching for
an artist tag that doesn't have an artist entry. This way the user is prompted to create a
new artist entry instead of a new wiki.
2022-02-09 01:16:05 -06:00
evazion
37ad6f5a71 Fix non-existent .js requests raising DoubleRenderError.
Fix requests for non-existent .js pages, for example https://danbooru.donmai.us/oaisfj.js,
raising AbstractController::DoubleRenderError when trying to render the 404 response.
2022-02-08 00:14:38 -06:00
evazion
1a61e329ba uploads: add column for error messages.
Change it so uploads store errors in an `error` column instead of in the
`status` field.
2022-02-07 15:44:39 -06:00
evazion
572878fb0d uploads: allow uploading .m4v format videos.
Fix not being able to upload .m4v format videos as reported here:

* https://danbooru.donmai.us/forum_posts/205248
* https://github.com/danbooru/danbooru/issues/3615#issuecomment-1030950924

From https://en.wikipedia.org/wiki/M4V:

  The M4V file format is a video container format developed by Apple and
  is very similar to the MP4 format. The primary difference is that M4V
  files may optionally be protected by DRM copy protection.

This could be a problem if it allows uploading videos that are
unplayable because of DRM.
2022-02-06 21:41:35 -06:00
evazion
7bed81812d Don't show error messages that could contain private information.
Fix a potential exploit where private information could be leaked if
it was contained in the error message of an unexpected exception.

For example, NoMethodError contains a raw dump of the object in the
error message, which could leak private user data if you could force a
User object to raise a NoMethodError.

Fix the error page to only show known-safe error messages from expected
exceptions, not unknown error messages from unexpected exceptions.

API changes:

* JSON errors now have a `message` param. The message will be blank for unknown exceptions.
* XML errors have a new format. This is a breaking change. They now look like this:

    <result>
      <success type="boolean">false</success>
      <error>PaginationExtension::PaginationError</error>
      <message>You cannot go beyond page 5000.</message>
      <backtrace type="array">
        <backtrace>app/logical/pagination_extension.rb:54:in `paginate'</backtrace>
        <backtrace>app/models/application_record.rb:17:in `paginate'</backtrace>
        <backtrace>app/logical/post_query_builder.rb:529:in `paginated_posts'</backtrace>
        <backtrace>app/logical/post_sets/post.rb:95:in `posts'</backtrace>
        <backtrace>app/controllers/posts_controller.rb:22:in `index'</backtrace>
      </backtrace>
    </result>

  instead of like this:

    <result success="false">You cannot go beyond page 5000.</result>
2022-02-06 18:09:54 -06:00
evazion
b695c4ccb1 modreports: fix private user information leak in new modreport action.
Fix this:

    https://danbooru.donmai.us/moderation_reports/new.json?moderation_report[model_id]=52664&moderation_report[model_type]=User

raising an `undefined method `reportable?' for #<UserPolicy ...>`
exception, which contained the full user object in the error message,
which leaked private user information.
2022-02-06 14:39:08 -06:00