Commit Graph

9878 Commits

Author SHA1 Message Date
evazion
4dc1a109c5 media metadata: fix exception in /media_metadata.json?only=media_asset.
Fix NameError exception caused by the MediaMetadataController trying to lookup "MediaMetadatum" as the model name.
2022-10-14 15:59:50 -05:00
evazion
93887b00b8 Fix #5294: pixiv_ugoira_frame_data alternative
Allow including the media metadata with the post like this:

* https://danbooru.donmai.us/posts.json?tags=ugoira&only=id,file_url,media_metadata
2022-10-14 15:00:20 -05:00
evazion
4b5d4e5c20 thumbnails: remove crossorigin="anonymous" html attribute.
Possible fix for images not loading when Cloudflare Managed Challenges are enabled.
`crossorigin="anonymous"` prevents the `cf_clearance` cookie from being passed (maybe?).
2022-10-14 02:11:39 -05:00
evazion
115521906c tumblr: fix failure to upload new Tumblr URLs containing the post title.
Fix failure to upload Tumblr URLs of this form:

* https://www.tumblr.com/munespice/683613396085719040/saur-family
2022-10-13 21:11:07 -05:00
evazion
268ec9118a tumblr: fix failure to upload certain video posts.
Fix failure to upload Tumblr video posts that contained a video URL of this form:

* https://va.media.tumblr.com/tumblr_rjoh0hR8Xe1teimlz_720.mp4
2022-10-13 21:10:29 -05:00
evazion
a07234121d tumblr: fixup for parsing www.tumblr.com/name URLs. 2022-10-13 00:26:16 -05:00
evazion
2e7b3cd80b tumblr: normalize https://www.tumblr.com/name artist URLs. 2022-10-12 23:55:17 -05:00
evazion
eb8f98e4a6 artists: normalize foriio.com artist URLs.
Normalize `https://fori.io/comori22` to `https://www.foriio.com/comori22` in artist entries.
2022-10-12 23:46:50 -05:00
evazion
0831b8dc9a Fix #5292: post edit tag_string - unquoted metatag value trailing escapes aren't parsed correctly when last term 2022-10-12 19:33:19 -05:00
evazion
49d0842f00 dmcas: dmail copy of complaint to owner user. 2022-10-12 03:34:05 -05:00
evazion
879243d8f1 dmcas: update wording on form. 2022-10-11 19:30:48 -05:00
evazion
395a3e45f6 css: add more utility classes. 2022-10-11 17:31:19 -05:00
evazion
24bc6aa949 Add DMCA complaint form.
Add a form for submitting DMCA complaints. The complaint is emailed to
the site owner, and a confirmation email is sent to the submitter.
2022-10-11 15:45:47 -05:00
evazion
8fbc6d1d3a gelbooru: fix exception in md5-based post urls.
Fix exception when trying to get the image URL for sources like
https://gelbooru.com/index.php?page=post&s=list&md5=04f2767c64593c3030ce74ecc2528704.
2022-10-11 01:31:49 -05:00
evazion
f05268df7f sources: add Gelbooru support.
Add support for uploading posts from Gelbooru. Note that the translated
tags will include both the Gelbooru tags and the tags from the Gelbooru
post's source. The commentary and artist information will also be taken
from the Gelbooru post's source. The source of the Danbooru post however
will be left as the Gelbooru post itself, not as the Gelbooru post's source.
2022-10-11 00:06:45 -05:00
evazion
c2adf279ee ugoira: remove the PixivUgoiraFrameData model.
Remove the last remaining uses of the PixivUgoiraFrameData model. As of
32bfb8407, Ugoira frame data is now stored in the MediaMetadata model,
under the `Ugoira:FrameDelays` EXIF field.

The pixiv_ugoira_frame_data table still exists, but it can be removed
after this commit is deployed.

Fixes #5264: Error when replacing with ugoira.
2022-10-10 18:21:30 -05:00
evazion
a23c02d0cb posts: include id attribute in API. 2022-10-10 06:28:22 -05:00
evazion
d3a5a60aec posts: detect more AI-generated posts. 2022-10-10 04:39:15 -05:00
evazion
1d5db37f56 posts: automatically tag AI-generated on NovelAI posts.
Automatically add the AI-generated tag to posts that have the
`PNG:Software=NovelAI` EXIF attribute.

This is not foolproof because this metadata may get removed if an
AI-generated post is resaved or uploaded to a site that strips EXIF
metadata. It also only works for NovelAI. Currently it detects 29 out of
177 AI-generated uploads on Danbooru.
2022-10-10 04:04:35 -05:00
evazion
32bfb84075 ugoira: switch to frame delays from MediaMetadata model.
Switch the Ugoira Javascript player to using frame delays from the
MediaMetadata model instead of from the PixivUgoiraFrameData model.
After this is deployed it will be safe to remove the PixivUgoiraFrameData
model.
2022-10-10 03:46:14 -05:00
evazion
4066ee52b1 db: add tsvector indexes on commentary, dmail, and wiki page text columns. 2022-10-10 02:57:23 -05:00
evazion
01d10a54f8 ugoira: store frame delays in MediaMetadata model.
Store Ugoira frame delays in the MediaMetadata model as a fake EXIF
field instead of in the PixivUgoiraFrameData model. This way we can get
rid of the PixivUgoiraFrameData model completely. This is a step towards
fixing #5264.
2022-10-09 22:25:20 -05:00
evazion
73cc0f65c2 ugoira: remove legacy browser workarounds from JS player.
Remove various fallbacks and workarounds from the Ugoira Javascript
player for legacy browser issues.

* Remove fallbacks for URL, Blob, and ArrayBuffer (these have been
  supported in most browsers for years).
* Remove workaround for iOS <6 not supporting Blob URLs.
* Remove workaround for a bug in Safari's handling of the Range HTTP
  header. This was fixed in 2015.
* Remove support for playing a list of image URLs instead of playing
  images from the .zip file (this was never used).
* Remove support for limiting loadahead of images (it was hardcoded to
  always use unlimited loadahead anyways).
2022-10-09 18:47:15 -05:00
evazion
16f3250067 ugoira: fix player to only use list of frame delays.
The frame data for Ugoira files is stored like this:

    [{"file"=>"000000.jpg", "delay"=>65},
    {"file"=>"000001.jpg", "delay"=>65},
    {"file"=>"000002.jpg", "delay"=>65},
    {"file"=>"000003.jpg", "delay"=>65},
    {"file"=>"000004.jpg", "delay"=>65},
    {"file"=>"000005.jpg", "delay"=>65},
    {"file"=>"000006.jpg", "delay"=>65},
    {"file"=>"000007.jpg", "delay"=>65},
    {"file"=>"000008.jpg", "delay"=>65},
    {"file"=>"000009.jpg", "delay"=>65},
    {"file"=>"000010.jpg", "delay"=>65}]

This is stored in the pixiv_ugoira_frame_data table in YAML format. This
is a problem because a) we only need the frame delays to play the Ugoira,
not the filenames, and b) storing the data in YAML format is a security
issue that's blocking the upgrade to Rails 7.0.4 (see [1]).

This commit changes the Ugoira Javascript player so that it only uses
the list of frame delays, not the filenames, to play the Ugoira. This
paves the way for storing the frame delays as a simple integer array
instead as a serialized YAML object.

This assumes that the images in a Ugoira zip file are stored in the same
order they should be played back in. This was confirmed by checking every
zip file and verifying that files are actually stored in filename order.

[1]: https://discuss.rubyonrails.org/t/cve-2022-32224-possible-rce-escalation-bug-with-serialized-columns-in-active-record/81017
2022-10-09 18:47:10 -05:00
evazion
9e91373432 views: fix hardcoded references to Danbooru. 2022-10-06 23:16:29 -05:00
evazion
56395f8c71 posts: fixup bug in 24216379b 2022-10-05 02:50:31 -05:00
evazion
a4cca8b020 posts: fixup for b4a67331c. 2022-10-05 01:44:10 -05:00
evazion
528d03a60b forum: add more min_levels options. 2022-10-05 01:32:57 -05:00
evazion
b4a67331ce posts: remove sidebar for banned searches. 2022-10-05 01:25:26 -05:00
evazion
24216379b7 posts: increase levelblocked to approver 2022-10-05 01:14:15 -05:00
evazion
3ac6d33a37 Merge pull request #5276 from nottalulah/wiki-locks
show locked/unlocked changes on wiki page histories
2022-10-04 18:27:00 -05:00
evazion
ef4c6f61d4 Merge pull request #5257 from eltociear/patch-1
Fix typo in session_loader.rb
2022-10-04 18:26:47 -05:00
evazion
88eb617ec4 html: fix footer margin on mobile.
Fix not having any space between the footer and the body on mobile.
2022-10-04 18:26:09 -05:00
evazion
68de87f085 Merge pull request #5258 from nottalulah/hide-upgrade-banner
upgrades: hide upgrade banner when upgrades are disabled
2022-10-04 18:23:50 -05:00
Talulah
6a71508c3b show locked/unlocked changes on wiki page histories 2022-10-03 19:20:03 -03:00
evazion
0cfd0ff436 emails: add fix script to renormalize email addresses.
Whenever the email address normalization procedure changes, the
`normalized_address` column of the email address table must be updated.
This is normally when the list of canonical domain mappings changes.

Renormalizing addresses may also require deleting duplicates.
2022-10-03 02:55:30 -05:00
evazion
86e69e3401 emails: add fix script to delete duplicate email addresses.
In the past it was possible for users to create multiple accounts with
the same email address. We had about 9000 such accounts. This removes
the email address from these accounts.

When multiple accounts have the same email address, the account that
visited the site last gets to keep the address.
2022-10-02 23:59:54 -05:00
evazion
906316c48e users: fix bug in changing usernames.
Fix the change username page not showing the user's current name when a mod is changing a user's name.
2022-10-02 02:38:27 -05:00
evazion
99846b7e3d users: allow mods to change the names of other users.
Allow moderators to forcibly change the username of other users. This is
so mods can change abusive or invalid usernames.

* A mod can only change the username of Builder-level users and below.
* The user can't change their own name again until one week has passed.
* A modaction is logged when a mod changes a user's name.
* A dmail is sent to the user notifying them of the change.
* The dmail does not send the user an email notification. This is so we
  don't spam users if their name is changed after they're banned, or if
  they haven't visited the site in a long time.

The rename button is on the user's profile page, and when you hover over
the user's name and open the "..." menu.
2022-10-02 01:32:10 -05:00
nonamethanks
775326dc37 Tumblr: fix crash when uploading image links from custom domains 2022-10-01 00:26:29 +02:00
nonamethanks
1d7caf703c Lofter: support another theme and rewrite tests 2022-09-30 22:04:40 +02:00
evazion
559bf1ae0a modqueue: fix the disapproved: metatag showing posts outside the queue.
Fix a bug where filtering the modqueue by the `disapproved:<reason>` tag
would return posts outside the modqueue.
2022-09-30 02:03:15 -05:00
nonamethanks
d51cc17eaf Nicoseiga: rewrite tests and fix several bugs
* Fixed a bug where manga posts with a single tag would raise an error
* Fixed a bug where dic.nicovideo.jp/oekaki posts weren't uploadable due
  to SSL issues
* Added support for more manga corner cases
2022-09-29 14:37:46 +02:00
evazion
d629c81aa1 Fix #5267: order[custom] no longer works when only a single ID is being searched 2022-09-29 04:36:12 -05:00
evazion
530d8cf762 searchable: fix searching for invalid IP addresses.
Fix an ArgumentError exception when searching for an invalid IP address.

Also allow searching for multiple subnets at once.
2022-09-29 04:36:12 -05:00
evazion
c5b215ffcb searchable: fix searching for polymorphic attributes.
Fix not being able to use the full set of search operators on polymorphic `model_id` and
`model_type` attributes. Before things like `search[model_type]=Post` worked, but
`search[model_type_not_eq]=Post` or other `model_type_*` operators didn't.
2022-09-29 04:36:12 -05:00
evazion
09eb763e3c searchable: fix searching by json attribute.
Fix accidentally hardcoding the attribute name to `metadata`.
2022-09-29 04:36:12 -05:00
evazion
b625f95981 emails: use separate addresses for different types of emails. 2022-09-29 04:36:12 -05:00
evazion
b94cb7d824 emails: include logging information in email headers.
Log the following information in email headers:

* X-Danbooru-User: the user's name and ID.
* X-Danbooru-IP: the user's IP.
* X-Danbooru-Session: the users' session ID.
* X-Danbooru-URL: the page that triggered the email.
* X-Danbooru-Job-Id: the ID of the background job that sent the email.
* X-Danbooru-Enqueued-At: when the email was queued as a background job.
* X-Danbooru-Dmail: for Dmail notifications, the link to the Dmail.
* X-Request-Id: the request ID of the HTTP request that triggered the email.

Also make it so we log an event in the APM when we send an email.
2022-09-29 04:36:11 -05:00
evazion
ed9986def6 emails: fix one-click unsubscription.
Fix it so that emails are (hopefully) able to show the one-click
unsubscribe button in Gmail and other mail providers that support the
List-Unsubscribe header. This way users can unsubscribe instead of
marking emails as spam.

* Add the List-Unsubscribe-Post header.
* Fix the disable email notifications endpoint to support POST as well as DELETE requests.
* Fix the disable email notifications endpoint to disable XSRF protection (we don't need
  users to be logged in because we use a signed URL instead).

https://www.rfc-editor.org/rfc/rfc8058#section-3.1
https://www.rfc-editor.org/rfc/rfc8058#section-8.1
2022-09-29 04:36:11 -05:00