Commit Graph
437 Commits
Author SHA1 Message Date
nonamethanks d525df9ad5 Newgrounds: fix exception for deleted videos
Fixup for 8c0f2255f9
2022-11-11 12:28:23 +01:00
nonamethanks 8c0f2255f9 Newgrounds: fix support for some old videos 2022-11-11 11:01:13 +01:00
evazion 15f611ca8b Add Rule34.us support. 2022-11-10 12:53:06 -06:00
evazion ec87cffe02 Add Rule34.xxx support. 2022-11-10 00:39:52 -06:00
evazion 0bad53e6ac Add TBIB.org support. 2022-11-09 16:49:20 -06:00
evazion 5b4856273f Add safebooru.org support.
Refactor the Gelbooru source extractor to support Safebooru.org as well.
2022-11-09 15:44:04 -06:00
nonamethanks 35bfcbc3bd Newgrounds: support video uploads 2022-11-09 15:01:28 +01:00
nonamethanks d7d35cb16f Fanbox: add support for embedded videos
Fixes #5339.
2022-11-09 14:02:48 +01:00
evazion 53d4052387 Merge pull request #5329 from nonamethanks/feat-bilibili
Add bilibili support
2022-11-09 01:17:18 -06:00
evazion 153d5f3e67 moebooru: support md5-based post URLs.
Support URLs like these:

* https://yande.re/post/show?md5=2c95b8975b73744da2bcbed9619c1d59
* https://konachan.com/post/show?md5=955aa45f3b452b415509b47dcc9475ac

This makes scraping images from these sites by md5 easier.
2022-11-08 21:40:16 -06:00
evazion 03a4f1a46e gelbooru: fix exception when fetching data for deleted post.
Fix exceptions when fetching deleted or nonexistent posts from Gelbooru. The Gelbooru API doesn't
return any data for deleted posts.
2022-11-08 15:34:44 -06:00
nonamethanks 0a78a6188d Reddit: support user galleries and embedded images 2022-11-06 15:04:05 +01:00
evazion af9e570419 skeb: parse www.skeb.jp URLs. 2022-11-05 19:09:56 -05:00
evazion ab08ad24e6 baraag: parse https://baraag.net/web/@{name} URLs. 2022-11-05 19:09:56 -05:00
evazion fcb68048e3 fanbox: parse https://pixiv.net/fanbox URLs. 2022-11-05 19:09:56 -05:00
evazion f5b003fc94 furaffinity: parse https://www.furaffinity.net/stats/{name} URLs. 2022-11-05 19:09:56 -05:00
nonamethanks 9f3f302328 Lofter: support another theme
Fixes #5334.
2022-11-06 00:03:15 +01:00
nonamethanks fc4d620002 Add bilibili support 2022-11-04 12:11:08 +01:00
nonamethanks e294e1193e Add reddit support 2022-10-31 15:10:27 +01:00
evazion c76e0bd4c1 gelbooru: fix normalization of old image URLs. 2022-10-30 17:26:43 -05:00
evazion 042863b2a6 tests: fix broken tests. 2022-10-30 17:26:42 -05:00
N. Oname 61112bc9a1 Merge pull request #5285 from nonamethanks/tests
Rewrite the tests for various source strategies
2022-10-23 18:05:54 +02:00
nonamethanks f5a3a7da9f Stash: rewrite tests 2022-10-22 20:09:23 +02:00
nonamethanks ec293eec2d Plurk: rewrite tests 2022-10-22 20:04:11 +02:00
nonamethanks 1b11321161 Pixiv Sketch: rewrite tests 2022-10-21 11:09:10 +02:00
nonamethanks 72528bdcb1 Twitter: rewrite tests
Also add partial support for profile banner images.
2022-10-17 18:53:11 +02:00
evazion 115521906c tumblr: fix failure to upload new Tumblr URLs containing the post title.
Fix failure to upload Tumblr URLs of this form:

* https://www.tumblr.com/munespice/683613396085719040/saur-family
2022-10-13 21:11:07 -05:00
evazion 268ec9118a tumblr: fix failure to upload certain video posts.
Fix failure to upload Tumblr video posts that contained a video URL of this form:

* https://va.media.tumblr.com/tumblr_rjoh0hR8Xe1teimlz_720.mp4
2022-10-13 21:10:29 -05:00
evazion 2e7b3cd80b tumblr: normalize https://www.tumblr.com/name artist URLs. 2022-10-12 23:55:17 -05:00
evazion 8fbc6d1d3a gelbooru: fix exception in md5-based post urls.
Fix exception when trying to get the image URL for sources like
https://gelbooru.com/index.php?page=post&s=list&md5=04f2767c64593c3030ce74ecc2528704.
2022-10-11 01:31:49 -05:00
evazion f05268df7f sources: add Gelbooru support.
Add support for uploading posts from Gelbooru. Note that the translated
tags will include both the Gelbooru tags and the tags from the Gelbooru
post's source. The commentary and artist information will also be taken
from the Gelbooru post's source. The source of the Danbooru post however
will be left as the Gelbooru post itself, not as the Gelbooru post's source.
2022-10-11 00:06:45 -05:00
evazion c2adf279ee ugoira: remove the PixivUgoiraFrameData model.
Remove the last remaining uses of the PixivUgoiraFrameData model. As of
32bfb8407, Ugoira frame data is now stored in the MediaMetadata model,
under the `Ugoira:FrameDelays` EXIF field.

The pixiv_ugoira_frame_data table still exists, but it can be removed
after this commit is deployed.

Fixes #5264: Error when replacing with ugoira.
2022-10-10 18:21:30 -05:00
nonamethanks 8008b7a5a2 Newgrounds: rewrite tests 2022-10-08 16:23:14 +02:00
nonamethanks 79a9081efa Moebooru: rewrite tests 2022-10-08 16:10:39 +02:00
nonamethanks f4b14ba23e Mastodon: rewrite tests 2022-10-08 15:55:06 +02:00
nonamethanks da46b13a64 Hentai-Foundry: rewrite tests 2022-10-07 22:22:24 +02:00
nonamethanks 1dc60feeb7 Fanbox: rewrite tests 2022-10-07 22:04:00 +02:00
nonamethanks 3c8e8ad8d9 Artstation: rewrite tests 2022-10-07 21:37:22 +02:00
nonamethanks 775326dc37 Tumblr: fix crash when uploading image links from custom domains 2022-10-01 00:26:29 +02:00
nonamethanks 1d7caf703c Lofter: support another theme and rewrite tests 2022-09-30 22:04:40 +02:00
nonamethanks d51cc17eaf Nicoseiga: rewrite tests and fix several bugs
* Fixed a bug where manga posts with a single tag would raise an error
* Fixed a bug where dic.nicovideo.jp/oekaki posts weren't uploadable due
  to SSL issues
* Added support for more manga corner cases
2022-09-29 14:37:46 +02:00
nonamethanks 5051c6649d Tumblr: parse new dashboard links 2022-09-28 17:00:08 +02:00
evazion fc122cbc5a tests: fix broken tests. 2022-09-24 03:49:10 -05:00
evazion 1d2bac7b95 Remove CurrentUser.ip_addr.
Remove the `CurrentUser.ip_addr` global variable and replace it with
`request.remote_ip`. Before we had to track the current user's IP in a
global variable so that when we edited a post for example, we could pass
down the user's IP to the model and save it in the post_versions table.
Now that we now longer save IPs in version tables, we don't need a global
variable to get access to the current user's IP outside of controllers.
2022-09-18 05:02:10 -05:00
evazion abf493794f twitter: fix misparsing of https://twitter.com/i/status/:id urls.
Fix URLs like `https://twitter.com/i/status/943446161586733056` parsing
the username as `i`. This led to the new artist page recommending the
tag name `i` when creating an artist for a source like this.

Also fix these URLs not being normalized to `https://twitter.com/:username/status/:id` after upload.
2022-09-15 19:57:12 -05:00
nonamethanks 425a905b83 tests: update tumblr tests 2022-09-15 09:48:28 +02:00
evazion d2147eca80 tumblr: fix exception when fetching data for video urls.
Fix an exception when trying to fetch source data for URLs like
https://va.media.tumblr.com/tumblr_pgohk0TjhS1u7mrsl.mp4.

For these URLs it's not possible to use the trick where we try to open
the URL as a HTML page and scrape the post id from the HTML. Instead we
get the raw video if we try to to this.
2022-09-05 16:15:47 -05:00
evazion f55951ab58 tumblr: fix exception when parsing mangled image urls.
Fix a nil exception when trying to parse invalid URLs like `https://25.media.tumblr.com/91719d337b218681abc48cdc24e`.
2022-09-05 16:15:46 -05:00
evazion 2b76a4c5ba tumblr: fix exception when parsing subdomainless Tumblr URLs.
Fix exception when a post has a Tumblr source without a subdomain, such
as `https://tumblr.com`.
2022-08-30 01:52:55 -05:00
evazion f7794de0b7 weibo: fix bad artist name suggestions in new artist form.
Fix the new artist form suggesting invalid Chinese tag names for Weibo
artists. Suggest `weibo_123456` instead as a placeholder.
2022-08-26 01:25:05 -05:00