jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
12,073 Commits 1 Branch 0 Tags
c19fc16885e69947a554647e304ffa7bda59a9db
Commit Graph

165 Commits

Author SHA1 Message Date
evazion
c19fc16885 sources: don't escape Unicode characters in tag search URLs. 
Fix it so that Unicode characters aren't unnecessarily percent-encoded when generating tag search
URLs. For example, generate URLs like this:

* https://www.pixiv.net/tags/オリジナル/artworks

Not like this:

* https://www.pixiv.net/tags/%E3%82%AA%E3%83%AA%E3%82%B8%E3%83%8A%E3%83%AB/artworks
 2022-12-02 16:35:49 -06:00
evazion
9e34f4c3ed pixiv: update tag search URL to https://www.pixiv.net/tags/{name}/artworks. 2022-12-02 15:56:57 -06:00
evazion
2f508f03cf Fix #5362: Suggest ai-generated as a translated tag from pixiv response 2022-12-02 15:42:11 -06:00
evazion
2836b4b929 sources: fix 4chan URL parsing. 
Add support for is2.4chan.org image URLs and various old URL formats.
 2022-11-28 21:23:55 -06:00
evazion
096efdee7b sources: add rule34.xxx, safebooru.org, and tbib.org site icons. 
Fixes these sources incorrectly using the Gelbooru site icon.
 2022-11-28 15:56:54 -06:00
evazion
2d83500885 sources: fix dummy names being suggested as other names. 
Fix placeholder names like weibo_1234, bilibili_1234, nijie1234, and nicoseiga1234 being suggested
as Other Names when creating new artist entries. These are meant to be placeholders for the tag
name, not used as other names.
 2022-11-21 14:55:10 -06:00
evazion
d99da72744 4chan: fix commentary linebreaks. 2022-11-20 14:34:25 -06:00
evazion
902cd0bbdf sources: add 4chan.org upload support. 
Add upload support for 4chan.org. You can upload either individual posts or entire threads at a time.
 2022-11-18 01:23:15 -06:00
evazion
94824fb171 bilibili: fix API calls returning HTTP 412. 
Fix API calls returning HTTP 412 (Precondition Failed) errors. It seems we need to spoof the Referer as
`https://bilibili.com` for API calls to succeed.
 2022-11-16 22:26:01 -06:00
evazion
96e1b6a512 Fix #5063: Support downloading ArtStation video clips. 2022-11-16 22:26:01 -06:00
evazion
a747911df8 foundation: fix foundation.app/@name/~/$id type URLs. 
Fix URLs like this not working:

    https://foundation.app/@mochiiimo/~/97376

The correct URL is now:

    https://foundation.app/@mochiiimo/foundation/97376
 2022-11-16 22:26:01 -06:00
evazion
e12ce9b472 Fix #5343: Add icons for websites linked to at least 9 artists. 2022-11-12 19:49:20 -06:00
evazion
4c3b988d15 sources: fix site icons. 2022-11-11 14:03:06 -06:00
nonamethanks
d525df9ad5 Newgrounds: fix exception for deleted videos 
Fixup for 8c0f2255f9
 2022-11-11 12:28:23 +01:00
nonamethanks
8c0f2255f9 Newgrounds: fix support for some old videos 2022-11-11 11:01:13 +01:00
evazion
0fd95375fa sources: add more site icons. 
Add site icons for the following sites:

* 4chan
* Catbox
* Danbooru
* Drawcrowd
* E-Hentai
* Enty
* Gelbooru
* Hitomi
* Imgur
* Joyreactor
* Kemono Party
* Konachan
* Luscious.net
* Mega.nz
* Monappy
* Mihoyo
* Overdoll
* Pinterest
* Rule34.us
* Sankaku Complex
* Steam
* Webmshare
* Yande.re
* Zerochan

These are mainly used for source links on the https://danbooru.donmai.us/post_events?search[category]=Replacement page.
 2022-11-11 00:54:50 -06:00
evazion
15f611ca8b Add Rule34.us support. 2022-11-10 12:53:06 -06:00
evazion
ec87cffe02 Add Rule34.xxx support. 2022-11-10 00:39:52 -06:00
evazion
0bad53e6ac Add TBIB.org support. 2022-11-09 16:49:20 -06:00
evazion
5b4856273f Add safebooru.org support. 
Refactor the Gelbooru source extractor to support Safebooru.org as well.
 2022-11-09 15:44:04 -06:00
nonamethanks
35bfcbc3bd Newgrounds: support video uploads 2022-11-09 15:01:28 +01:00
nonamethanks
d7d35cb16f Fanbox: add support for embedded videos 
Fixes #5339.
 2022-11-09 14:02:48 +01:00
evazion
53d4052387 Merge pull request #5329 from nonamethanks/feat-bilibili 
Add bilibili support
 2022-11-09 01:17:18 -06:00
evazion
153d5f3e67 moebooru: support md5-based post URLs. 
Support URLs like these:

* https://yande.re/post/show?md5=2c95b8975b73744da2bcbed9619c1d59
* https://konachan.com/post/show?md5=955aa45f3b452b415509b47dcc9475ac

This makes scraping images from these sites by md5 easier.
 2022-11-08 21:40:16 -06:00
evazion
03a4f1a46e gelbooru: fix exception when fetching data for deleted post. 
Fix exceptions when fetching deleted or nonexistent posts from Gelbooru. The Gelbooru API doesn't
return any data for deleted posts.
 2022-11-08 15:34:44 -06:00
nonamethanks
0a78a6188d Reddit: support user galleries and embedded images 2022-11-06 15:04:05 +01:00
evazion
af9e570419 skeb: parse www.skeb.jp URLs. 2022-11-05 19:09:56 -05:00
evazion
ab08ad24e6 baraag: parse https://baraag.net/web/@{name} URLs. 2022-11-05 19:09:56 -05:00
evazion
fcb68048e3 fanbox: parse https://pixiv.net/fanbox URLs. 2022-11-05 19:09:56 -05:00
evazion
f5b003fc94 furaffinity: parse https://www.furaffinity.net/stats/{name} URLs. 2022-11-05 19:09:56 -05:00
nonamethanks
9f3f302328 Lofter: support another theme 
Fixes #5334.
 2022-11-06 00:03:15 +01:00
nonamethanks
758b14ad46 Remove superfluous site name definitions and update artist url order 2022-11-05 20:03:07 +01:00
evazion
60efde0e68 reddit: fix exception when work_id is nil. 2022-11-05 01:26:47 -05:00
nonamethanks
fc4d620002 Add bilibili support 2022-11-04 12:11:08 +01:00
nonamethanks
e294e1193e Add reddit support 2022-10-31 15:10:27 +01:00
evazion
c76e0bd4c1 gelbooru: fix normalization of old image URLs. 2022-10-30 17:26:43 -05:00
N. Oname
61112bc9a1 Merge pull request #5285 from nonamethanks/tests 
Rewrite the tests for various source strategies
 2022-10-23 18:05:54 +02:00
evazion
412b7f2727 http: split requests into internal and external requests. 
Split requests made by Danbooru::Http into either internal or external
requests. Internal requests are API calls to internal services run by
Danbooru. External requests are requests to external websites, for
example fetching sources or downloading files. External requests may use
a HTTP proxy if one is configured. Internal requests don't.

Fixes a few source extractors not using the HTTP proxy for certain API calls.
 2022-10-19 01:49:28 -05:00
nonamethanks
72528bdcb1 Twitter: rewrite tests 
Also add partial support for profile banner images.
 2022-10-17 18:53:11 +02:00
evazion
115521906c tumblr: fix failure to upload new Tumblr URLs containing the post title. 
Fix failure to upload Tumblr URLs of this form:

* https://www.tumblr.com/munespice/683613396085719040/saur-family
 2022-10-13 21:11:07 -05:00
evazion
268ec9118a tumblr: fix failure to upload certain video posts. 
Fix failure to upload Tumblr video posts that contained a video URL of this form:

* https://va.media.tumblr.com/tumblr_rjoh0hR8Xe1teimlz_720.mp4
 2022-10-13 21:10:29 -05:00
evazion
a07234121d tumblr: fixup for parsing www.tumblr.com/name URLs. 2022-10-13 00:26:16 -05:00
evazion
2e7b3cd80b tumblr: normalize https://www.tumblr.com/name artist URLs. 2022-10-12 23:55:17 -05:00
evazion
eb8f98e4a6 artists: normalize foriio.com artist URLs. 
Normalize `https://fori.io/comori22` to `https://www.foriio.com/comori22` in artist entries.
 2022-10-12 23:46:50 -05:00
evazion
8fbc6d1d3a gelbooru: fix exception in md5-based post urls. 
Fix exception when trying to get the image URL for sources like
https://gelbooru.com/index.php?page=post&s=list&md5=04f2767c64593c3030ce74ecc2528704.
 2022-10-11 01:31:49 -05:00
evazion
f05268df7f sources: add Gelbooru support. 
Add support for uploading posts from Gelbooru. Note that the translated
tags will include both the Gelbooru tags and the tags from the Gelbooru
post's source. The commentary and artist information will also be taken
from the Gelbooru post's source. The source of the Danbooru post however
will be left as the Gelbooru post itself, not as the Gelbooru post's source.
 2022-10-11 00:06:45 -05:00
evazion
c2adf279ee ugoira: remove the PixivUgoiraFrameData model. 
Remove the last remaining uses of the PixivUgoiraFrameData model. As of
32bfb8407, Ugoira frame data is now stored in the MediaMetadata model,
under the `Ugoira:FrameDelays` EXIF field.

The pixiv_ugoira_frame_data table still exists, but it can be removed
after this commit is deployed.

Fixes #5264: Error when replacing with ugoira.
 2022-10-10 18:21:30 -05:00
nonamethanks
f4b14ba23e Mastodon: rewrite tests 2022-10-08 15:55:06 +02:00
nonamethanks
3c8e8ad8d9 Artstation: rewrite tests 2022-10-07 21:37:22 +02:00
nonamethanks
775326dc37 Tumblr: fix crash when uploading image links from custom domains 2022-10-01 00:26:29 +02:00