jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
8,092 Commits 1 Branch 0 Tags
be36968b6d8ae4b2c8d2b2dfd544eada20046bea
Commit Graph

190 Commits

Author SHA1 Message Date
evazion
eba6440b8b Fix #4144: Deviantart Eclipse update broke strategy. 2019-08-28 23:40:29 -05:00
evazion
7f482dc35b deviantart: normalize wixmp.com sources to page urls. 
Normalize sources like this:

   https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com/f/ab917938-d8c1-4b58-933d-0d38c390461f/ddcyi98-cbd39da2-f528-4b26-aadb-a16fe91442b2.jpg/v1/fill/w_1280,h_1760,q_100,strp/tifa_by_chubymi_ddcyi98-fullview.jpg?token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJ1cm46YXBwOjdlMGQxODg5ODIyNjQzNzNhNWYwZDQxNWVhMGQyNmUwIiwiaXNzIjoidXJuOmFwcDo3ZTBkMTg4OTgyMjY0MzczYTVmMGQ0MTVlYTBkMjZlMCIsIm9iaiI6W1t7ImhlaWdodCI6Ijw9MTc2MCIsInBhdGgiOiJcL2ZcL2FiOTE3OTM4LWQ4YzEtNGI1OC05MzNkLTBkMzhjMzkwNDYxZlwvZGRjeWk5OC1jYmQzOWRhMi1mNTI4LTRiMjYtYWFkYi1hMTZmZTkxNDQyYjIuanBnIiwid2lkdGgiOiI8PTEyODAifV1dLCJhdWQiOlsidXJuOnNlcnZpY2U6aW1hZ2Uub3BlcmF0aW9ucyJdfQ.Uochlma4QJmLwL2ZGMmvTr4HMva4m4bCeF3vnyPSw4I

to this:

   https://www.deviantart.com/chubymi/art/Tifa-807825644

on the sidebar of the posts show page.
 2019-08-06 12:04:05 -05:00
evazion
a8896b664d twitter: fix batch bookmarklet selecting wrong image. 
Fix regression in 7e465aeda.

https://danbooru.donmai.us/forum_topics/9127?page=276#forum_post_158779
 2019-08-06 10:42:45 -05:00
evazion
7e465aedae Fix #4110: New Twitter image urls are broken in bookmarklet. 2019-08-04 20:23:10 -05:00
evazion
9ecf36585c nijie: update for new image urls. 
Nijie moved from this:

    https://pic03.nijie.info/nijie_picture/236014_20170620101426_0.png (page: https://www.nijie.info/view.php?id=218856)

to this:

    https://pic.nijie.net/03/nijie_picture/236014_20170620101426_0.png (page: https://www.nijie.info/view.php?id=218856)
 2019-08-04 17:49:54 -05:00
Albert Yi
6612aa1af9 fix unit tests 2019-04-25 18:13:59 -07:00
Albert Yi
6f4ca75bb1 try to handle deivantart images hosted on images-wixmp better 2019-04-16 13:55:26 -07:00
Albert Yi
eb8ce2e955 Revert "eliminate old algorithm for deviant art sources, fix canonical_url on deviantart" 
This reverts commit 58ca52b43e.
 2019-04-16 13:22:09 -07:00
Albert Yi
58ca52b43e eliminate old algorithm for deviant art sources, fix canonical_url on deviantart 2019-04-12 16:35:41 -07:00
Albert Yi
f6a11e6363 remove residual code 2019-02-25 14:46:43 -08:00
Albert Yi
90ce42a537 add support for nico seiga manga (fixes #4060) 2019-02-25 14:44:45 -08:00
evazion
8cadef2dd7 pixiv: fix illust id parsing (fix #4043). 
* Tighten up illust id parsing to avoid misparsing ids from
  non-illust urls (sketch urls and novel urls).

* Move id parsing tests from post_test.rb to sources/pixiv_test.rb.

* Drop support for touch.pixiv.net urls. These urls are no longer used
  by Pixiv and aren't present as the source of any posts on Danbooru.
 2019-01-13 14:28:51 -06:00
evazion
04d5b16da7 pixiv: fix failure to upload bad pixiv id images (fix #4031) 
Bug: Uploading bad pixiv id images failed because the pixiv strategy
raised a BadIDError exception when the upload service checked for the
ugoira frame data.
 2019-01-03 18:01:20 -06:00
evazion
6a77b68b74 tumblr: fix tests. 2018-12-27 15:03:11 -06:00
evazion
0f513d1a1b twitter: include intent url in new artist entries (#4028). 2018-12-27 15:03:11 -06:00
evazion
2129e60b2b pixiv: include stacc url in new artist entries (#4028). 2018-12-27 15:03:11 -06:00
evazion
c700ea4b5f Fix #4016: Translated tags failing to find some tags. 
* Normalize spaces to underscores when saving other names. Preserve case
  since case can be significant.

* Fix WikiPage#other_names_include to search case-insensitively (note:
  this prevents using the index).

* Fix sources to return the raw tags in `#tags` and the normalized tags
  in `#normalized_tags`. The normalized tags are the tags that will be
  matched against other names.
 2018-12-16 11:37:57 -06:00
evazion
fd2d56a388 Fix #3996: Some direct artstation urls cause the bookmarklet to fail. 
* Don't fail on urls that don't contain the project id (direct image urls).
* Don't fail when the work is deleted.
* Parse artist name from url when possible. This way the artist finder works on bad_artstation_id posts.
* Set canonical source url to `https://artist.artstation.com/projects/12345` instead of
  `https://www.artstation.com/artwork/1235` (this way we preserve the artist name).
* Cache api call.
* Include api call results in /source.json.
 2018-12-03 15:48:29 -06:00
evazion
e8c3d92696 Fix #3977: DeviantArt: Support new wixmp.com urls. 2018-11-30 00:10:59 -06:00
evazion
8da3474a8b Fix #3994: Some deviantart images mistakenly choosing preview image. 2018-11-29 23:01:08 -06:00
evazion
bea8c2a4b8 nijie: fix failure to handle certain image urls. 
Fix IMAGE_URL regex not matching urls of this form:

* https://pic04.nijie.info/nijie_picture/diff/main/287736_161475_20181112032855_1.png

This caused the illust id to not be parsed from the url, which led to `#image_url`
returning nil, which led to uploads failing because the url to download was missing.
 2018-11-12 18:04:07 -06:00
evazion
8f6c710c6b tests: fix translated tags test failures. 2018-11-12 18:04:07 -06:00
evazion
6a7cd6ce8e Fix #3984: Twitter: undefined method `first' for nil:NilClass. 
Fix Sources::Strategies::Twitter#image_urls to return an empty array
instead of nil when the tweet doesn't contain any images.
 2018-11-11 17:41:32 -06:00
evazion
5cf6a43918 sources: fix sources sometimes choosing wrong strategy (fix #3968) 
Fix sources choosing the wrong strategy when the referer belongs to a
different site (for example, when uploading a twitter post with a pixiv
referer).

* Fix `match?` to only consider the main url, not the referer.

* Change `match?` to match against a list of domains given by the `domains` method.

* Change `match?` to an instance method.
 2018-11-04 13:00:17 -06:00
evazion
53a51310a3 tumblr: add canonical url tests (#3385). 2018-10-09 12:55:48 -05:00
evazion
16b1b72da5 tumblr: fix video urls not being recognized. 2018-10-09 12:44:59 -05:00
evazion
184a5ebf3e tumblr: fix _640 images not being recognized (#3944). 
Fixes _640 images not being matched by the IMAGE regex and therefore not
being rewritten to the largest size.
 2018-10-09 12:44:59 -05:00
evazion
d874c68419 tumblr: fix image_urls when api data is unavailable. 2018-10-09 12:44:59 -05:00
evazion
b0d7d90103 tumblr: extract info from url when api data is unavailable. 
Derive the artist name / profile url / page url from the source URLs when
the API response is unavailable because the Tumblr post was deleted.

This fixes the artist finder to work on bad_tumblr_id posts.
 2018-10-09 12:44:59 -05:00
evazion
0c31a5d6a9 tumblr: don't fail when api data is unavailable (#3948). 
The api data is unavailable when the work is deleted (bad_tumblr_id), or
when the source is a direct image url with no page referer.
 2018-10-09 12:44:59 -05:00
evazion
4c55c809b0 tumblr: don't fail when api key isn't configured. 2018-10-09 12:44:59 -05:00
evazion
c8d538f618 moebooru: delegate to substrategy based on post source (#3911). 
If the yande.re or konachan.com post has a source from a supported site,
for example Pixiv or Twitter, then delegate the artist and commentary
lookup to that substrategy.

Only do this for sources from recognized sites, not the null strategy.
 2018-10-06 14:27:49 -05:00
evazion
e5a4193dd4 moebooru: support batch bookmarklet previews (#3911). 2018-10-06 00:58:22 -05:00
evazion
fdb6e4ecee moebooru: rewrite konachan urls for Post#normalized_source (#3911). 2018-10-06 00:58:22 -05:00
evazion
864349dc7b moebooru: fetch tags (#3911). 2018-10-06 00:58:22 -05:00
evazion
bd3fb7d70e Post#normalized_source: fix for yande.re urls. 
Fix regex for yande.re urls like this:

    https://files.yande.re/image/b66909b940e8d77accab7c9b25aa4dc3/yande.re%20377828.png
 2018-10-01 20:03:21 -05:00
evazion
d9063a9f2a nijie: support preview urls (#3919). 2018-09-24 17:08:37 -05:00
evazion
5525bbe1ca nijie: normalize all thumbnail urls (#3919). 2018-09-23 20:08:14 -05:00
evazion
d294514dc0 nijie: don't crash on invalid urls or deleted works (#3919). 2018-09-23 20:08:14 -05:00
evazion
b6228505aa nijie: fix page_url method. 
The id in a bare image url is the member id, not the illust id.
 2018-09-23 20:08:13 -05:00
evazion
39f9e01b13 nicoseiga: fix canonical_url to use the image url. 2018-09-22 11:07:18 -05:00
evazion
958a9f505b moebooru: rewrite sample urls + support bookmarklet on html page. 
* Fixes #2942: Add Moebooru Rewrite for Sample Images.
* Addresses #3911: Improve Moebooru support.
 2018-09-19 23:32:21 -05:00
evazion
96e89cecfb tests: move twitter canonical url test. 2018-09-17 23:27:53 -05:00
evazion
4a99cb098f moebooru: use the image url as the canonical url. 2018-09-16 21:00:11 -05:00
evazion
d9ce953752 Fix #3906: Moebooru strategy raises NotImplementedError. 2018-09-16 21:00:11 -05:00
evazion
cae78fa8ee moebooru: move tests from unit/downloads to unit/sources. 2018-09-16 21:00:11 -05:00
evazion
f135a7c064 twitter: normalize canonical urls. 
Normalize http://mobile.twitter.com to http://twitter.com in canonical urls.
 2018-09-16 15:03:47 -05:00
evazion
bd47641601 twitter: don't fail when api key isn't configured. 2018-09-16 15:03:47 -05:00
evazion
325120ee51 twitter: fix parsing of the artist name from the url. 
Fixes URLs like https://twitter.com/intent/user?user_id=123 being
incorrectly normalized to http://twitter.com/intent/ in artist entries.

Also fixes the artist name to be taken from the url when it can't be
obtained from the api (when the tweet is deleted).
 2018-09-16 15:03:23 -05:00
evazion
fbd5f6b7f2 pixiv: fix preview_urls for ugoiras (#3891). 2018-09-12 00:43:10 -05:00