Commit Graph

57 Commits

Author SHA1 Message Date
Toks
462e1ed324 Implement #2395 for nijie twitter and tumblr 2015-06-13 11:26:47 -04:00
Toks
7360fc4633 fix typo 2015-06-13 10:56:27 -04:00
r888888888
2ddc93723c fixes #2327 2015-06-02 19:20:09 -07:00
Toks
367b6495ae rename 2015-05-28 02:25:25 -04:00
Toks
9481f09b93 Support rewriting direct twitter url -> full size url
If the user gives us the direct image url the twitter api is useless
since it doesn't have the status id. So just rewrite the url itself.
2015-05-28 02:20:45 -04:00
Toks
bb4267af7d remove debug 2015-05-13 18:32:02 -04:00
r888888888
ac6d1fa498 bug fix with pixiv api 2015-05-11 15:53:45 -07:00
Toks
04e6aa7474 #1866 update for new deviantart urls 2015-03-13 14:12:17 -04:00
r888888888
b353b21c53 fixes #2353 2015-03-05 17:58:00 -08:00
r888888888
f12fb40e3e implements #2350 2015-02-20 20:04:23 -08:00
r888888888
fc30b9d606 rely on twitter api for source calls 2015-02-09 17:50:19 -08:00
r888888888
e57cad7b0a add new uploads/batch endpoint for handling twitter galleries 2015-02-09 16:29:00 -08:00
Toks
712a5844b3 Update for new twitpic link format
related #1866
2015-01-05 21:39:19 -05:00
Toks
6dca8d5915 Fix thumbnail rewrite for png images 2014-12-22 11:35:36 -05:00
Toks
4231fac4cb #2328: fix image board src rewrite 2014-12-18 11:14:25 -05:00
r888888888
43ee2875bf potential fix for #2328 2014-12-17 14:18:25 -08:00
r888888888
4fcb1d2bbc support for twitter downloads 2014-12-05 14:19:36 -08:00
r888888888
cdc6467a91 fix regexp 2014-12-03 12:48:07 -08:00
r888888888
97a71fe5a0 fixes #2298 2014-12-03 12:46:05 -08:00
r888888888
16f9a61d63 fixes #2299 2014-10-29 15:14:17 -07:00
r888888888
981e6dab67 no longer require presence of ugoira tag to process ugoira posts 2014-10-23 16:34:31 -07:00
r888888888
69564c4bff fix async processing for ugoira 2014-10-22 22:48:20 -07:00
evazion
38cb35ffbd Only fetch ugoira data for ugoira URLs.
Don't fetch ugoira data for http://i1.pixiv.net/img35/img/kinokoyarou/46165361.png URLs.
2014-10-19 02:30:03 -07:00
evazion
c0a803e3a6 Allow ugoira to be uploaded directly from the zip URL.
This refactors things such that

* Fetching the ugoira frame data happens at the end of rewrite, not
  rewrite_html_pages. This way uploading zip URLs should work.
* The source data is cached in an instance variable in case it
  was indeed fetched during rewrite_html_pages. This way it doesn't get
  fetched a second time.
2014-10-19 02:30:03 -07:00
Toks
d47327be1d Fix argument error 2014-10-19 02:30:02 -07:00
r888888888
4c73fb9f79 add ugoira support in view 2014-10-19 02:30:02 -07:00
r888888888
3bb06c2be4 integrate ugoiras into zip+webm+preview 2014-10-19 02:30:02 -07:00
r888888888
0a61aac231 refactor 2014-10-19 02:30:02 -07:00
r888888888
fb2219d4ac integrate ugoira converted into upload flow 2014-10-19 02:30:01 -07:00
evazion
964b5efcd3 Rewrite Pixiv small/medium images to full size images.
There are two kinds of thumbnails that need to be rewritten. First case:
new /img-master/ URLs need to be rewritten to /img-original/ URLs like this:

    http://i2.pixiv.net/c/600x600/img-master/img/2014/10/04/03/59/52/46337015_p0_master1200.jpg
    => http://i2.pixiv.net/img-original/img/2014/10/04/03/59/52/46337015_p0.png

This is what `rewrite_new_medium_images` does. In order to do this, it
has to use the Pixiv API to get the correct file extension.

Second case: Old small/medium size URLs need to be rewritten to full
size URLs like this:

    http://i2.pixiv.net/img18/img/evazion/14901720_m.png
    => http://i2.pixiv.net/img18/img/evazion/14901720.png

But when the medium size URL is actually for a manga image, it needs to be
rewritten to the big manga URL instead:

    http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_m.jpg
    => http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_big_p0.jpg

But we can't tell whether it's a manga image from the URL, so we have to
use the manga page count from either the HTML page or the API to
determine whether it's part of a manga gallery.

So in order to make this work, `rewrite_old_small_and_medium_images`
takes an `is_manga` flag. `Sources::Strategies::Pixiv#get` gets the
page count from the HTML and passes the `is_manga` flag on down through
the call chain until `rewrite_old_small_and_medium_images` gets it.

When `rewrite_old_small_and_medium_images` is called from
`Downloads::Strategies::Pixiv#rewrite_thumbnails`, the `is_manga` flag
isn't passed in because we didn't scrape the HTML. This causes
`rewrite_old_small_and_medium_images` to look it up in the API instead.
2014-10-04 12:45:37 -05:00
evazion
daf196bd0e Rewrite old Pixiv small manga images correctly.
This prevents incorrectly rewriting these new Pixiv URLs:

* http://i1.pixiv.net/c/600x600/img-master/img/2014/09/24/23/25/08/46168376_p0_master1200.jpg
* http://i1.pixiv.net/img-original/img/2014/09/25/23/09/29/46183440_p0.jpg
2014-10-04 12:45:37 -05:00
Toks
4b9605075b Rewrite tumblr cdn urls 2014-10-03 14:36:37 -04:00
Toks
d499bb4d04 Remove tumblr 1280w -> 500h source rewriting
It was thought that 500h was always the biggest size when it exists, but
it seems it can actually be either bigger or smaller than 1280w.
2014-10-03 14:21:53 -04:00
Toks
3cac0458df fix regex 2014-07-16 21:10:54 -04:00
Toks
364feba64a Rewrite yandere/konachan jpeg samples 2014-07-16 20:53:33 -04:00
Toks
302fa7efcd Rewrite pixiv cdn urls 2014-07-11 02:26:19 -04:00
Toks
3cdb9fe1c2 Support new 4chan url format 2014-06-22 20:02:51 -04:00
r888888888
f65691b5e6 fix tests 2014-06-18 16:51:55 -07:00
Toks
be28a8e624 Fix Seiga sample/thumbnail rewriting 2014-06-13 16:59:08 -04:00
Toks
cb1f00b448 Fix bug where Pixiv /img-inf/ urls were incorrectly normalized 2014-06-13 16:18:25 -04:00
Toks
a3d120c632 #1866: Support HTTPS urls 2014-05-15 23:35:57 -04:00
Toks
0507064004 #1866: Add nico seiga support and fix various seiga bugs
* Support rewriting source when user uploads from a thumbnail url or
html page url
* Fix bug where site did not log in correctly
* Fix bug where the image url couldn't be extracted from the page if the
image was rated as adults only on seiga
* Normalize direct image url to html page url so tags, etc., can be
extracted
2014-04-29 11:46:08 -04:00
Toks
0cba44e3bf Fix deviantart regex bugs 2014-04-28 15:49:50 -04:00
Toks
659e0b9457 #1866: Rewrite dA PRE thumbnails 2013-12-24 16:59:04 -05:00
Toks
2ce8520e72 #1866: Support Tumblr 500h size 2013-11-29 16:12:19 -05:00
Toks
2facc02f7a #1866: Tumblr thumbnail rewriting 2013-11-28 15:02:54 -05:00
Toks
6fbd373873 #1866: Support deviantart source rewriting
* html work page -> full image
* thumbnail -> full image
2013-11-26 22:43:23 -05:00
Toks
100f3a666b #1866: Support twitpic source rewriting
* html work page -> full image
* thumbnail -> full image
2013-11-26 15:48:51 -05:00
Toks
940909ca9c #1866: Support pixiv small "_s" thumbnail source rewriting 2013-11-26 14:36:39 -05:00
albert
d1127da13f add test case for download retry 2013-03-22 19:15:12 -04:00