Commit Graph

28 Commits

Author SHA1 Message Date
evazion
964b5efcd3 Rewrite Pixiv small/medium images to full size images.
There are two kinds of thumbnails that need to be rewritten. First case:
new /img-master/ URLs need to be rewritten to /img-original/ URLs like this:

    http://i2.pixiv.net/c/600x600/img-master/img/2014/10/04/03/59/52/46337015_p0_master1200.jpg
    => http://i2.pixiv.net/img-original/img/2014/10/04/03/59/52/46337015_p0.png

This is what `rewrite_new_medium_images` does. In order to do this, it
has to use the Pixiv API to get the correct file extension.

Second case: Old small/medium size URLs need to be rewritten to full
size URLs like this:

    http://i2.pixiv.net/img18/img/evazion/14901720_m.png
    => http://i2.pixiv.net/img18/img/evazion/14901720.png

But when the medium size URL is actually for a manga image, it needs to be
rewritten to the big manga URL instead:

    http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_m.jpg
    => http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_big_p0.jpg

But we can't tell whether it's a manga image from the URL, so we have to
use the manga page count from either the HTML page or the API to
determine whether it's part of a manga gallery.

So in order to make this work, `rewrite_old_small_and_medium_images`
takes an `is_manga` flag. `Sources::Strategies::Pixiv#get` gets the
page count from the HTML and passes the `is_manga` flag on down through
the call chain until `rewrite_old_small_and_medium_images` gets it.

When `rewrite_old_small_and_medium_images` is called from
`Downloads::Strategies::Pixiv#rewrite_thumbnails`, the `is_manga` flag
isn't passed in because we didn't scrape the HTML. This causes
`rewrite_old_small_and_medium_images` to look it up in the API instead.
2014-10-04 12:45:37 -05:00
evazion
daf196bd0e Rewrite old Pixiv small manga images correctly.
This prevents incorrectly rewriting these new Pixiv URLs:

* http://i1.pixiv.net/c/600x600/img-master/img/2014/09/24/23/25/08/46168376_p0_master1200.jpg
* http://i1.pixiv.net/img-original/img/2014/09/25/23/09/29/46183440_p0.jpg
2014-10-04 12:45:37 -05:00
Toks
4b9605075b Rewrite tumblr cdn urls 2014-10-03 14:36:37 -04:00
Toks
d499bb4d04 Remove tumblr 1280w -> 500h source rewriting
It was thought that 500h was always the biggest size when it exists, but
it seems it can actually be either bigger or smaller than 1280w.
2014-10-03 14:21:53 -04:00
Toks
3cac0458df fix regex 2014-07-16 21:10:54 -04:00
Toks
364feba64a Rewrite yandere/konachan jpeg samples 2014-07-16 20:53:33 -04:00
Toks
302fa7efcd Rewrite pixiv cdn urls 2014-07-11 02:26:19 -04:00
Toks
3cdb9fe1c2 Support new 4chan url format 2014-06-22 20:02:51 -04:00
r888888888
f65691b5e6 fix tests 2014-06-18 16:51:55 -07:00
Toks
be28a8e624 Fix Seiga sample/thumbnail rewriting 2014-06-13 16:59:08 -04:00
Toks
cb1f00b448 Fix bug where Pixiv /img-inf/ urls were incorrectly normalized 2014-06-13 16:18:25 -04:00
Toks
a3d120c632 #1866: Support HTTPS urls 2014-05-15 23:35:57 -04:00
Toks
0507064004 #1866: Add nico seiga support and fix various seiga bugs
* Support rewriting source when user uploads from a thumbnail url or
html page url
* Fix bug where site did not log in correctly
* Fix bug where the image url couldn't be extracted from the page if the
image was rated as adults only on seiga
* Normalize direct image url to html page url so tags, etc., can be
extracted
2014-04-29 11:46:08 -04:00
Toks
0cba44e3bf Fix deviantart regex bugs 2014-04-28 15:49:50 -04:00
Toks
659e0b9457 #1866: Rewrite dA PRE thumbnails 2013-12-24 16:59:04 -05:00
Toks
2ce8520e72 #1866: Support Tumblr 500h size 2013-11-29 16:12:19 -05:00
Toks
2facc02f7a #1866: Tumblr thumbnail rewriting 2013-11-28 15:02:54 -05:00
Toks
6fbd373873 #1866: Support deviantart source rewriting
* html work page -> full image
* thumbnail -> full image
2013-11-26 22:43:23 -05:00
Toks
100f3a666b #1866: Support twitpic source rewriting
* html work page -> full image
* thumbnail -> full image
2013-11-26 15:48:51 -05:00
Toks
940909ca9c #1866: Support pixiv small "_s" thumbnail source rewriting 2013-11-26 14:36:39 -05:00
albert
d1127da13f add test case for download retry 2013-03-22 19:15:12 -04:00
albert
a4b67fa072 fix tests and typo bugs 2013-03-22 18:54:37 -04:00
albert
b21dcc47a7 fixes #978 2013-03-22 10:00:46 -07:00
小太
cba839ba76 Kill trailing whitespace in ruby files 2013-03-19 23:10:10 +11:00
Peter Graham
4330e85b17 Support HTTPS in Downloads::File#http_get_streaming 2013-02-22 23:07:34 -08:00
albert
17881068e1 * Removed Pixa/Tinami sources
* Upgraded to Rails 3.2.3
* Fixed tests
2012-06-01 19:22:58 -04:00
albert
365e04bb40 fixes #194 2011-12-30 16:30:32 -05:00
albert
aa2b65c48b tinami/pixa downloads now supported 2011-09-30 14:27:04 -04:00