Commit Graph

98 Commits

Author SHA1 Message Date
r888888888
f12fb40e3e implements #2350 2015-02-20 20:04:23 -08:00
r888888888
fc30b9d606 rely on twitter api for source calls 2015-02-09 17:50:19 -08:00
r888888888
e57cad7b0a add new uploads/batch endpoint for handling twitter galleries 2015-02-09 16:29:00 -08:00
Toks
712a5844b3 Update for new twitpic link format
related #1866
2015-01-05 21:39:19 -05:00
Toks
6dca8d5915 Fix thumbnail rewrite for png images 2014-12-22 11:35:36 -05:00
Toks
4231fac4cb #2328: fix image board src rewrite 2014-12-18 11:14:25 -05:00
r888888888
43ee2875bf potential fix for #2328 2014-12-17 14:18:25 -08:00
r888888888
4fcb1d2bbc support for twitter downloads 2014-12-05 14:19:36 -08:00
r888888888
cdc6467a91 fix regexp 2014-12-03 12:48:07 -08:00
r888888888
97a71fe5a0 fixes #2298 2014-12-03 12:46:05 -08:00
r888888888
16f9a61d63 fixes #2299 2014-10-29 15:14:17 -07:00
r888888888
981e6dab67 no longer require presence of ugoira tag to process ugoira posts 2014-10-23 16:34:31 -07:00
r888888888
69564c4bff fix async processing for ugoira 2014-10-22 22:48:20 -07:00
evazion
38cb35ffbd Only fetch ugoira data for ugoira URLs.
Don't fetch ugoira data for http://i1.pixiv.net/img35/img/kinokoyarou/46165361.png URLs.
2014-10-19 02:30:03 -07:00
evazion
c0a803e3a6 Allow ugoira to be uploaded directly from the zip URL.
This refactors things such that

* Fetching the ugoira frame data happens at the end of rewrite, not
  rewrite_html_pages. This way uploading zip URLs should work.
* The source data is cached in an instance variable in case it
  was indeed fetched during rewrite_html_pages. This way it doesn't get
  fetched a second time.
2014-10-19 02:30:03 -07:00
Toks
d47327be1d Fix argument error 2014-10-19 02:30:02 -07:00
r888888888
4c73fb9f79 add ugoira support in view 2014-10-19 02:30:02 -07:00
r888888888
3bb06c2be4 integrate ugoiras into zip+webm+preview 2014-10-19 02:30:02 -07:00
r888888888
0a61aac231 refactor 2014-10-19 02:30:02 -07:00
r888888888
fb2219d4ac integrate ugoira converted into upload flow 2014-10-19 02:30:01 -07:00
evazion
964b5efcd3 Rewrite Pixiv small/medium images to full size images.
There are two kinds of thumbnails that need to be rewritten. First case:
new /img-master/ URLs need to be rewritten to /img-original/ URLs like this:

    http://i2.pixiv.net/c/600x600/img-master/img/2014/10/04/03/59/52/46337015_p0_master1200.jpg
    => http://i2.pixiv.net/img-original/img/2014/10/04/03/59/52/46337015_p0.png

This is what `rewrite_new_medium_images` does. In order to do this, it
has to use the Pixiv API to get the correct file extension.

Second case: Old small/medium size URLs need to be rewritten to full
size URLs like this:

    http://i2.pixiv.net/img18/img/evazion/14901720_m.png
    => http://i2.pixiv.net/img18/img/evazion/14901720.png

But when the medium size URL is actually for a manga image, it needs to be
rewritten to the big manga URL instead:

    http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_m.jpg
    => http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_big_p0.jpg

But we can't tell whether it's a manga image from the URL, so we have to
use the manga page count from either the HTML page or the API to
determine whether it's part of a manga gallery.

So in order to make this work, `rewrite_old_small_and_medium_images`
takes an `is_manga` flag. `Sources::Strategies::Pixiv#get` gets the
page count from the HTML and passes the `is_manga` flag on down through
the call chain until `rewrite_old_small_and_medium_images` gets it.

When `rewrite_old_small_and_medium_images` is called from
`Downloads::Strategies::Pixiv#rewrite_thumbnails`, the `is_manga` flag
isn't passed in because we didn't scrape the HTML. This causes
`rewrite_old_small_and_medium_images` to look it up in the API instead.
2014-10-04 12:45:37 -05:00
evazion
daf196bd0e Rewrite old Pixiv small manga images correctly.
This prevents incorrectly rewriting these new Pixiv URLs:

* http://i1.pixiv.net/c/600x600/img-master/img/2014/09/24/23/25/08/46168376_p0_master1200.jpg
* http://i1.pixiv.net/img-original/img/2014/09/25/23/09/29/46183440_p0.jpg
2014-10-04 12:45:37 -05:00
Toks
4b9605075b Rewrite tumblr cdn urls 2014-10-03 14:36:37 -04:00
Toks
d499bb4d04 Remove tumblr 1280w -> 500h source rewriting
It was thought that 500h was always the biggest size when it exists, but
it seems it can actually be either bigger or smaller than 1280w.
2014-10-03 14:21:53 -04:00
Toks
3cac0458df fix regex 2014-07-16 21:10:54 -04:00
Toks
364feba64a Rewrite yandere/konachan jpeg samples 2014-07-16 20:53:33 -04:00
Toks
302fa7efcd Rewrite pixiv cdn urls 2014-07-11 02:26:19 -04:00
Toks
3cdb9fe1c2 Support new 4chan url format 2014-06-22 20:02:51 -04:00
r888888888
f65691b5e6 fix tests 2014-06-18 16:51:55 -07:00
Toks
be28a8e624 Fix Seiga sample/thumbnail rewriting 2014-06-13 16:59:08 -04:00
Toks
cb1f00b448 Fix bug where Pixiv /img-inf/ urls were incorrectly normalized 2014-06-13 16:18:25 -04:00
Toks
a3d120c632 #1866: Support HTTPS urls 2014-05-15 23:35:57 -04:00
Toks
0507064004 #1866: Add nico seiga support and fix various seiga bugs
* Support rewriting source when user uploads from a thumbnail url or
html page url
* Fix bug where site did not log in correctly
* Fix bug where the image url couldn't be extracted from the page if the
image was rated as adults only on seiga
* Normalize direct image url to html page url so tags, etc., can be
extracted
2014-04-29 11:46:08 -04:00
Toks
0cba44e3bf Fix deviantart regex bugs 2014-04-28 15:49:50 -04:00
Toks
659e0b9457 #1866: Rewrite dA PRE thumbnails 2013-12-24 16:59:04 -05:00
Toks
2ce8520e72 #1866: Support Tumblr 500h size 2013-11-29 16:12:19 -05:00
Toks
2facc02f7a #1866: Tumblr thumbnail rewriting 2013-11-28 15:02:54 -05:00
Toks
6fbd373873 #1866: Support deviantart source rewriting
* html work page -> full image
* thumbnail -> full image
2013-11-26 22:43:23 -05:00
Toks
100f3a666b #1866: Support twitpic source rewriting
* html work page -> full image
* thumbnail -> full image
2013-11-26 15:48:51 -05:00
Toks
940909ca9c #1866: Support pixiv small "_s" thumbnail source rewriting 2013-11-26 14:36:39 -05:00
albert
d1127da13f add test case for download retry 2013-03-22 19:15:12 -04:00
albert
a4b67fa072 fix tests and typo bugs 2013-03-22 18:54:37 -04:00
albert
b21dcc47a7 fixes #978 2013-03-22 10:00:46 -07:00
小太
cba839ba76 Kill trailing whitespace in ruby files 2013-03-19 23:10:10 +11:00
Peter Graham
4330e85b17 Support HTTPS in Downloads::File#http_get_streaming 2013-02-22 23:07:34 -08:00
albert
17881068e1 * Removed Pixa/Tinami sources
* Upgraded to Rails 3.2.3
* Fixed tests
2012-06-01 19:22:58 -04:00
albert
365e04bb40 fixes #194 2011-12-30 16:30:32 -05:00
albert
aa2b65c48b tinami/pixa downloads now supported 2011-09-30 14:27:04 -04:00