jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6,611 Commits 1 Branch 0 Tags
59cd4d71d3cf3ed0611494ff07ae6d26a1c6c8a3
Commit Graph

102 Commits

Author SHA1 Message Date
evazion
0b5ed163fe Fix #3395: fix pixiv API exception when uploading bad pixiv id images. 
Fixes an exception when attempting to upload a Pixiv image from a
deleted work. The download strategy tries to fetch the source data in
the course of rewriting the URL, which fails if the work has been
deleted from Pixiv.

Raise a BadIDError and leave the URL as-is (don't rewrite it).
 2017-11-21 14:40:39 -06:00
evazion
fa22e419af Fix #3385: Bookmarklet sets wrong source for Tumblr posts 
Due to missing || operators, for Pawoo/Tumblr/Artstation uploads the
source wasn't changed from the direct image URL to the html page URL
after upload.
 2017-11-18 15:44:16 -06:00
r888888888
3c524aa5e6 fix bug with source::site 2017-11-17 17:26:33 -08:00
evazion
eeb41d2ffd twitter: get status id from referer url if not in direct url. 2017-11-16 13:29:58 -06:00
r888888888
c2b49bf2b7 fixes #3293 2017-09-13 11:14:35 -07:00
evazion
7ae860a8b8 Drop Pixiv whitecube support (#2681, #3206). 2017-07-26 18:06:46 -05:00
evazion
96fddc5bec Fix #3234: Incorrect filesize on upload page. 2017-07-22 00:34:30 -05:00
evazion
8125d8e796 Set the default User-Agent sent for outgoing http requests. 
* Add a `Danbooru.config.http_headers` config option.

* Refactor various places to use this option instead of setting the user
  agent manually.
 2017-07-19 20:25:27 -05:00
r888888888
ee6581ab7f restore streaming behavior for Downloads::File#http_get_streaming 2017-07-13 15:31:29 -07:00
evazion
5066f75d8c Fix Net::HTTPOK#read_body called twice error during upload (#3135) 
Fixes this exception:

    An error occurred: error: IOError - Net::HTTPOK#read_body called twice

    /home/admin/.rbenv/versions/2.3.4/lib/ruby/2.3.0/net/http/response.rb:196:in `read_body'
    /home/admin/.rbenv/versions/2.3.4/lib/ruby/gems/2.3.0/gems/httparty-0.15.5/lib/httparty/response.rb:91:in `method_missing'
    /home/admin/src/danbooru/app/logical/downloads/file.rb:43:in `block (2 levels) in download!'
    /home/admin/src/danbooru/app/logical/downloads/file.rb:41:in `open'
    /home/admin/src/danbooru/app/logical/downloads/file.rb:41:in `block in download!'
    /home/admin/src/danbooru/app/logical/downloads/file.rb:100:in `http_get_streaming'
    /home/admin/src/danbooru/app/logical/downloads/file.rb:39:in `download!'
    /home/admin/src/danbooru/app/models/upload.rb:408:in `download_from_source'
    /home/admin/src/danbooru/app/models/upload.rb:113:in `block in process_upload'
 2017-07-13 15:41:36 -05:00
r888888888
97ce9cafac add httparty option config 2017-07-12 15:52:48 -07:00
r888888888
eb6c5e3af5 switch to httparty 2017-07-12 15:52:48 -07:00
evazion
c3ac4f3329 Fix #3197 - Post Replacement comment: use Replacement URL instead of Final Source 
Record the URL of the image that was actually downloaded as the replacement URL.
 2017-07-02 21:00:26 -05:00
evazion
030052bf14 tumblr: rewrite html pages to image url. 2017-06-25 15:34:15 -05:00
evazion
118843b26a tumblr: fetch tags, commentary, and image urls. 2017-06-25 15:34:15 -05:00
evazion
421bbc35a2 tumblr: rewrite samples to biggest available version. 2017-06-21 22:30:53 -05:00
evazion
a9b3d28215 tumblr: fix edgecastcdn.net rewrite + add test. 
https://data.tumblr.com doesn't have a valid SSL cert, so we must use
http:// instead when we rewrite to it.
 2017-06-21 21:10:07 -05:00
evazion
655e53fe65 pawoo: rewrite /small/ images to /original/ 2017-06-19 17:10:02 -05:00
evazion
215569fbe0 artstation: fix source to page url after download. 2017-06-07 10:20:17 -05:00
evazion
b3e533f3ff Add download rewrite for https://$artist.artstation.com/projects/$id. 2017-06-05 14:03:51 -05:00
evazion
b2f62482a1 Remove artist commentary handling from download rewrite strategies. 2017-05-10 18:01:51 -05:00
evazion
db1fa60a43 Set pawoo sources to html page after upload. 2017-05-03 00:09:51 -05:00
r888888888
bd7f4820d7 fix source js for pawoo sites 2017-04-20 16:43:41 -07:00
r888888888
b053a2d783 add source + rewrite strategy for pawoo 2017-04-20 15:50:22 -07:00
evazion
af4708dde9 Fix typo in i.pximg.net download strategy. 2017-04-03 17:17:22 -05:00
r888888888
8d58e27d0b additional support for pximg.net hosts 2017-04-03 15:03:26 -07:00
r888888888
7a3d1c7d1d dont download original artstation images if they dont exist 2017-02-07 13:40:19 -08:00
r888888888
9996030496 include default referer url for art station strategy 2017-02-06 17:37:45 -08:00
r888888888
2e69cd619e resize medium + small artstation urls also 2017-02-06 15:22:06 -08:00
r888888888
5c16e9ce48 add download strategy for artstation 2017-02-06 12:03:00 -08:00
Albert Yi
b0f2b1e8dd Merge pull request #2782 from evazion/fix-2779 
Fix #2779: Erroneous pixiv_id parsed for novel/background/profile images
 2016-12-05 11:44:38 -08:00
Albert Yi
bd9939a50b some additional documentation about rewrite strategies 2016-12-05 11:43:18 -08:00
evazion
838d6f6eea Fix #2460: Uploading pixiv user's background image throws error. 
Fix exception when uploading these types of URLs:

* http://img12.pixiv.net/profile/rapattu/119950.jpg
* http://i2.pixiv.net/img130/profile/minono_aki/8733472.jpg
* http://i2.pixiv.net/img20/img/a-park/novel/3607898.jpg
* http://i1.pixiv.net/novel-cover-original/img/2016/11/03/20/10/58/7436075_f75af69f3eacd1656d3733c72aa959cf.jpg
* http://i1.pixiv.net/background/img/2016/05/17/12/05/48/2074388_d4ac52034f7ca0af3e083d59fde7e97f.jpg

Skip rewriting sources that don't have illust IDs to avoid this
exception. Also fix `tags` so that it doesn't blow up when fetching
source data for these types of sources.
 2016-12-04 02:04:44 -06:00
evazion
3eafc5ae6a Don't clobber image board sources. 2016-11-19 02:29:33 -06:00
evazion
f94dfbf17e Fix bookmarklet for whitecube URLs. 
Make the bookmarklet work for URLs like this:

* http://www.pixiv.net/whitecube/user/607414/illust/59510800
* http://www.pixiv.net/whitecube/user/772090/illust/59595705
 2016-10-22 16:11:44 -05:00
r888888888
4fa2741ed2 include whitecube source/rewrite strategies 2016-09-28 11:24:54 -07:00
Toks
98ef1816ba Rewrite nijie html view page to image url 2015-12-22 12:29:46 -05:00
Toks
aa212e3616 Fix error when uploading old deviantart url format 2015-09-21 10:47:24 -04:00
Toks
2359f41441 Reorganize rewrite strats slightly 2015-09-19 10:26:40 -04:00
Toks
a072fae19d Fix "EOFError - end of file reached" for https tumblr uploads 2015-09-09 10:41:59 -04:00
Toks
12be880fa9 Support fetching artist commentary from deviantart 2015-09-03 20:03:56 -04:00
r888888888
f85d5a5570 addresses #2498: revised implementation 2015-08-18 12:45:10 -07:00
r888888888
407523b04c addresses #2498: Protect source downloader against server-side request forgery attacks 2015-08-18 12:26:26 -07:00
Toks
e165cb0ab2 fixes #2491 2015-08-15 17:12:00 -04:00
Toks
6ea556944b Add support for uploading from seiga /o/ pages 2015-07-07 18:00:47 -04:00
Toks
462e1ed324 Implement #2395 for nijie twitter and tumblr 2015-06-13 11:26:47 -04:00
Toks
7360fc4633 fix typo 2015-06-13 10:56:27 -04:00
r888888888
2ddc93723c fixes #2327 2015-06-02 19:20:09 -07:00
Toks
367b6495ae rename 2015-05-28 02:25:25 -04:00
Toks
9481f09b93 Support rewriting direct twitter url -> full size url 
If the user gives us the direct image url the twitter api is useless
since it doesn't have the status id. So just rewrite the url itself.
 2015-05-28 02:20:45 -04:00