jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6,050 Commits 1 Branch 0 Tags
d786fc7a55cabc70ea35dfee283b7cd2ccf11e32
Commit Graph

171 Commits

Author SHA1 Message Date
evazion
71f84b10af tumblr: convert commentary to dtext. 
* Convert Tumblr commentary to DText.
* Strip extraneous whitespace in links and blockquotes.
* Add newlines after block elements to ensure they're separated from
  subsequent blocks.
 2017-07-01 11:15:48 -05:00
evazion
b880b07387 sources: factor out html-to-dtext code to DText.from_html. 2017-07-01 11:15:48 -05:00
evazion
118843b26a tumblr: fetch tags, commentary, and image urls. 2017-06-25 15:34:15 -05:00
evazion
294358b4a6 nijie: fetch image_urls for batch bookmarklet. 2017-06-20 16:29:31 -05:00
evazion
2d5fc191dd nijie: convert commentary to dtext. 2017-06-20 16:11:16 -05:00
evazion
25e7db860a nijie: fetch artist commentary. 2017-06-20 16:11:16 -05:00
evazion
14196f36d8 pawoo: convert commentary to dtext. 2017-06-19 17:10:02 -05:00
evazion
6edb5a807b pawoo: fetch tags and commentary. 2017-06-19 17:10:02 -05:00
evazion
1bfc0a7d8a pawoo: get status id from normalized url. 2017-06-19 17:09:57 -05:00
evazion
e91942dff3 pawoo: add source tests and docs. 2017-06-19 17:09:57 -05:00
Albert Yi
584b01457b Merge pull request #3172 from evazion/fix-twitter-tags 
Fix #3171: Translate Twitter Hashtags to assist with tagging
 2017-06-19 13:24:13 -07:00
evazion
128e655aef twitter: fetch hashtags for translated tags (fixes #3171). 2017-06-17 16:15:23 -05:00
evazion
94e48aa6f3 pixiv: convert commentary to dtext. 
Convert pixiv links to dtext in this manner:

* https://www.pixiv.net/member_illust.php?mode=medium&illust_id=1234
  => pixiv #1234 "»":[/posts?tags=pixiv:1234]

* https://www.pixiv.net/member.php?id=1234
  => "user/1234":[https://www.pixiv.net/member.php?id=1234] "»":[/artists?search[url_matches]=https://www.pixiv.net/member.php?id=1234]
 2017-06-16 18:42:37 -05:00
evazion
5dd3151d5b twitter: convert commentary to dtext. 
* Convert hashtags and mentions to dtext links.
* Replace http://t.co urls to the actual url.
* Strip the http://t.co url linking to the tweet itself.
 2017-06-16 12:56:55 -05:00
evazion
9909ff20cd twitter: memoize api response. 2017-06-16 12:56:55 -05:00
r888888888
6174d0eef2 normalize nijie popup urls (fixes #3153) 2017-06-14 12:26:13 -07:00
evazion
f3f55daea6 Fix #3138: Support DeviantArt login-only works. 2017-06-10 10:34:51 -05:00
evazion
82c7537270 deviantart: force commentary encoding to utf-8. 2017-06-09 13:19:24 -05:00
evazion
ccacfb40be deviantart: translate artist commentary from html to dtext. 2017-06-09 13:19:24 -05:00
Albert Yi
7fb0e561cc Merge pull request #3124 from evazion/fix-artstation-sources 
Multiple ArtStation fixes
 2017-06-08 14:11:21 -07:00
evazion
b1171306a1 artstation: include only images in image_urls. 2017-06-07 10:20:18 -05:00
evazion
a68ff3c347 artstation: accept www.artstation.com/artwork/foo-bar-baz style urls. 2017-06-07 10:20:17 -05:00
evazion
fed23c0baa fetch source data: fix gallery warning for non-pixiv galleries. 2017-06-06 23:03:07 -05:00
evazion
666aaef69c Add source support for https://$artist.artstation.com/projects/$id. 2017-06-05 14:03:38 -05:00
r888888888
09ed1ea720 another bug fix for nico seiga artist url normalization 2017-05-31 15:50:40 -07:00
r888888888
216ca06fee fixes #3100 2017-05-30 15:38:01 -07:00
Albert Yi
2f1faf3c38 Merge pull request #3060 from evazion/fix-3048 
Fix #3048: Artist Lookup Failing Due to Changing Usernames
 2017-05-15 16:33:53 -07:00
Albert Yi
7d2e1f871b Merge pull request #3056 from evazion/fix-3054 
Fix #3054: Artist link broken on Post Upload page
 2017-05-15 16:17:26 -07:00
evazion
eac827973b Normalize pixiv artist urls to pixiv member page (fix #3048). 2017-05-15 14:12:08 -05:00
evazion
6ee3b2635a Fix returning incorrect pixiv profile url (fixes #3054). 2017-05-14 22:00:06 -05:00
evazion
61a37a09cc Fix translated tags for pixiv (fix #3046). 
Translated tags expects `tags` to be an array of `[tag_name, search_link]` pairs.
 2017-05-13 19:17:33 -05:00
r888888888
7ae18d1068 bug fix for source pixiv 2017-05-12 16:42:25 -07:00
r888888888
bcec293efe pull additional data from pixiv api (fixes #3042) 2017-05-12 11:17:31 -07:00
evazion
757a7da49b fixup! /source.json: don't encode '&' as '&' in commentaries. 2017-05-10 21:03:29 -05:00
evazion
a2a6a0ce27 /source.json: don't encode '&' as '&' in commentaries. 2017-05-10 18:01:52 -05:00
evazion
8015d62b46 /source.json: strip html tags from artist commentary. 2017-05-10 18:01:51 -05:00
evazion
b2f62482a1 Remove artist commentary handling from download rewrite strategies. 2017-05-10 18:01:51 -05:00
r888888888
bd7f4820d7 fix source js for pawoo sites 2017-04-20 16:43:41 -07:00
r888888888
b053a2d783 add source + rewrite strategy for pawoo 2017-04-20 15:50:22 -07:00
evazion
4b138be6b9 Rewrite i.pximg.net manga gallery samples. 2017-04-03 17:17:22 -05:00
r888888888
54f0474d49 fixes #2930: Support new Pixiv image hostname (i.pximg.net) 2017-03-27 15:53:32 -07:00
evazion
21e76d55e6 Remove extraneous byte-ordering marks. 2017-02-28 19:01:10 -06:00
r888888888
9996030496 include default referer url for art station strategy 2017-02-06 17:37:45 -08:00
r888888888
39b6601afe Merge branch 'artstation' 2017-02-06 17:07:57 -08:00
r888888888
7a2f652152 add source strategy for artstation 2017-02-06 17:07:09 -08:00
evazion
98b0b2c5d8 tests: fix Net::HTTP::Persistent::Error: too many connection resets. 
Works around connection reset errors in the test suite by disabling
persistent connections.

  20) Error:
Sources::PixivTest#test_: in all cases fetching source data for a new manga image should get the tags. :
Net::HTTP::Persistent::Error: too many connection resets (due to closed stream - IOError) after 0 requests on 47071328584700, last used 1.842702476 seconds ago
  app/logical/pixiv_web_agent.rb:46:in `build'
  app/logical/sources/strategies/pixiv.rb:104:in `agent'
  app/logical/sources/strategies/pixiv.rb:72:in `get'
  app/logical/sources/site.rb:6:in `get'
  test/unit/sources/pixiv_test.rb:7:in `get_source'
  test/unit/sources/pixiv_test.rb:64:in `block (3 levels) in <class:PixivTest>'

ref: github.com/sparklemotion/mechanize/issues/123
ref: http://www.rubydoc.info/gems/mechanize/Mechanize#retry_change_requests%3D-instance_method
 2017-02-04 17:07:00 -06:00
Albert Yi
268b7b99ad fix nil error 2017-01-16 15:41:46 -08:00
evazion
5f4535c408 Fix #2839: "Original" tag shows up twice in Pixiv source data 2017-01-14 21:14:06 +00:00
Albert Yi
3ad639521f fixes #2805: Improve nico seiga support 2016-12-27 16:11:22 -08:00
Albert Yi
b0f2b1e8dd Merge pull request #2782 from evazion/fix-2779 
Fix #2779: Erroneous pixiv_id parsed for novel/background/profile images
 2016-12-05 11:44:38 -08:00