jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
6,294 Commits 1 Branch 0 Tags
da170aa4136525bc9a79d86480758e194812f931
Commit Graph

210 Commits

Author SHA1 Message Date
r888888888
449385f08f fixes #3313 2017-10-09 12:17:15 -07:00
lllusion3469
4c3733ce5a Keep auth_secure cookie from deviantart 
Unlike this projects, all other DA scrapers I found[0] require that both
"auth" and "auth_secure" are set.
While the current approach, for some reason, seems to have worked fine in
the past, it has stopped functioning on some systems, sometimes
permanently (apparently sometimes only temporarily).
I don't have the means to find out the reason behind that, since all
configurations I've tried (mobile, VPN, and regular connection +
4 seperate accounts) resulted in me being effectively logged out and
unable to view mature works after deleting the "auth_secure" cookie.
The "if auth_cookie" part might not actually be necessary, its only
purpose is to prevent unintended breakage due to lack of information
on my part.

[0]: b662b46a44 (diff-aab4e263d591d1c7f23d015a2e540a32R161)
     cbaecf5507 (diff-9f9821e7d744874a2fe42e0a69e4b14aR33)
 2017-08-30 20:47:59 +02:00
r888888888
bc69badadd better error handling of failed deviant art logins 2017-08-28 16:19:41 -07:00
r888888888
d124ff8128 fix case when fetching pixiv sources when illust_id param comes before mode 2017-08-23 17:50:33 -07:00
evazion
9b031a9c20 Fix #3266: Normalize fullwidth Unicode characters in translated tags. 2017-08-12 17:13:52 -05:00
Albert Yi
521927ae35 Merge pull request #3255 from evazion/fix-3254 
Fix #3254, #3072: Some Twitter URLs upload wrong image.
 2017-08-07 16:03:20 -07:00
evazion
c270954991 Fix #3258: Fetch source data broken for https://i.pximg.net/* ugoiras. 2017-08-06 17:23:29 -05:00
evazion
8b70e0099b twitter: fix to handle extended tweets (fix #3254, #3072). 2017-08-05 23:12:55 -05:00
Albert Yi
36a17d79f9 Merge pull request #3242 from evazion/fix-drop-whitecube 
Drop Pixiv whitecube support (#2681, #3206).
 2017-07-26 17:07:27 -07:00
evazion
80e8c9d174 strategies/pixiv.rb: removed unused methods (#3206). 
The page count and the moniker are now obtained from the API.
 2017-07-26 18:17:04 -05:00
evazion
7ae860a8b8 Drop Pixiv whitecube support (#2681, #3206). 2017-07-26 18:06:46 -05:00
r888888888
c80291c072 fix twitter:error:forbidden catch block 2017-07-21 18:12:46 -07:00
Albert Yi
9b982b3379 Merge pull request #3228 from r888888888/ccs 
integrate with ccs
 2017-07-21 18:00:05 -07:00
r888888888
c5a40628e6 integrate with ccs 2017-07-20 15:25:19 -07:00
evazion
93d44c6b84 Fix #3224: 403 Forbidden when logging in to DeviantArt. 
Login fails if the User-Agent isn't set while POST'ing the login
credentials.
 2017-07-19 20:25:27 -05:00
r888888888
97ce9cafac add httparty option config 2017-07-12 15:52:48 -07:00
r888888888
eb6c5e3af5 switch to httparty 2017-07-12 15:52:48 -07:00
evazion
cc8986641b Fix #3208: Fix translated tag suggestions for Pixiv. 
* Only suggest the Danbooru tag with the same name if there is no
  matching wiki other name. Example: if we have the Pixiv tag `Fate` and
  the Danbooru tag `fate_(series)` with other name `fate`, suggest that,
  not the Danbooru tag `fate`.

* Don't suggest tags that are empty or whose wiki is deleted.

* Only split tags on "/" if there are no other matches, and only for Pixiv.

* For Pixiv, only include traditional media tags in tag list, not digital media (Photoshop, SAI).

* Add some tests.
 2017-07-09 12:10:30 -05:00
evazion
71f84b10af tumblr: convert commentary to dtext. 
* Convert Tumblr commentary to DText.
* Strip extraneous whitespace in links and blockquotes.
* Add newlines after block elements to ensure they're separated from
  subsequent blocks.
 2017-07-01 11:15:48 -05:00
evazion
b880b07387 sources: factor out html-to-dtext code to DText.from_html. 2017-07-01 11:15:48 -05:00
evazion
118843b26a tumblr: fetch tags, commentary, and image urls. 2017-06-25 15:34:15 -05:00
evazion
294358b4a6 nijie: fetch image_urls for batch bookmarklet. 2017-06-20 16:29:31 -05:00
evazion
2d5fc191dd nijie: convert commentary to dtext. 2017-06-20 16:11:16 -05:00
evazion
25e7db860a nijie: fetch artist commentary. 2017-06-20 16:11:16 -05:00
evazion
14196f36d8 pawoo: convert commentary to dtext. 2017-06-19 17:10:02 -05:00
evazion
6edb5a807b pawoo: fetch tags and commentary. 2017-06-19 17:10:02 -05:00
evazion
1bfc0a7d8a pawoo: get status id from normalized url. 2017-06-19 17:09:57 -05:00
evazion
e91942dff3 pawoo: add source tests and docs. 2017-06-19 17:09:57 -05:00
Albert Yi
584b01457b Merge pull request #3172 from evazion/fix-twitter-tags 
Fix #3171: Translate Twitter Hashtags to assist with tagging
 2017-06-19 13:24:13 -07:00
Albert Yi
586a3186fd Merge pull request #3167 from evazion/feat-dtextify-pixiv-artcomm 
Convert Pixiv commentaries to DText (fix #2458)
 2017-06-19 13:22:31 -07:00
Type-kun
faa2d99160 Fix #3174 2017-06-19 18:22:58 +05:00
evazion
128e655aef twitter: fetch hashtags for translated tags (fixes #3171). 2017-06-17 16:15:23 -05:00
evazion
94e48aa6f3 pixiv: convert commentary to dtext. 
Convert pixiv links to dtext in this manner:

* https://www.pixiv.net/member_illust.php?mode=medium&illust_id=1234
  => pixiv #1234 "»":[/posts?tags=pixiv:1234]

* https://www.pixiv.net/member.php?id=1234
  => "user/1234":[https://www.pixiv.net/member.php?id=1234] "»":[/artists?search[url_matches]=https://www.pixiv.net/member.php?id=1234]
 2017-06-16 18:42:37 -05:00
evazion
5dd3151d5b twitter: convert commentary to dtext. 
* Convert hashtags and mentions to dtext links.
* Replace http://t.co urls to the actual url.
* Strip the http://t.co url linking to the tweet itself.
 2017-06-16 12:56:55 -05:00
evazion
9909ff20cd twitter: memoize api response. 2017-06-16 12:56:55 -05:00
r888888888
6174d0eef2 normalize nijie popup urls (fixes #3153) 2017-06-14 12:26:13 -07:00
evazion
f3f55daea6 Fix #3138: Support DeviantArt login-only works. 2017-06-10 10:34:51 -05:00
evazion
82c7537270 deviantart: force commentary encoding to utf-8. 2017-06-09 13:19:24 -05:00
evazion
ccacfb40be deviantart: translate artist commentary from html to dtext. 2017-06-09 13:19:24 -05:00
Albert Yi
7fb0e561cc Merge pull request #3124 from evazion/fix-artstation-sources 
Multiple ArtStation fixes
 2017-06-08 14:11:21 -07:00
evazion
39bd141667 artstation: fix fetch source data for image urls. 
Fixes 'Fetch source data' when uploading an image url and the referer
url is available:

* https://danbooru.donmai.us/uploads/new?url=https://cdna.artstation.com/p/assets/images/images/006/029/978/large/amama-l-z.jpg&ref=https://www.artstation.com/artwork/4BWW2
 2017-06-07 10:20:18 -05:00
evazion
b1171306a1 artstation: include only images in image_urls. 2017-06-07 10:20:18 -05:00
evazion
a68ff3c347 artstation: accept www.artstation.com/artwork/foo-bar-baz style urls. 2017-06-07 10:20:17 -05:00
evazion
fed23c0baa fetch source data: fix gallery warning for non-pixiv galleries. 2017-06-06 23:03:07 -05:00
evazion
666aaef69c Add source support for https://$artist.artstation.com/projects/$id. 2017-06-05 14:03:38 -05:00
r888888888
09ed1ea720 another bug fix for nico seiga artist url normalization 2017-05-31 15:50:40 -07:00
r888888888
69fd6e6ca1 fixes #3102 2017-05-31 10:58:47 -07:00
r888888888
216ca06fee fixes #3100 2017-05-30 15:38:01 -07:00
r888888888
3a49b3c54b include normalized artist url in artist json 2017-05-30 12:01:18 -07:00
evazion
1a52f9db2b wiki pages: add wildcard support to "other names" search. 2017-05-24 14:59:00 -05:00