Commit Graph

29 Commits

Author SHA1 Message Date
evazion
9b031a9c20 Fix #3266: Normalize fullwidth Unicode characters in translated tags. 2017-08-12 17:13:52 -05:00
evazion
cc8986641b Fix #3208: Fix translated tag suggestions for Pixiv.
* Only suggest the Danbooru tag with the same name if there is no
  matching wiki other name. Example: if we have the Pixiv tag `Fate` and
  the Danbooru tag `fate_(series)` with other name `fate`, suggest that,
  not the Danbooru tag `fate`.

* Don't suggest tags that are empty or whose wiki is deleted.

* Only split tags on "/" if there are no other matches, and only for Pixiv.

* For Pixiv, only include traditional media tags in tag list, not digital media (Photoshop, SAI).

* Add some tests.
2017-07-09 12:10:30 -05:00
evazion
fed23c0baa fetch source data: fix gallery warning for non-pixiv galleries. 2017-06-06 23:03:07 -05:00
evazion
757a7da49b fixup! /source.json: don't encode '&' as '&' in commentaries. 2017-05-10 21:03:29 -05:00
evazion
a2a6a0ce27 /source.json: don't encode '&' as '&' in commentaries. 2017-05-10 18:01:52 -05:00
evazion
8015d62b46 /source.json: strip html tags from artist commentary. 2017-05-10 18:01:51 -05:00
evazion
b2f62482a1 Remove artist commentary handling from download rewrite strategies. 2017-05-10 18:01:51 -05:00
Albert Yi
268b7b99ad fix nil error 2017-01-16 15:41:46 -08:00
evazion
5f4535c408 Fix #2839: "Original" tag shows up twice in Pixiv source data 2017-01-14 21:14:06 +00:00
Albert Yi
b0f2b1e8dd Merge pull request #2782 from evazion/fix-2779
Fix #2779: Erroneous pixiv_id parsed for novel/background/profile images
2016-12-05 11:44:38 -08:00
Albert Yi
bd9939a50b some additional documentation about rewrite strategies 2016-12-05 11:43:18 -08:00
evazion
838d6f6eea Fix #2460: Uploading pixiv user's background image throws error.
Fix exception when uploading these types of URLs:

* http://img12.pixiv.net/profile/rapattu/119950.jpg
* http://i2.pixiv.net/img130/profile/minono_aki/8733472.jpg
* http://i2.pixiv.net/img20/img/a-park/novel/3607898.jpg
* http://i1.pixiv.net/novel-cover-original/img/2016/11/03/20/10/58/7436075_f75af69f3eacd1656d3733c72aa959cf.jpg
* http://i1.pixiv.net/background/img/2016/05/17/12/05/48/2074388_d4ac52034f7ca0af3e083d59fde7e97f.jpg

Skip rewriting sources that don't have illust IDs to avoid this
exception. Also fix `tags` so that it doesn't blow up when fetching
source data for these types of sources.
2016-12-04 02:04:44 -06:00
Toks
0a5f1b53ba #2426 refactor to use predetermined referer 2015-07-05 11:02:01 -04:00
r888888888
866347a9f7 fix errors 2015-06-11 21:38:41 -07:00
r888888888
fd74f860ee potential fix for #2404 2015-06-10 17:28:51 -07:00
Toks
854d587373 Fix upload source fetcher fetching from wrong work page for all sites
e.g. If you were on an html work page on pixiv, clicked a link to a
different html work page on pixiv, and then clicked the bookmarklet,
then it used to fetch the source from the FIRST work you were on instead
of the second.
2015-06-03 20:59:24 -04:00
r888888888
2ddc93723c fixes #2327 2015-06-02 19:20:09 -07:00
r888888888
2d266fc0b6 uploads/batch now supports pixiv 2015-03-15 13:56:16 -07:00
r888888888
f12fb40e3e implements #2350 2015-02-20 20:04:23 -08:00
evazion
57bb51621e Add debugger gem.
Fix random VCR failures in Pixiv tests.

Sometimes tests randomly fail because the PHPSESSID they use in their
HTTP requests to Pixiv is different than the one that was originally
recorded by VCR. This causes VCR to complain that the requests don't
match.

This is caused by the PHPSESSID being globally cached in Memcache.
Depending on the order the tests run in (which is random), one set of
tests can use a PHPSESSID that was recorded for a /different/ set of
tests.

Improve Pixiv URL matching.

* Allow URLs that are missing the http:// part. These are sometimes seen
  in artist entries.
* Ignore URLs from random Pixiv domains such as dic.pixiv.net,
  blog.pixiv.net, etc. These are also sometimes in artist entries.

Improve normalize_for_artist_finder! URL matching.

* Normalize www.pixiv.net/stacc/username URLs.
* Correctly normalize URLs that are missing the illust ID part on the end
  (i.e. http://i2.pixiv.net/img04/img/syounen_no_uta/). These are common
  in artist entries.

Match URLs strictly when normalizing for artist entries.

Only normalize Pixiv URLs that strictly match a known format. Pass any
unrecognized URLs through without attempting to normalize them, just to
be safe.

Normalize URLs when saving artist entries.
2014-12-03 13:16:05 -08:00
evazion
7f3b98969f Refactor normalize_for_artist_finder!
Refactors things such that Sources::Site has a normalize_for_artist_finder!
method that delegates to the strategy for the appropriate site. This way
any site that needs to normalize URLs for the artist finder can do so.
2014-10-05 14:11:31 -05:00
Toks
3230ab8781 Add warning when Pixiv post is a gallery of multiple images 2014-06-13 16:33:38 -04:00
小太
cba839ba76 Kill trailing whitespace in ruby files 2013-03-19 23:10:10 +11:00
albert
43177cbad8 fixes #827 2013-03-10 16:56:04 -04:00
albert
2287bc8d61 fixed pixiv sources 2011-09-30 13:27:22 -04:00
albert
19add9aa4e refactored sources 2011-09-29 12:55:15 -04:00
albert
dc31e74809 added tinami support 2011-09-26 17:30:35 -04:00
albert
284141aace refactored source code, work on pixiv integration 2011-09-26 16:47:22 -04:00
albert
a1657366c1 renamed artsiteproxy to sources 2011-09-26 12:54:42 -04:00