57bb51621e
Fix random VCR failures in Pixiv tests. Sometimes tests randomly fail because the PHPSESSID they use in their HTTP requests to Pixiv is different than the one that was originally recorded by VCR. This causes VCR to complain that the requests don't match. This is caused by the PHPSESSID being globally cached in Memcache. Depending on the order the tests run in (which is random), one set of tests can use a PHPSESSID that was recorded for a /different/ set of tests. Improve Pixiv URL matching. * Allow URLs that are missing the http:// part. These are sometimes seen in artist entries. * Ignore URLs from random Pixiv domains such as dic.pixiv.net, blog.pixiv.net, etc. These are also sometimes in artist entries. Improve normalize_for_artist_finder! URL matching. * Normalize www.pixiv.net/stacc/username URLs. * Correctly normalize URLs that are missing the illust ID part on the end (i.e. http://i2.pixiv.net/img04/img/syounen_no_uta/). These are common in artist entries. Match URLs strictly when normalizing for artist entries. Only normalize Pixiv URLs that strictly match a known format. Pass any unrecognized URLs through without attempting to normalize them, just to be safe. Normalize URLs when saving artist entries.
73 lines
1.7 KiB
Ruby
73 lines
1.7 KiB
Ruby
module Sources
|
|
module Strategies
|
|
class Base
|
|
attr_reader :url
|
|
attr_reader :artist_name, :profile_url, :image_url, :tags, :page_count
|
|
|
|
def self.url_match?(url)
|
|
false
|
|
end
|
|
|
|
def initialize(url)
|
|
@url = url
|
|
@page_count = 1
|
|
end
|
|
|
|
# No remote calls are made until this method is called.
|
|
def get
|
|
raise NotImplementedError
|
|
end
|
|
|
|
# Subclasses should return true only if the URL is in its final normalized form.
|
|
#
|
|
# Sources::Site.new("http://img.pixiv.net/img/evazion").normalized_for_artist_finder?
|
|
# => true
|
|
# Sources::Site.new("http://i2.pixiv.net/img18/img/evazion/14901720_m.png").normalized_for_artist_finder?
|
|
# => false
|
|
def normalized_for_artist_finder?
|
|
false
|
|
end
|
|
|
|
# Subclasses should return true only if the URL is a valid URL that could
|
|
# be converted into normalized form.
|
|
#
|
|
# Sources::Site.new("http://www.pixiv.net/member_illust.php?mode=medium&illust_id=18557054").normalizable_for_artist_finder?
|
|
# => true
|
|
# Sources::Site.new("http://dic.pixiv.net/a/THUNDERproject").normalizable_for_artist_finder?
|
|
# => false
|
|
def normalizable_for_artist_finder?
|
|
false
|
|
end
|
|
|
|
def normalize_for_artist_finder!
|
|
url
|
|
end
|
|
|
|
def site_name
|
|
raise NotImplementedError
|
|
end
|
|
|
|
def unique_id
|
|
artist_name
|
|
end
|
|
|
|
def artist_record
|
|
if artist_name.present?
|
|
Artist.other_names_match(artist_name)
|
|
else
|
|
nil
|
|
end
|
|
end
|
|
|
|
def referer_url(template)
|
|
template.params[:ref] || template.params[:url]
|
|
end
|
|
|
|
protected
|
|
def agent
|
|
raise NotImplementedError
|
|
end
|
|
end
|
|
end
|
|
end
|