Fix Nicoseiga strategy to work with certain direct image urls that we
can't otherwise extract any information from.
Examples:
* https://dic.nicovideo.jp/oekaki/52833.png
* Get rid of mechanize, fully switch to Danbooru::Http
* Switch to mobile api, improving speed
* Merge main and manga clients
* Add full support for manga pages
* Add support for anonymous and r-15 images
* Don't fail when attempting to upload oekaki direct links
* Various misc fixes
Get rid of `normalized_for_artist_finder?` and `normalizable_for_artist_finder?`.
This was legacy bullshit that was originally designed to avoid API calls
when saving artist entries containing old Pixiv direct image urls that
had already been normalized, or that couldn't be normalized because they
were bad id.
Nowadays we store profile urls in artist entries instead of direct image
urls, so we don't normally need to do any API calls to normalize the
profile url. Strategies should take care to avoid triggering API calls
inside `profile_url` when possible.
* Move the source normalization logic out of the post model
and into individual sources' strategies.
* Rewrite normalization tests to be handled into each source's test,
and expand them significantly. Previously we were only testing
a very small subset of domains and variants.
* Fix up normalization for several sites.
* Normalize fav.me urls into normal deviantart urls.
* Rename `unique_id` to `tag_name`.
* Add `other_names` and `profile_urls` methods that sources can override
to provide extra names or urls when creating new artist entries.
Fix sources choosing the wrong strategy when the referer belongs to a
different site (for example, when uploading a twitter post with a pixiv
referer).
* Fix `match?` to only consider the main url, not the referer.
* Change `match?` to match against a list of domains given by the `domains` method.
* Change `match?` to an instance method.
Works around connection reset errors in the test suite by disabling
persistent connections.
20) Error:
Sources::PixivTest#test_: in all cases fetching source data for a new manga image should get the tags. :
Net::HTTP::Persistent::Error: too many connection resets (due to closed stream - IOError) after 0 requests on 47071328584700, last used 1.842702476 seconds ago
app/logical/pixiv_web_agent.rb:46:in `build'
app/logical/sources/strategies/pixiv.rb:104:in `agent'
app/logical/sources/strategies/pixiv.rb:72:in `get'
app/logical/sources/site.rb:6:in `get'
test/unit/sources/pixiv_test.rb:7:in `get_source'
test/unit/sources/pixiv_test.rb:64:in `block (3 levels) in <class:PixivTest>'
ref: github.com/sparklemotion/mechanize/issues/123
ref: http://www.rubydoc.info/gems/mechanize/Mechanize#retry_change_requests%3D-instance_method
e.g. If you were on an html work page on pixiv, clicked a link to a
different html work page on pixiv, and then clicked the bookmarklet,
then it used to fetch the source from the FIRST work you were on instead
of the second.
* Support rewriting source when user uploads from a thumbnail url or
html page url
* Fix bug where site did not log in correctly
* Fix bug where the image url couldn't be extracted from the page if the
image was rated as adults only on seiga
* Normalize direct image url to html page url so tags, etc., can be
extracted
