* Prevent non-ugoira posts from getting dummy frame data records with
all the columns set to null.
* Make `data` and `content_type` columns NOT NULL, since they should
always be present.
* Remove the timestamps while we're at it, since they don't seem
necessary for anything.
* Change the id of the canvas tag to 'image'. This is what the notes JS
expects. It's also what the shift+e edit shortcut depends on.
* Add data-original-width/height attributes, which are what the notes JS
needs to position notes properly.
* Add all the other data attributes that regular images have, because
why not.
This refactors things such that
* Fetching the ugoira frame data happens at the end of rewrite, not
rewrite_html_pages. This way uploading zip URLs should work.
* The source data is cached in an instance variable in case it
was indeed fetched during rewrite_html_pages. This way it doesn't get
fetched a second time.
1) Put `<script>alert("xss 1")</script>` in the Other Names field in an
artist entry.
2) Put `<script>alert("xss 2")</script>` in the URLs field.
3) Trick someone into the viewing the history page for that artist.
This vulnerability allows someone to create a source link that appears
to lead to Pixiv like normal, but that actually executes Javascript code
when clicked.
1) Set the source of a post to javascript:"http://img1.pixiv.net/img/xss/";alert("xss");
2) Wait for someone to click the source link in the Information sidebar.
3) Profit.
The cause is that the regexes for detecting Pixiv URLs aren't anchored
to the front of the string using \A. This allows non-http:// links to be
created.
1) Set a pool name to '<script>alert("xss")</script>'.
2) Wait for people to view it in /pools/gallery.
At first glance the fact that the pool name is truncated to 80 chars
appears to limit how much can be done in the exploit. This poses no
problem though, since someone could inject '<script src="http://www.evil.com/evil.js"></script>' instead.
Move the call to normalize_for_artist_finder! from the artist
controller into Artist#find_all_by_url. This makes testing easier.
It also makes it so that URLs are normalized when using the search
form on the artist listing page.
Refactors things such that Sources::Site has a normalize_for_artist_finder!
method that delegates to the strategy for the appropriate site. This way
any site that needs to normalize URLs for the artist finder can do so.
* Add tests for finding artists using the new Pixiv URLs in Artist#find_all_by_url.
* Add tests for the artist finder JSON API in ArtistsController#finder.
* Add tests for the artist page search form in ArtistsController#index.
* Test that downloading the HTML page downloads the full size image
instead.
* Test that downloading a small or medium size image downloads the
full size instead.
* Test the above for both single illustrations and for manga galleries,
for new and old posts (i.e. uploaded before the Pixiv URL changes versus
after), and for different file extensions.
* Test trying to download ugoira zip files. These tests are expected to
fail because we don't support ugoira yet.
* Also add some more tests for fetching source data.
There are two kinds of thumbnails that need to be rewritten. First case:
new /img-master/ URLs need to be rewritten to /img-original/ URLs like this:
http://i2.pixiv.net/c/600x600/img-master/img/2014/10/04/03/59/52/46337015_p0_master1200.jpg
=> http://i2.pixiv.net/img-original/img/2014/10/04/03/59/52/46337015_p0.png
This is what `rewrite_new_medium_images` does. In order to do this, it
has to use the Pixiv API to get the correct file extension.
Second case: Old small/medium size URLs need to be rewritten to full
size URLs like this:
http://i2.pixiv.net/img18/img/evazion/14901720_m.png
=> http://i2.pixiv.net/img18/img/evazion/14901720.png
But when the medium size URL is actually for a manga image, it needs to be
rewritten to the big manga URL instead:
http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_m.jpg
=> http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_big_p0.jpg
But we can't tell whether it's a manga image from the URL, so we have to
use the manga page count from either the HTML page or the API to
determine whether it's part of a manga gallery.
So in order to make this work, `rewrite_old_small_and_medium_images`
takes an `is_manga` flag. `Sources::Strategies::Pixiv#get` gets the
page count from the HTML and passes the `is_manga` flag on down through
the call chain until `rewrite_old_small_and_medium_images` gets it.
When `rewrite_old_small_and_medium_images` is called from
`Downloads::Strategies::Pixiv#rewrite_thumbnails`, the `is_manga` flag
isn't passed in because we didn't scrape the HTML. This causes
`rewrite_old_small_and_medium_images` to look it up in the API instead.
rmagick-2.13.2 is incompatible with versions of ImageMagick built with
--enable-hdri. This includes the version of ImageMagick supplied
with Arch Linux. This prevents rmagick, and consequently Danbooru,
from being installed on Arch Linux.
rmagick-2.13.3 fixed this issue. See https://github.com/rmagick/rmagick/issues/18.
evazion
