Commit Graph

270 Commits

Author SHA1 Message Date
evazion
716cdb0126 tests: fix broken tests.
* Fix a broken Twitter profile image upload test.
* Skip a broken DeviantArt flash file upload test (flash no longer
  supported by DeviantArt?)
* Skip user upgrade tests when Stripe is not configured.
2021-01-27 00:40:53 -06:00
evazion
65be2c99b0 Fix #4657: Hentai-Foundry: Document tree depth limit exceeded. 2021-01-06 03:05:36 -06:00
evazion
873d719481 tests: fix broken Newgrounds test.
Broken again by the source post being deleted.
2021-01-04 23:54:18 -06:00
evazion
3542c401d4 Merge pull request #4654 from nonamethanks/fix_gelbooru_normalization
Fix gelbooru source normalization
2021-01-04 01:31:17 -06:00
nonamethanks
b3f92dd2c7 Fix gelbooru source normalization 2021-01-03 20:12:40 +01:00
evazion
dbb66ace90 routes: replace hardcoded routes in models with route helpers.
Add a Routes module that gives models access to route helpers outside of
views, and use it to replace various hardcoded routes.
2020-12-24 00:17:19 -06:00
evazion
c8a9015e8e Merge pull request #4611 from aaronfranke/formatting
Make file formatting comply with POSIX standards and remove trailing space characters
2020-12-05 12:45:38 -06:00
evazion
cc64f8b7ee tests: fix broken source tests.
Fix various tests broken by source files changing or being deleted.
2020-11-10 14:52:54 -06:00
Aaron Franke
6cdafdf136 Remove trailing space characters
Trailing space characters do nothing except take up disk space, they should be removed
2020-10-04 05:15:02 -04:00
Aaron Franke
191b528ad7 Ensure files end in newlines (POSIX compliance) 2020-10-04 05:13:39 -04:00
evazion
1d2a8a7898 fanbox: don't raise error on age-restricted posts.
Prevent age-restricted fanbox posts from raising errors when source data
is fetched. This prevents error messages from being shown to users when
switching to the edit tab on a post.

This will cause uploads of age-restricted posts to fail with an
unrelated error because we either can't find the image url (if we were
given only the html page) or we can't download the image (because we're
not logged in to Fanbox).
2020-08-18 15:34:12 -05:00
evazion
86c376e90d Merge pull request #4590 from nonamethanks/nijie_doujinshi
Nijie: Add doujin support
2020-08-18 14:59:18 -05:00
evazion
0c96c2172c Merge pull request #4577 from nonamethanks/fanbox
Add (partial) fanbox support
2020-08-18 14:13:43 -05:00
evazion
d9f0a1c5e3 Merge pull request #4585 from nonamethanks/fix_tumblr
Tumblr: support highest res for new image urls
2020-08-17 14:11:48 -05:00
nonamethanks
e3fd3f42cc Nijie: Add doujin support 2020-08-15 09:29:29 +02:00
nonamethanks
5615719218 Tumblr: support highest res for new image urls 2020-08-14 01:23:27 +02:00
evazion
ff83cbeba9 tests: fix broken nijie test.
This post apparently had a tag added to it.
2020-08-12 11:01:48 -05:00
evazion
a70c4a72e6 Merge pull request #4570 from nonamethanks/fix_4543
Nijie: fix login
2020-08-12 09:46:09 -05:00
nonamethanks
9a7a1e20ca Add fanbox support 2020-08-09 00:21:57 +02:00
evazion
e31afd0827 tests: fix broken tests. 2020-08-05 12:41:48 -05:00
nonamethanks
76b0380927 Nijie: fix login
Fixes a regression in 6e6ce6e62f
2020-07-31 23:26:35 +02:00
evazion
4074cc99f9 uploads: fix incorrect remote sizes on pixiv uploads.
Bug: the uploads page showed a remote size of 146 bytes for Pixiv uploads.

Cause: we didn't spoof the Referer header when making the HEAD request
for the image, causing Pixiv to return a 403 error.

Also fix the case where the Content-Length header is absent.
2020-06-24 03:02:45 -05:00
evazion
8eac82a971 pixiv: fix regression with new user profile urls.
* Update tests to use new Pixiv profile urls.
* Fix issue with artist finder not working when given direct image or
  html page urls.
2020-06-24 02:41:11 -05:00
evazion
85f58bf2f6 newgrounds: fix style nitpicks. 2020-06-24 00:25:45 -05:00
evazion
42c06b0f1e Merge pull request #4516 from nonamethanks/add_newgrounds_support
Add NewGrounds support
2020-06-24 00:20:16 -05:00
evazion
185693b99b Merge branch 'master' into fix-pixiv-profile-url 2020-06-24 00:06:55 -05:00
evazion
83a8468ee9 tests: remove unnecessary rescueing of Net::OpenTimeout errors.
These exceptions are no longer thrown now that we've switched from
HTTParty to http.rb. Swallowing unexpected exceptions during testing was
a bad practice anyway.
2020-06-23 03:12:44 -05:00
evazion
8a21c9a8db Merge pull request #4523 from nonamethanks/revert_pixiv_tools
Revert "Pixiv: don't blacklist digital tools"
2020-06-23 02:39:18 -05:00
evazion
73506bac33 twitter: add tests for uploading profile banners (#4520). 2020-06-23 02:37:21 -05:00
evazion
7e471fe223 sources: replace HTTParty with Danbooru::Http in http_exists?. 2020-06-21 15:11:56 -05:00
evazion
5604ab0079 pixiv: remove fanbox support.
This is broken and it needs to be rewritten as a separate source
strategy anyway.
2020-06-21 11:59:51 -05:00
evazion
6e6ce6e62f nijie: replace Mechanize with Danbooru::Http.
The Nijie login process works like this:

* First we submit our `email` and `password` to `https://nijie.info/login_int.php`.
* Then we save the NIJIEIEID session cookie from the response.
* We optionally retry if login failed. Nijie returns 429 errors with a
  `Retry-After: 5` header if we send too many login requests. This can
  happen during parallel testing.
* We cache the login cookies for only 1 hour so we don't have to worry
  about them becoming invalid if we cache them too long.

Cookies and retrying errors on failure are handled transparently by Danbooru::Http.
2020-06-21 05:22:57 -05:00
evazion
67a52dbc2d tumblr: support new va.media.tumblr.com urls. 2020-06-19 13:53:35 -05:00
nonamethanks
0a396c8b95 Revert "Pixiv: don't blacklist digital tools"
This reverts commit e83d07ea7b.

It was worth a try, but unfortunately it seems that once
someone sets tools in a Pixiv upload, they become defaults and
are applied to all of their subsequent uploads, so we get some
posts with two or three different digital tags.
2020-06-19 08:08:46 +02:00
nonamethanks
7a41ee9c34 Add NewGrounds support 2020-06-18 03:22:30 +02:00
BrokenEagle
158a4aa916 Fix Pixiv user profile URL to use the latest format
This will only affect new artist and commentary records going forward.
2020-06-17 07:07:33 +00:00
BrokenEagle
05f9b78ee3 Distinctly separate and label explicit/guro content in Pixiv test
This helps discern why these tests might be failing and serve as a
reminder to set the permissions for the Pixiv account correctly.
2020-06-17 07:07:33 +00:00
evazion
b583b3c810 tests: fix nicoseiga download tests. 2020-06-16 00:10:35 -05:00
evazion
962e60f4f1 Merge pull request #4496 from nonamethanks/refactor_nicoseiga
Refactor nicoseiga
2020-06-15 21:05:10 -05:00
evazion
19727ab5c4 Merge pull request #4505 from nonamethanks/pixiv_digital_tags
Pixiv: don't blacklist digital tools anymore
2020-06-15 20:56:56 -05:00
nonamethanks
9f0e85e1b5 Refactor nicoseiga strategy
* Get rid of mechanize, fully switch to Danbooru::Http
* Switch to mobile api, improving speed
* Merge main and manga clients
* Add full support for manga pages
* Add support for anonymous and r-15 images
* Don't fail when attempting to upload oekaki direct links
* Various misc fixes
2020-06-15 03:37:51 +02:00
nonamethanks
e83d07ea7b Pixiv: don't blacklist digital tools anymore 2020-06-12 04:15:20 +02:00
evazion
2d05004bef tests: don't cache pixiv sessions. 2020-06-11 00:47:12 -05:00
evazion
2ede41c4dc tests: fix twitter test broken by deleted tweet. 2020-06-10 20:22:16 -05:00
evazion
d6b266514b tests: disable known broken pixiv fanbox tests. 2020-06-10 18:21:44 -05:00
evazion
d002701bc1 Merge pull request #4494 from nonamethanks/fix_deviantart_api_downloads
Deviantart: fix api downloads
2020-06-09 01:37:03 -05:00
nonamethanks
25b801619f Deviantart: fix api downloads 2020-05-31 07:01:43 +02:00
evazion
855e31ac90 nijie: fetch commentary as html instead of plaintext.
Fix regression in #4475. Fetch the commentary as html instead of
plaintext so that we don't lose links or other formatting.

Also fix it so that /jump.php redirect links are replaced with the
actual url.
2020-05-29 15:36:21 -05:00
evazion
9ca848d732 tests: fix more ruby 2.7 deprecation warnings. 2020-05-29 15:36:21 -05:00
nonamethanks
d339947647 Weibo: add source normalization 2020-05-28 01:05:11 +02:00