Commit Graph

58 Commits

Author SHA1 Message Date
evazion
8cadef2dd7 pixiv: fix illust id parsing (fix #4043).
* Tighten up illust id parsing to avoid misparsing ids from
  non-illust urls (sketch urls and novel urls).

* Move id parsing tests from post_test.rb to sources/pixiv_test.rb.

* Drop support for touch.pixiv.net urls. These urls are no longer used
  by Pixiv and aren't present as the source of any posts on Danbooru.
2019-01-13 14:28:51 -06:00
evazion
04d5b16da7 pixiv: fix failure to upload bad pixiv id images (fix #4031)
Bug: Uploading bad pixiv id images failed because the pixiv strategy
raised a BadIDError exception when the upload service checked for the
ugoira frame data.
2019-01-03 18:01:20 -06:00
evazion
2129e60b2b pixiv: include stacc url in new artist entries (#4028). 2018-12-27 15:03:11 -06:00
evazion
8f6c710c6b tests: fix translated tags test failures. 2018-11-12 18:04:07 -06:00
evazion
fbd5f6b7f2 pixiv: fix preview_urls for ugoiras (#3891). 2018-09-12 00:43:10 -05:00
evazion
37fc215d75 pixiv: fix preview_urls to use correct url (#3891). 2018-09-11 23:55:46 -05:00
Albert Yi
a5df178bcc Merge pull request #3886 from r888888888/source-api-caching
cache api clients
2018-09-11 17:34:25 -07:00
Albert Yi
4972c998f8 rely on preview urls if available for gallery 2018-09-11 15:06:12 -07:00
Albert Yi
266c7c0d5b cache api clients 2018-09-11 14:19:17 -07:00
evazion
736c22c3ce pixiv: fix fetch source data failing on fanbox images.
ref: https://danbooru.donmai.us/forum_posts/150000
2018-08-31 00:21:03 -05:00
Albert Yi
762dc3da24 Refactor sources 2018-08-24 12:10:51 -07:00
Albert Yi
5ae37597cd fixes #3728 2018-05-25 13:24:49 -07:00
Albert Yi
f3364b9892 skip earlier on failed pixiv tests 2018-05-15 16:15:59 -07:00
Albert Yi
8d5db3723b skip tests that throw an net::opentimeout error 2018-05-14 10:45:46 -07:00
Albert Yi
6839249e9d add logic for persisting pixiv sessions in tests 2018-05-10 13:59:57 -07:00
Albert Yi
5bca31bad1 add retry monkey patch for mechanize 2018-05-09 16:45:03 -07:00
Albert Yi
e23814be92 fix pixiv agent shutdown 2018-05-09 16:13:47 -07:00
Albert Yi
d4a2521eec reset connection for pixiv tests 2018-05-09 14:59:18 -07:00
Albert Yi
59dde72924 fix pixiv tests, install ffmpeg 2018-05-09 14:11:34 -07:00
evazion
302994e5d9 Fix #3639: Favorite count pixiv tags aren't skipped by translated tags. 2018-04-13 22:39:52 -05:00
r888888888
abce4d2551 Raise error on unpermitted params.
Fail loudly if we forget to whitelist a param instead of silently
ignoring it.

misc models: convert to strong params.

artist commentaries: convert to strong params.

* Disallow changing or setting post_id to a nonexistent post.

artists: convert to strong params.

* Disallow setting `is_banned` in create/update actions. Changing it
  this way instead of with the ban/unban actions would leave the artist in
  a partially banned state.

bans: convert to strong params.

* Disallow changing the user_id after the ban has been created.

comments: convert to strong params.

favorite groups: convert to strong params.

news updates: convert to strong params.

post appeals: convert to strong params.

post flags: convert to strong params.

* Disallow users from setting the `is_deleted` / `is_resolved` flags.

ip bans: convert to strong params.

user feedbacks: convert to strong params.

* Disallow users from setting `disable_dmail_notification` when creating feedbacks.
* Disallow changing the user_id after the feedback has been created.

notes: convert to strong params.

wiki pages: convert to strong params.

* Also fix non-Builders being able to delete wiki pages.

saved searches: convert to strong params.

pools: convert to strong params.

* Disallow setting `post_count` or `is_deleted` in create/update actions.

janitor trials: convert to strong params.

post disapprovals: convert to strong params.

* Factor out quick-mod bar to shared partial.
* Fix quick-mod bar to use `Post#is_approvable?` to determine visibility
  of Approve button.

dmail filters: convert to strong params.

password resets: convert to strong params.

user name change requests: convert to strong params.

posts: convert to strong params.

users: convert to strong params.

* Disallow setting password_hash, last_logged_in_at, last_forum_read_at,
  has_mail, and dmail_filter_attributes[user_id].

* Remove initialize_default_image_size (dead code).

uploads: convert to strong params.

* Remove `initialize_status` because status already defaults to pending
  in the database.

tag aliases/implications: convert to strong params.

tags: convert to strong params.

forum posts: convert to strong params.

* Disallow changing the topic_id after creating the post.
* Disallow setting is_deleted (destroy/undelete actions should be used instead).
* Remove is_sticky / is_locked (nonexistent attributes).

forum topics: convert to strong params.

* merges https://github.com/evazion/danbooru/tree/wip-rails-5.1
* lock pg gem to 0.21 (1.0.0 is incompatible with rails 5.1.4)
* switch to factorybot and change all references

Co-authored-by: r888888888 <r888888888@gmail.com>
Co-authored-by: evazion <noizave@gmail.com>

add diffs
2018-04-06 18:09:57 -07:00
evazion
265377bdbb Fix #3450: Aliased tags show up under translated tags.
Resolves aliases in translated tags. For example, say we lookup `遠坂凛`
and find `tohsaka_rin` and `toosaka_rin`. We apply aliases so that
`tohsaka_rin` becomes `toosaka_rin`, which is then returned as the only
translated tag.
2017-12-23 12:27:58 -06:00
evazion
0b5ed163fe Fix #3395: fix pixiv API exception when uploading bad pixiv id images.
Fixes an exception when attempting to upload a Pixiv image from a
deleted work. The download strategy tries to fetch the source data in
the course of rewriting the URL, which fails if the work has been
deleted from Pixiv.

Raise a BadIDError and leave the URL as-is (don't rewrite it).
2017-11-21 14:40:39 -06:00
evazion
f8581c8113 Fix #3321: HTML tags not converted in artist commentaries. 2017-11-21 02:42:49 -06:00
r888888888
9d5e4f969f fix source tests 2017-11-20 12:30:29 -08:00
r888888888
d124ff8128 fix case when fetching pixiv sources when illust_id param comes before mode 2017-08-23 17:50:33 -07:00
evazion
c270954991 Fix #3258: Fetch source data broken for https://i.pximg.net/* ugoiras. 2017-08-06 17:23:29 -05:00
evazion
7ae860a8b8 Drop Pixiv whitecube support (#2681, #3206). 2017-07-26 18:06:46 -05:00
evazion
22980c845a tests: fix pixiv ugoira urls. 2017-07-25 00:36:18 -05:00
evazion
cc8986641b Fix #3208: Fix translated tag suggestions for Pixiv.
* Only suggest the Danbooru tag with the same name if there is no
  matching wiki other name. Example: if we have the Pixiv tag `Fate` and
  the Danbooru tag `fate_(series)` with other name `fate`, suggest that,
  not the Danbooru tag `fate`.

* Don't suggest tags that are empty or whose wiki is deleted.

* Only split tags on "/" if there are no other matches, and only for Pixiv.

* For Pixiv, only include traditional media tags in tag list, not digital media (Photoshop, SAI).

* Add some tests.
2017-07-09 12:10:30 -05:00
evazion
94e48aa6f3 pixiv: convert commentary to dtext.
Convert pixiv links to dtext in this manner:

* https://www.pixiv.net/member_illust.php?mode=medium&illust_id=1234
  => pixiv #1234 "»":[/posts?tags=pixiv:1234]

* https://www.pixiv.net/member.php?id=1234
  => "user/1234":[https://www.pixiv.net/member.php?id=1234] "»":[/artists?search[url_matches]=https://www.pixiv.net/member.php?id=1234]
2017-06-16 18:42:37 -05:00
evazion
fed23c0baa fetch source data: fix gallery warning for non-pixiv galleries. 2017-06-06 23:03:07 -05:00
r888888888
bffa1f3dc3 fix unit tests 2017-05-30 17:37:42 -07:00
evazion
757a7da49b fixup! /source.json: don't encode '&' as '&amp;' in commentaries. 2017-05-10 21:03:29 -05:00
evazion
b68cb174f5 tests: fix pixiv tests to use https://i.pixiv.net. 2017-04-15 23:14:41 -05:00
r888888888
0b8d4105aa fix tests 2017-04-04 12:39:17 -07:00
Albert Yi
0ea7d78584 remove usage of vcr cassettes; delete unused fixtures; fix some broken unit tests 2016-12-28 15:47:28 -08:00
Albert Yi
a862c5bf03 fixes #2746: Uploading from touch.pixiv.net broken 2016-10-31 12:22:32 -07:00
r888888888
fc7afd44ea refactor source pixiv test
refactor pixiv download tests
refactor upload test
refactor nico seiga test
refactor twitter tests
2016-09-28 11:25:29 -07:00
r888888888
4fa2741ed2 include whitecube source/rewrite strategies 2016-09-28 11:24:54 -07:00
r888888888
58aa5c6d66 fix tests 2016-05-28 14:08:44 -07:00
r888888888
66dd4f072e update tests 2015-06-02 19:20:48 -07:00
r888888888
2ddc93723c fixes #2327 2015-06-02 19:20:09 -07:00
r888888888
9bdc0116bf fix pixiv tests 2015-05-14 16:50:24 -07:00
r888888888
2d266fc0b6 uploads/batch now supports pixiv 2015-03-15 13:56:16 -07:00
r888888888
16f9a61d63 fixes #2299 2014-10-29 15:14:17 -07:00
r888888888
8d4c9d7955 fix pixiv tests 2014-10-22 17:22:36 -07:00
r888888888
3bb06c2be4 integrate ugoiras into zip+webm+preview 2014-10-19 02:30:02 -07:00
evazion
339e88584d Rewrite tests for downloading new Pixiv URLs.
* Test that downloading the HTML page downloads the full size image
  instead.
* Test that downloading a small or medium size image downloads the
  full size instead.
* Test the above for both single illustrations and for manga galleries,
  for new and old posts (i.e. uploaded before the Pixiv URL changes versus
  after), and for different file extensions.
* Test trying to download ugoira zip files. These tests are expected to
  fail because we don't support ugoira yet.
* Also add some more tests for fetching source data.
2014-10-04 12:45:37 -05:00
Toks
76d1f0a66b Update seiga and pixiv source tests, and seiga vcr cassettes
fixes #2142
2014-04-30 15:26:58 -04:00