Commit Graph

491 Commits

Author SHA1 Message Date
evazion
07e23204b6 rubocop: fix various Rubocop warnings. 2021-06-17 04:17:53 -05:00
evazion
4003ddd788 Merge pull request #4808 from nonamethanks/fix-weibo
Weibo: fix new lengthier image urls breaking the batch bookmarklet
2021-06-02 15:17:54 -05:00
nonamethanks
ac4b4a151c Weibo: fix new lengthier image urls breaking the batch bookmarklet 2021-05-31 00:36:53 +02:00
nonamethanks
1234d93292 Nijie: get correct image when using bookmarklet 2021-05-25 12:20:39 +02:00
nonamethanks
ac071b147f Lofter: fix strategy for additional type of html theme 2021-05-23 18:05:16 +02:00
nonamethanks
bc4fd16c2c Add Lofter support 2021-05-19 16:40:33 +02:00
evazion
12eacbe76f Merge pull request #4800 from nonamethanks/fix-skeb
Skeb: add support for multi-image posts and fix tests
2021-05-17 12:16:21 -05:00
nonamethanks
1bc266a3db Skeb: add support for multi-image posts and fix tests 2021-05-16 12:50:39 +02:00
evazion
ba5a2dcd79 Merge pull request #4794 from nonamethanks/fix-skeb
Skeb: fix broken strategy
2021-05-05 05:08:52 -05:00
nonamethanks
e95731d58f Skeb: fix broken strategy 2021-05-04 21:18:12 +02:00
nonamethanks
cb6196c259 Nicoseiga: auto-add spoiler tags to commentary 2021-04-06 14:08:49 +02:00
evazion
b8f7c3795c Merge pull request #4767 from nonamethanks/pixiv-other-names
Pixiv: don't add auto-generated usernames to the other names field
2021-03-18 21:29:21 -05:00
nonamethanks
073f63cfa7 Pixiv: don't add auto-generated usernames to the other names field 2021-03-16 02:44:49 +01:00
nonamethanks
a123bf0ee8 Mastodon: fix strategy not working for direct or dead links 2021-03-09 22:35:36 +01:00
evazion
0249c290fd skeb: remove skeb from site_name in base strategy.
Fixup a mistake with the way the merge conflict was resolved in 9dd903d21.
2021-03-08 03:56:44 -06:00
evazion
9dd903d212 Merge branch 'master' into skeb 2021-03-08 03:43:15 -06:00
evazion
66af4986c6 Merge pull request #4754 from nonamethanks/lazymap
Optimize Sources::Strategies.find()
2021-03-08 03:40:28 -06:00
evazion
64440757c1 Merge pull request #4737 from nonamethanks/fix-mastodon-page-url
Sources: get correct mastodon page url
2021-03-08 01:49:56 -06:00
evazion
224d4932bf Merge pull request #4745 from nonamethanks/twitter-common-regexes
Twitter: update common hashtag regexes
2021-03-08 01:49:30 -06:00
evazion
e17cefe3ab Merge pull request #4753 from nonamethanks/fix-baraag-regex
Baraag: fix image regex
2021-03-08 01:49:15 -06:00
evazion
1716cc5bf9 artists: add more artist url icons. 2021-03-08 01:30:02 -06:00
evazion
869a99d9a3 nijie: clear session cookie if it's expired (#4665).
If we detect that the session cookie has expired (by the presence of the
`#login_illust` element on the page), then clear the cached session
cookie. The current source fetch will still fail, but the next fetch
will try to login again and hopefully succeed.
2021-03-08 01:30:02 -06:00
evazion
5e4865701c nijie: reduce login cookie cache duration to 60 minutes (#4665). 2021-03-08 01:30:02 -06:00
nonamethanks
a4ae07416a Optimize Sources::Strategies.find()
Use lazy.map instead of map, so that we do not run every single
strategy's match method for every source
2021-03-07 18:20:10 +01:00
nonamethanks
4118a3f3c9 Baraag: fix image regex
Fixes a bug for Baraag.net that caused the batch bookmarklet to always pick the first picture in multi-image posts.
2021-03-06 08:50:08 +01:00
nonamethanks
dc97f4483b Add skeb support 2021-03-04 21:53:27 +01:00
nonamethanks
e75489aa94 Twitter: update common hashtag regexes 2021-03-02 17:05:47 +01:00
nonamethanks
ba826ff6fa Sources: get correct mastodon page url 2021-02-26 19:45:01 +01:00
evazion
7b60a476e5 sources: add artist profile links to fetch source data box.
Add site icons linking to all the artist's sites in the fetch source
data box.

Some artist entries have a large number of URLs. Various heuristics are
applied to try to present the most useful URLs first. Dead URLs and
redundant URLs (Pixiv stacc and Twitter intent URLs) are filtered out.
Remaining URLs are sorted first by site (to put sites like Pixiv and
Twitter first), then by URL (to break ties when an artist has multiple
accounts on the same site).

Some sites have shitty hard-to-read icons. It can't be helped. The icons
are the official favicons of each site.
2021-02-26 01:24:30 -06:00
evazion
8b8a3f3836 Merge pull request #4723 from nonamethanks/pawoo2mastodon
Convert pawoo strategy into mastodon; add baraag.net support
2021-02-16 18:22:20 -06:00
evazion
2fccf203bd Merge pull request #4719 from nonamethanks/fanbox-r18
Fanbox: fix tests and a bug with image ordering in the bookmarklet
2021-02-16 18:18:04 -06:00
nonamethanks
ffe8f8c92d Add baraag.net support 2021-02-15 13:51:52 +01:00
nonamethanks
5366b0781e Convert pawoo strategy to mastodon 2021-02-15 13:51:49 +01:00
evazion
23a06aff1d Fix #4720: Pixiv commentary links all create invalid urls.
Regression caused by the switch from the mobile API to the Ajax API. In
the Ajax API, commentaries have /jump.php?<url> links that we have to strip out.
2021-02-13 17:41:01 -06:00
nonamethanks
482c1e4c7c Fanbox: fix tests and image ordering
* fix the test for R18 posts, which are now supported
* replace the dead post for the imageMap test with an active one
* fix the ordering of images: it was possible to get the wrong image in
some cases when not using the batch bookmarklet, because of how fanbox
orders imageMap posts
* fix a bug where the sample wasn't properly replaced with the full
size
* rename test file to match the rest of the other files
2021-02-10 16:53:22 +01:00
evazion
39cc3ed5cf pixiv: fix API breakage.
Fix the Pixiv API no longer working by rewriting the Pixiv strategy to
use the Ajax API instead of the mobile API.

Before we could authenticate in the mobile API by using the OAuth 2.0
grant_type=password authentication flow. This no longer works. Now it
requires logging in through a HTML page, which is protected by Google
reCaptcha. This makes using the mobile API infeasible.

Instead we switch to the Ajax API, which only needs a PHPSESSID to
authenticate. This can be obtained by logging in manually and using the
devtools to extract the cookie.

This also temporarily removes support for Pixiv novels. This should be
moved to a separate source strategy.
2021-02-09 06:18:36 -06:00
evazion
7520c4db49 uploads: temp disable Pixiv source strategy.
Pixiv API client is currently broken. Temporarily disable Pixiv source
strategy so direct image Pixiv uploads can still go through. The
posts.pixiv_id field will need to be backfilled later.
2021-02-08 23:48:26 -06:00
BrokenEagle
c90ef9f1b0 Add the Twitter name to the list of other names for new artists
The artist name is supposed to be the display name according to the
base file, however the artist name was treated like the tag name
instead. This commit renames all instances of "artist_name" to
"tag_name" and then adds an "artist_name" function that uses the
Twitter display name if available.
2021-01-19 00:46:21 +00:00
evazion
3542c401d4 Merge pull request #4654 from nonamethanks/fix_gelbooru_normalization
Fix gelbooru source normalization
2021-01-04 01:31:17 -06:00
evazion
5612413546 Merge pull request #4641 from nonamethanks/weibo_previews
Weibo: use proxy for upload previews
2021-01-04 01:20:50 -06:00
nonamethanks
b3f92dd2c7 Fix gelbooru source normalization 2021-01-03 20:12:40 +01:00
evazion
dbb66ace90 routes: replace hardcoded routes in models with route helpers.
Add a Routes module that gives models access to route helpers outside of
views, and use it to replace various hardcoded routes.
2020-12-24 00:17:19 -06:00
nonamethanks
2423c8a447 Weibo: use proxy for upload previews 2020-12-21 15:28:28 +01:00
evazion
1d2a8a7898 fanbox: don't raise error on age-restricted posts.
Prevent age-restricted fanbox posts from raising errors when source data
is fetched. This prevents error messages from being shown to users when
switching to the edit tab on a post.

This will cause uploads of age-restricted posts to fail with an
unrelated error because we either can't find the image url (if we were
given only the html page) or we can't download the image (because we're
not logged in to Fanbox).
2020-08-18 15:34:12 -05:00
evazion
86c376e90d Merge pull request #4590 from nonamethanks/nijie_doujinshi
Nijie: Add doujin support
2020-08-18 14:59:18 -05:00
evazion
0c96c2172c Merge pull request #4577 from nonamethanks/fanbox
Add (partial) fanbox support
2020-08-18 14:13:43 -05:00
evazion
4b18361aaf tumblr: fix uses of $ in regexes. 2020-08-17 15:31:18 -05:00
evazion
160b099445 sources: use http client from base strategy. 2020-08-17 15:30:01 -05:00
evazion
d9f0a1c5e3 Merge pull request #4585 from nonamethanks/fix_tumblr
Tumblr: support highest res for new image urls
2020-08-17 14:11:48 -05:00
nonamethanks
e3fd3f42cc Nijie: Add doujin support 2020-08-15 09:29:29 +02:00