evazion
3aa5cab2aa
sources: refactor normalize_for_source.
...
`normalize_for_source` was used to convert image URLs to page URLs when displaying sources
on the post show page. Move all the code for converting image URLs to page URLs from
`Sources::Strategies#normalize_for_source` to `Source::URL#page_url`.
Before we had to be very careful in source strategies not to make any network calls in
`normalize_for_source`, since it was used in the view for the post show page. Now all the
code for generating page URLs is isolated in Source::URL, which makes source strategies
simpler. It also makes it easier to check if a source is an image URL or page URL, and if
the image URL is convertible to a page URL, which will make autotagging bad_link or
bad_source feasible.
Finally, this fixes it to generate better page URLs in a handful of cases:
* https://www.artstation.com/artwork/qPVGP instead of https://anubis1982918.artstation.com/projects/qPVGP
* https://yande.re/post/show?md5=b4b1d11facd1700544554e4805d47bb6s instead of https://yande.re/post?tags=md5:b4b1d11facd1700544554e4805d47bb6
* http://gallery.minitokyo.net/view/365677 instead of http://gallery.minitokyo.net/download/365677
* https://valkyriecrusade.fandom.com/wiki/File:Crimson_Hatsune_H.png instead of https://valkyriecrusade.wikia.com/wiki/File:Crimson_Hatsune_H.png
* https://rule34.paheal.net/post/view/852405 instead of https://rule34.paheal.net/post/list/md5:854806addcd3b1246424e7cea49afe31/1
2022-03-23 01:34:04 -05:00
evazion
7394660ba9
posts: fix exception when post has source like ' https://www.twitter.com/username '.
...
`twitter.com` sources worked but `www.twitter.com` didn't.
Also match the URL by class instead of by site name to ensure we match
the expected class.
2022-03-20 21:08:05 -05:00
evazion
28971fe103
sources: factor out site_name method.
2022-03-11 23:20:53 -06:00
evazion
4701027f45
sources: remove unused methods from base strategy.
...
Remove unused `urls`, `parsed_urls`, and `domains` methods.
2022-03-10 23:11:00 -06:00
nonamethanks
ff6bfff311
Lofter: simplify tag extraction logic
...
Now that we have a separate parsing class we can just use it to properly
parse tag urls as well.
2022-03-08 17:01:50 +01:00
evazion
86d8e2d13d
sources: factor out Source::URL::Lofter.
2022-02-25 23:43:10 -06:00
evazion
27d71f2727
uploads: raise download timeout.
...
Raise the timeout for downloading files from the source to 60 seconds globally.
Previously had a lower timeout because uploads were processed in the
foreground when not using the bookmarklet, and we didn't want to tie up
Puma worker processes with slow downloads. Now that all uploads are
processed in the background, we can have a higher timeout.
2022-02-15 00:56:51 -06:00
nonamethanks
1c9014a5bb
Fix lofter not working with iqdb
2022-02-05 09:43:17 +01:00
evazion
a7dc05ce63
Enable frozen string literals.
...
Make all string literals immutable by default.
2021-12-14 21:33:27 -06:00
nonamethanks
5946544f71
Lofter: Add support for additional theme
2021-10-30 17:22:45 +02:00
nonamethanks
45313c56a6
Lofter: fix tag extraction
2021-10-04 14:21:07 +02:00
nonamethanks
9a6a6e52ea
Lofter: raise timeout for file download
2021-09-10 13:10:29 +02:00
nonamethanks
f60fce614b
Fix lofter strategy due to changes in their image urls
2021-08-15 02:16:57 -05:00
nonamethanks
bb4bdefc23
Lofter: add support for another theme
2021-08-15 02:16:56 -05:00
nonamethanks
ac071b147f
Lofter: fix strategy for additional type of html theme
2021-05-23 18:05:16 +02:00
nonamethanks
bc4fd16c2c
Add Lofter support
2021-05-19 16:40:33 +02:00
