jaculabilis/danbooru
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
11,318 Commits 1 Branch 0 Tags
fbca31d29eb7fd0ba8aa84b142bade6d8cf9dbc3
Commit Graph

13 Commits

Author SHA1 Message Date
evazion
3aa5cab2aa sources: refactor normalize_for_source. 
`normalize_for_source` was used to convert image URLs to page URLs when displaying sources
on the post show page. Move all the code for converting image URLs to page URLs from
`Sources::Strategies#normalize_for_source` to `Source::URL#page_url`.

Before we had to be very careful in source strategies not to make any network calls in
`normalize_for_source`, since it was used in the view for the post show page. Now all the
code for generating page URLs is isolated in Source::URL, which makes source strategies
simpler. It also makes it easier to check if a source is an image URL or page URL, and if
the image URL is convertible to a page URL, which will make autotagging bad_link or
bad_source feasible.

Finally, this fixes it to generate better page URLs in a handful of cases:

* https://www.artstation.com/artwork/qPVGP instead of https://anubis1982918.artstation.com/projects/qPVGP
* https://yande.re/post/show?md5=b4b1d11facd1700544554e4805d47bb6s instead of https://yande.re/post?tags=md5:b4b1d11facd1700544554e4805d47bb6
* http://gallery.minitokyo.net/view/365677 instead of http://gallery.minitokyo.net/download/365677
* https://valkyriecrusade.fandom.com/wiki/File:Crimson_Hatsune_H.png instead of https://valkyriecrusade.wikia.com/wiki/File:Crimson_Hatsune_H.png
* https://rule34.paheal.net/post/view/852405 instead of https://rule34.paheal.net/post/list/md5:854806addcd3b1246424e7cea49afe31/1
 2022-03-23 01:34:04 -05:00
evazion
7394660ba9 posts: fix exception when post has source like 'https://www.twitter.com/username'. 
`twitter.com` sources worked but `www.twitter.com` didn't.

Also match the URL by class instead of by site name to ensure we match
the expected class.
 2022-03-20 21:08:05 -05:00
evazion
28971fe103 sources: factor out site_name method. 2022-03-11 23:20:53 -06:00
evazion
b4aea72d04 sources: remove preview_urls method from base strategy. 
Remove the `preview_urls` method from strategies. The only place this was used was
when doing IQDB searches, to download the thumbnail image from the source instead of
the full image.

This wasn't worth it for a few reasons:

* Thumbnails on other sites are sometimes not the size we want, which could affect
  IQDB results.
* Grabbing thumbnails is complex for some sites. You can't always just rewrite the
  image URL. Sometimes it requires extra API calls, which can be slower than just
  grabbing the full image.
* For videos and animations, thumbnails from other sites don't always match our
  thumbnails. We do smart thumbnail generation to try to avoid blank thumbnails, which
  means we don't always pick the first frame, which could affect IQDB results.

API changes:

* /iqdb_queries?search[file_url] now downloads the URL as is without any modification.
  Before it tried to change thumbnail and sample size image URLs to the full version.

* /iqdb_queries?search[url] now returns an error if the URL is for a HTML page that
  contains multiple images. Before it would grab only the first image and silently
  ignore the rest.
 2022-03-11 03:22:23 -06:00
evazion
4701027f45 sources: remove unused methods from base strategy. 
Remove unused `urls`, `parsed_urls`, and `domains` methods.
 2022-03-10 23:11:00 -06:00
evazion
ffe52f5ead sources: factor out Source::URL::Foundation. 
Add support for a couple more URL types:

* https://foundation.app/@asuka111art/dinner-with-cats-82426
* https://f8n-production-collection-assets.imgix.net/0x3B3ee1931Dc30C1957379FAc9aba94D1C48a5405/128711/QmcBfbeCMSxqYB3L1owPAxFencFx3jLzCPFx6xUBxgSCkH/nft.png

Also include these URLs in the list of profile URLs:

* https://foundation.app/0x7E2ef75C0C09b2fc6BCd1C68B6D409720CcD58d2 (for https://foundation.app/@mochiiimo)

These URLs should be stable even if the user changes their name.
 2022-02-23 23:49:31 -06:00
evazion
112b323f01 foundation: fix exception when uploading new Foundation url format. 
Fix 'null value in column "source_url"' exception when uploading urls like this:

* https://foundation.app/@KILLERGF/kgfgen/4
* https://foundation.app/@mochiiimo/foundation/97376
 2022-02-22 13:29:28 -06:00
evazion
a7dc05ce63 Enable frozen string literals. 
Make all string literals immutable by default.
 2021-12-14 21:33:27 -06:00
evazion
c94fdef3b2 foundation: fix fetching artist commentary. 
The markup for the description changed from a <div> to a <h2>.
 2021-12-08 03:01:54 -06:00
nonamethanks
41f9fde2e0 Fix foundation urls not working with iqdb 2021-11-15 10:10:49 +01:00
nonamethanks
49e232f2ae Foundation: add support for unconventional account names 2021-11-09 13:35:52 +01:00
nonamethanks
6c9b49c194 Foundation: add support for videos 2021-11-05 09:43:49 +01:00
nonamethanks
043f2fb124 Add Foundation support 2021-11-01 01:39:56 +01:00