Commit Graph

11312 Commits

Author SHA1 Message Date
evazion
8705b8ec89 Merge pull request #5053 from NamelessContributor/fix-5052
posts: fix sidebar min-width on small screens (fix #5052)
2022-03-22 03:42:41 -05:00
evazion
a5115473d0 Merge pull request #5064 from CoreMack/master
Fix modqueue highlighting after topic #20445 (screenshots)
2022-03-22 03:42:12 -05:00
evazion
4d885acdbb Merge pull request #5061 from SystemZ/feat-conf-max-video-length
Configurable max video duration during upload
2022-03-22 03:41:32 -05:00
evazion
dd764bb4cf Merge pull request #5062 from NamelessContributor/fix-naver-duplicate
artists: add naver.com to artist finder blacklist.
2022-03-22 03:41:06 -05:00
CoreMack
dc45e6ddcb correct modqueue screencap highlighting 2022-03-21 16:03:07 -07:00
evazion
452ce8d165 artstation: add partial support for video clips (#5063).
Add partial support for fetching videos from ArtStation posts that
contain videos. Most of this code is disabled for now because actually
downloading these videos requires bypassing a Cloudflare captcha.
2022-03-21 16:51:42 -05:00
Michał Frąckiewicz
93635a20d9 Configurable max video duration 2022-03-21 19:22:34 +01:00
NamelessContributor
038c767455 artists: add naver.com to artist finder blacklist. 2022-03-21 17:34:44 +01:00
evazion
3fc01de19c reports: fix exception when deleting reported comments or forum posts.
Fix regression in 1a4efbda3. Locking the comment before validation
failed when the comment had unsaved changes, as is the case when
clearing reports from a comment before it is deleted.
2022-03-21 03:53:53 -05:00
evazion
56f47c60e1 posts: fix exception when viewing post with source Blog..
Fix a PublicSuffix::DomainNotAllowed exception raised with viewing or editing a post
with a source like `Blog.`.

This happened when parsing the post's source. `Danbooru::URL.parse("Blog.")` would
heuristically parse the source into `http://blog`. Calling any methods related to the
URL's hostname or domain would lead to calling `PublicSuffix.parse("blog")`, which
would fail with PublicSuffix::DomainNotAllowed.
2022-03-21 03:24:50 -05:00
evazion
defea08084 posts: fix exception in random:1 searches.
Fix regression in 1ad0e8688. Caused by `relation.order_values` returning
an array of Arel nodes instead of an array of strings when doing a
`random:1` search.
2022-03-21 01:29:10 -05:00
evazion
f52dc9e2ad tinami: fix 'http already memoized' warning. 2022-03-21 00:37:55 -05:00
evazion
1a4efbda33 Fix #5058: Duplicate report can't be rejected. 2022-03-21 00:36:59 -05:00
evazion
7c887f8adc artists: fix exception when adding TwitPic urls. 2022-03-20 21:56:38 -05:00
evazion
705edfb175 artists: add more patterns to artist finder blacklist. 2022-03-20 21:27:38 -05:00
evazion
7394660ba9 posts: fix exception when post has source like 'https://www.twitter.com/username'.
`twitter.com` sources worked but `www.twitter.com` didn't.

Also match the URL by class instead of by site name to ensure we match
the expected class.
2022-03-20 21:08:05 -05:00
evazion
1ad0e8688d posts: fix timeouts for searches using sequential navigation.
Fix certain searches timing out when using sequential navigation (page=b1234).

The problem was that the so-called "small search optimization" (AKA: force Postgres
to use the tag index for small searches instead a sequential scan) wasn't triggering
because the ORDER BY clause for sequential navigation was `posts.id desc`, and we
were only checking for `posts.id DESC`.
2022-03-20 18:46:06 -05:00
evazion
4b1264991f users: remove 'spoilers' tag from default blacklist.
Rationale:

* The spoilers tag is the most frequently removed tag from the default blacklist.
* It's frustrating for regular users to have posts randomly hidden because of trivial
  spoilers from a series they don't care about.
* The spoilers tag is used way too liberally for things that aren't considered
  spoilers on other sites.
* If you're looking up fanart on the internet, you should expect to see a certain
  level of spoilers.
* The tag is used very inconsistently, with some characters like Nia_(blade)_(xenoblade)
  getting the spoilers tag half the time and the rest of the time not.
2022-03-20 16:49:36 -05:00
evazion
71f42d67a7 tinami: return nothing if getting the full image fails.
Fix to make sure `image_urls` returns an empty array instead of `[nil]`
if grabbing the full image URL fails for whatever reason.
2022-03-19 23:42:34 -05:00
evazion
7f58cfbe5e tinami: get the full image.
Support grabbing the full image for Tinami uploads, rather than the sample.

Getting the full image requires making a request like this:

    curl -X POST \
    -H 'Referer: https://www.tinami.com/' \
    -H 'Content-Type: application/x-www-form-urlencoded' \
    -H 'Cookie: Tinami2SESSID=<redacted>;' \
    --data-raw 'action_view_original=true&cont_id=1087268&ethna_csrf=<redacted>' \
    https://www.tinami.com/view/1087268

Then scraping the <img> tag from the resulting HTML page.

If the post has multiple images, then we need to scrape and pass the
`sub_id` of the image too.

Fixes #2818.
2022-03-19 23:22:09 -05:00
evazion
0ddc09f011 forms: remove "Use * for wildcard" hints. 2022-03-19 21:14:37 -05:00
NamelessContributor
3b05abc012 posts: fix sidebar min-width on small screens
fix #5052
2022-03-20 03:09:07 +01:00
evazion
7dd6cf56ce Merge pull request #5051 from NamelessContributor/update-site-icons
url icons: replace some icons with better versions
2022-03-19 19:07:13 -05:00
evazion
f7cc941882 Merge pull request #5049 from NamelessContributor/show-hints-on-mobile
simple_form: Don't hide hints on mobile. Fix #4997
2022-03-19 19:02:46 -05:00
evazion
4215b77aaf Merge pull request #5048 from NamelessContributor/patch-1
editorconfig: add scss
2022-03-19 19:01:51 -05:00
evazion
01b683798e sources: add Tinami support. 2022-03-19 00:50:36 -05:00
evazion
40cbc0423c sources: add Instagram profile url normalization. 2022-03-18 18:20:29 -05:00
evazion
f0cd6227c3 tests: fix referer spoofing test. 2022-03-18 17:26:40 -05:00
evazion
c6e528a073 artstation: normalize https://artstation.com/artist/username/albums/all/ urls. 2022-03-18 17:10:26 -05:00
evazion
cc54b5f730 fanbox: normalize http://www.pixiv.net/fanbox/creator/3113804/post urls. 2022-03-18 17:10:26 -05:00
evazion
26d23c49d0 pawoo: normalize https://pawoo.net/users/evazion urls. 2022-03-18 17:10:26 -05:00
evazion
512c72bbc9 Show warnings in edit forms.
Make all edit forms show warnings in addition to errors. Also render
warnings and errors using DText.

Currently this only affects artists and wiki pages because they're the
only pages that have warnings.
2022-03-18 17:10:26 -05:00
evazion
912e996027 Fix #4470: Check URLs for duplicates when creating artists
Show a warning when creating a duplicate artist; that is, when adding a
URL that already belongs to another artist.

This is a soft warning rather than a hard error because there are some
cases where multiple artists legitimately share the same site or account.
2022-03-18 17:10:23 -05:00
evazion
a78b6528dc weibo: fix https://m.weibo.cn/detail/1234 urls not finding the artist.
Fix https://m.weibo.cn/detail/4506950043618873 type URLs not finding the
artist because the profile_url method returned nil instead of the actual
profile URL.
2022-03-18 06:01:51 -05:00
evazion
03d2098d6d artists: fix artist finder returning wrong results when given nil url.
Fix the artist finder returning incorrect results when given a nil URL.
This only happened when an artist with a URL like this existed:

    http:///blog.naver.com/dan_rak

Note the triple `///`; the extra `/` messed up the artist finder.

The artist finder may be given a nil URL when a source strategy returns
a nil profile URL, usually because the source is bad_id.
2022-03-18 06:01:36 -05:00
evazion
42144eaa4b Fix #5012: Fc2 image link paste not uploading.
Fix referer spoofing not working for certain fc2.com image URLs.

Spoofing the referer like this redirects to an HTML error page:

* curl -H "Referer: http://wwwew.web.fc2.com" http://wwwew.web.fc2.com/e/405.jpg

Spoofing it like this works:

* curl -H "Referer: http://wwwew.web.fc2.com/e/405.jpg" http://wwwew.web.fc2.com/e/405.jpg
2022-03-18 04:39:13 -05:00
evazion
c64df46de4 artists: make artist finder use url instead of normalized_url.
Make the artist finder search for artists using the `url` field instead
of the `normalized_url` field. This lets us get rid of `normalized_url`
in the future.

As described in 10dac3ee5, artist URLs have both a `url` column and a
`normalized_url` column. The `normalized_url` column was the one used
for artist finding. The `url` was secretly normalized behind the scenes
so that artist finding would work no matter how the URL was written in
the artist entry. This is no longer necessary now that URLs are directly
normalized in artist entries.

This fixes various cases where artist finding didn't work for non-obvious
reasons, usually because the URL wasn't written in the right format so
it wasn't properly normalized behind the scenes.

This also makes it so that artist finding is case-insensitive, which
fixes #4821. Hopefully no sites are perverse enough to allow two
different usernames that differ only in case.

Users running their own Danbooru instance may have to fix the URLs in
their artist entries for artist finding to work again. There are a few
fix scripts to help with this:

* script/fixes/104_normalize_weibo_artist_urls.rb
* script/fixes/105_normalize_pixiv_artist_urls.rb
* script/fixes/106_normalize_artist_urls.rb
2022-03-18 04:00:16 -05:00
evazion
10dac3ee51 artists: normalize urls added to artist entries.
When a URL is added to an artist entry, normalize it to a standard form.

Artist URLs have both a `url` column and a `normalized_url` column. The
`normalized_url` is used for artist finding and the `url` is the raw URL
entered by the user. Previously only the `normalized_url` field was
normalized; now the URL entered by the user is also converted to a
normalized form.

This means that if an URL like this is added to an artist entry:

* http://www.pixiv.net/member.php?id=1234
* http://www.pixiv.net/en/users/1234
* http://www.twitter.com/DanbooruBot/
* http://mobile.twitter.com/DanbooruBot/

It will get normalized to this:

* https://www.pixiv.net/users/1234
* https://twitter.com/DanbooruBot

This fixes problems with duplicate URLs being added to artist entries
because URLs weren't normalized to a single form.
2022-03-18 02:06:50 -05:00
evazion
455ee9a52a fc2: parse more url types. 2022-03-18 02:06:30 -05:00
evazion
03d2a86ef1 artists: normalize fc2.com profile urls. 2022-03-17 19:42:57 -05:00
evazion
04c03fa4e6 artist: normalize more artist url formats. 2022-03-16 17:17:50 -05:00
NamelessContributor
c669b16c32 url icons: replace some icons with better versions
mangaupdates, nijie, sakura-ne-jp: replaced with higher resolution,
  the old icons look blurry on hidpi screens.
skeb, tumblr: replaced with transparent versions.
2022-03-15 16:12:08 +01:00
NamelessContributor
c3a5ce9019 simple_form: Don't hide hints on mobile. Fix #4997
Hints are displayed below their fields on small screens.
2022-03-15 14:39:37 +01:00
NamelessContributor
1a4d9dc7da editorconfig: add scss 2022-03-15 14:06:16 +01:00
evazion
ded03df1ff tests: fix more broken tests. 2022-03-15 05:14:56 -05:00
evazion
644dfaf74c tests: fix broken tests. 2022-03-15 04:45:30 -05:00
evazion
133c45ee29 sources: parse more profile url formats.
Add support for parsing these URL formats:

* https://www.artstation.com/felipecartin/profile
* https://www.deviantart.com/nlpsllp/gallery
* https://fantia.jp/asanagi
* https://www.lofter.com/front/blog/home-page/noshiqian
* https://www.lofter.com/app/xiaokonggedmx
* https://www.lofter.com/blog/semblance
* https://q.nicovideo.jp/users/18700356
* https://dic.nicovideo.jp/u/11141663
* https://3d.nicovideo.jp/users/109584
* https://3d.nicovideo.jp/u/siobi
* https://game.nicovideo.jp/atsumaru/users/7757217
* https://www.pixiv.net/user/13569921/series/81967
* https://pixiv.cc/zerousagi/
* https://www.plurk.com/u/ddks2923
* https://www.plurk.com/m/u/leiy1225
* https://www.plurk.com/s/u/salmonroe13
* https://www.plurk.com/RSSSww/invite/4
* https://skeb.jp/@okku_oxn/works
* https://www.tumblr.com/blog/view/artofelaineho/187614935612
* https://www.tumblr.com/blog/view/artofelaineho
* https://www.tumblr.com/blog/artofelaineho
* https://www.tumblr.com/dashboard/blog/dankwartart
* https://rosarrie.tumblr.com/archive
* https://whereisnovember.tumblr.com/tagged/art
* https://twitpic.com/photos/Type10TK
* https://www.weibo.com/detail/4676597657371957
* https://www.weibo.com/u/5957640693/home?wvr=5
* https://www.weibo.com/lvxiuzi0/home
2022-03-15 00:49:54 -05:00
evazion
04226d3409 pixiv: normalize pixiv urls in artist entries.
Normalize Pixiv URLs to `https://www.pixiv.net/users/1234` format.
2022-03-14 16:43:19 -05:00
evazion
223742c365 weibo: normalize weibo urls in artist entries.
Normalize all Weibo URLs in artist entries to one of these forms:

* https://www.weibo.com/u/5399876326
* https://www.weibo.com/p/1005055399876326
* https://www.weibo.com/chengziyou666
2022-03-13 21:16:56 -05:00
evazion
1d9a15a119 weibo: handle a couple more profile url types.
Parse these profile URL types:

* https://www.weibo.cn/endlessnsmt
* https://www.weibo.com/p/1005055399876326

Also add anchors around the regexes so they have to match the full string.
2022-03-13 20:32:57 -05:00