Add partial support for fetching videos from ArtStation posts that
contain videos. Most of this code is disabled for now because actually
downloading these videos requires bypassing a Cloudflare captcha.
Fix regression in 1a4efbda3. Locking the comment before validation
failed when the comment had unsaved changes, as is the case when
clearing reports from a comment before it is deleted.
Fix a PublicSuffix::DomainNotAllowed exception raised with viewing or editing a post
with a source like `Blog.`.
This happened when parsing the post's source. `Danbooru::URL.parse("Blog.")` would
heuristically parse the source into `http://blog`. Calling any methods related to the
URL's hostname or domain would lead to calling `PublicSuffix.parse("blog")`, which
would fail with PublicSuffix::DomainNotAllowed.
Fix regression in 1ad0e8688. Caused by `relation.order_values` returning
an array of Arel nodes instead of an array of strings when doing a
`random:1` search.
Fix certain searches timing out when using sequential navigation (page=b1234).
The problem was that the so-called "small search optimization" (AKA: force Postgres
to use the tag index for small searches instead a sequential scan) wasn't triggering
because the ORDER BY clause for sequential navigation was `posts.id desc`, and we
were only checking for `posts.id DESC`.
Rationale:
* The spoilers tag is the most frequently removed tag from the default blacklist.
* It's frustrating for regular users to have posts randomly hidden because of trivial
spoilers from a series they don't care about.
* The spoilers tag is used way too liberally for things that aren't considered
spoilers on other sites.
* If you're looking up fanart on the internet, you should expect to see a certain
level of spoilers.
* The tag is used very inconsistently, with some characters like Nia_(blade)_(xenoblade)
getting the spoilers tag half the time and the rest of the time not.
Support grabbing the full image for Tinami uploads, rather than the sample.
Getting the full image requires making a request like this:
curl -X POST \
-H 'Referer: https://www.tinami.com/' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Cookie: Tinami2SESSID=<redacted>;' \
--data-raw 'action_view_original=true&cont_id=1087268ðna_csrf=<redacted>' \
https://www.tinami.com/view/1087268
Then scraping the <img> tag from the resulting HTML page.
If the post has multiple images, then we need to scrape and pass the
`sub_id` of the image too.
Fixes#2818.
Make all edit forms show warnings in addition to errors. Also render
warnings and errors using DText.
Currently this only affects artists and wiki pages because they're the
only pages that have warnings.
Show a warning when creating a duplicate artist; that is, when adding a
URL that already belongs to another artist.
This is a soft warning rather than a hard error because there are some
cases where multiple artists legitimately share the same site or account.
Fix the artist finder returning incorrect results when given a nil URL.
This only happened when an artist with a URL like this existed:
http:///blog.naver.com/dan_rak
Note the triple `///`; the extra `/` messed up the artist finder.
The artist finder may be given a nil URL when a source strategy returns
a nil profile URL, usually because the source is bad_id.
Make the artist finder search for artists using the `url` field instead
of the `normalized_url` field. This lets us get rid of `normalized_url`
in the future.
As described in 10dac3ee5, artist URLs have both a `url` column and a
`normalized_url` column. The `normalized_url` column was the one used
for artist finding. The `url` was secretly normalized behind the scenes
so that artist finding would work no matter how the URL was written in
the artist entry. This is no longer necessary now that URLs are directly
normalized in artist entries.
This fixes various cases where artist finding didn't work for non-obvious
reasons, usually because the URL wasn't written in the right format so
it wasn't properly normalized behind the scenes.
This also makes it so that artist finding is case-insensitive, which
fixes#4821. Hopefully no sites are perverse enough to allow two
different usernames that differ only in case.
Users running their own Danbooru instance may have to fix the URLs in
their artist entries for artist finding to work again. There are a few
fix scripts to help with this:
* script/fixes/104_normalize_weibo_artist_urls.rb
* script/fixes/105_normalize_pixiv_artist_urls.rb
* script/fixes/106_normalize_artist_urls.rb
mangaupdates, nijie, sakura-ne-jp: replaced with higher resolution,
the old icons look blurry on hidpi screens.
skeb, tumblr: replaced with transparent versions.
Change how artist URLs are normalized in artist entries. Don't try to secretly
convert image URLs to profile URLs in artist entries. For example, if someone puts a
Pixiv image URL in an artist entry, don't secretly try to fetch the source and
convert it into a profile URL in the `normalized_url` field.
We did this because years ago, it was standard practice to put image URLs in artist
entries. Pixiv image URLs used to contain the artist's username, so we used to put
image URLs in artist entries for artist finding purposes. But Pixiv changed it so
that image URLs no longer contained the username, so we dealt with it by adding a
`normalized_url` column to artist_urls and secretly converting image URLs to profile
URLs in this field. But this is no longer necessary because now we don't normally put
image URLs in artist entries in the first place.
Now the `profile_url` method in `Source::URL` is used to normalize URLs in artist
entries. This lets us parse various profile URL formats and normalize them into a
single canonical form.
This also removes the `normalize_for_artist_finder` method from source strategies.
Instead the `profile_url` method is used for artist finding purposes. So the profile
URL returned by the source strategy needs to be the same as the URL in the artist
entry in order for artist finding to work.
Add a method for converting a source URL into a profile URL. This will
be used for normalizing profile URLs in artist entries.
Also add the ability to parse a few more profile URL formats.
This turns out to be a little simpler than keeping them separate. The
only thing special we have to do for Sta.sh is use the Sta.sh page when
we have a DeviantArt image with a Sta.sh referer.
Bug: We assumed the referer URL was from the same site as the target
URL. We tried to call methods on the referer only supported by the
target URL.
Fix: Ignore the referer URL when it's from a different site than the
target URL.
