Commit Graph

9334 Commits

Author SHA1 Message Date
evazion
a160a3acce users: add stricter username rules.
Add stricter username rules:

* Only allow usernames to contain basic letters, numbers, CJK characters, underscores, dashes and periods.
* Don't allow names to start or end with punctuation.
* Don't allow names to have multiple underscores in a row.
* Don't allow active users to have names that look like deleted users (e.g. "user_1234").
* Don't allow emoji or any other Unicode characters except for Chinese, Japanese, and Korean
  characters. CJK characters are currently grandfathered in but will be disallowed in the future.

Users with an invalid name will be shown a permanent sitewide banner until they change their name.
2022-03-05 01:08:53 -06:00
evazion
ca98e218a1 users: don't bold usernames in comments and forum posts.
Make usernames in comments and forum posts take up less space.
2022-03-01 21:23:21 -06:00
evazion
b4620f561c users: lower max username length to 25 characters.
The median username length is 8 characters. The 99% percentile is 18
characters. The 99.9% percentile is 24 characters. About 750 users have
a name more than 24 characters long.

This doesn't do anything about existing users with long usernames.

Note that this is the length in Unicode codepoints, not grapheme
clusters. Some Unicode characters and emoji may be a single glyph but
composed of multiple codepoints.
2022-03-01 21:23:21 -06:00
evazion
2d4106154b Merge pull request #5025 from NamelessContributor/patch-1
Limit the width of usernames in tables
2022-03-01 20:45:07 -06:00
evazion
68c92b4536 js: fix Javascript failures in Seamonkey/Palemoon.
Fix site Javascript failing to load in Seamonkey, Palemoon, and other
older browsers.

The @alpinejs/morph library uses public instance fields, which is ES2022
syntax not supported in older browsers. This is the code:

    var DomManager = class {
        el = void 0; // `el` is a public instance field
    }
    // => SyntaxError: bad method definition

The fix here is to separate the Alpine code into a separate bundle so
that a failure to load it doesn't cause the rest of the site's
Javascript to fail to load.

A better fix would be to either transpile the @alpinejs/morph library to
ES5 (which seems difficult to do in webpacker), or to fix the library
upstream to not use this syntax.

* https://inspiredwebdev.com/everything-new-in-es2022/
* https://blog.saeloun.com/2021/10/21/ecmacscript-public-instance-fields-and-private-instance-fields.html
* https://caniuse.com/?search=public%20class%20fields
* https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Public_class_fields#public_instance_fields
2022-03-01 19:40:15 -06:00
evazion
ad3f3fdce3 Fix unqualified column references.
Fix various places to avoid unqualified column references to prevent any
potential ambiguous column errors.
2022-03-01 17:48:16 -06:00
evazion
036341d8ba aliases/implications: fix ambiguous column reference on index page.
Fix searches like this:

    https://danbooru.donmai.us/tag_aliases?search[consequent_tag][category]=1&search[order]=created_at

failing with an ambiguous column reference.
2022-03-01 17:38:46 -06:00
evazion
ece0b1b17d Fix exception on error page when handling PG::AmbiguousColumn error.
Fix an exception on the error page when a controller index action raised
an PG::AmbiguousColumn error because the model `search` method generated
SQL with an ambiguous column reference. In this case the error page
tried to generate data attributes for the <body> tag, but this failed
because evaluating the `current_item` raised an exception again.
2022-03-01 17:09:31 -06:00
NamelessContributor
a43291018a Limit the width of usernames in tables
Fixes the layout issues caused very long usernames in the forum and elsewhere.
2022-03-01 14:59:58 +01:00
evazion
99221af855 ugoiras: fix regression in 7031fd13d.
Fix `Cannot write log file 'ffmpeg2pass-0.log' for pass-1 encoding: Permission denied` error
when uploading ugoira files. Caused by the fact that 2-pass encoding tries to write a log file in
the current directory by default, which fails in production because the default working directory in
the Docker image is /danbooru, which is read-only.
2022-03-01 00:16:55 -06:00
evazion
03560bafc6 uploads: add limit to prevent users from submitting too many uploads at once.
Add a limit so that users can't upload more if they already have more
than 250 images queued for upload.

For example, if you upload a Pixiv post that has 200 images, then you'll
have 200 queued images for upload. This will go down as the images are
processed. If you exceed the limit, then trying to create new uploads
will return an error.

This is to prevent single users from overwhelming the site by uploading
too many images at once, thereby preventing other users from uploading
because the job queue is backed up and can't process new uploads by
other users until existing uploads are finished.
2022-02-28 23:10:12 -06:00
evazion
7031fd13d7 ugoiras: encode .webm samples using VP9 instead of VP8.
Switch the codec for .webm samples from VP8 to VP9. All modern browsers
support VP9 (Safari was the last to add support in ~2020), so it should
be safe to provide only VP9 .webms without a fallback.

VP9 lets us use two-pass encoding, which should offer better compression.

Fixes ugoira samples still having poor quality even after 4c652cf3e.
4c652cf3e tried to remove the max bitrate limit by setting `-b:v 0`, but
this only worked in FFmpeg 4.2. In production Danbooru uses FFmpeg 4.4,
and apparently in 4.4 `-b:v 0` means "use the default max bitrate of
256kb/s" instead of "no bitrate limit".

https://trac.ffmpeg.org/wiki/Encode/VP9
https://developers.google.com/media/vp9/bitrate-modes
https://developers.google.com/media/vp9/settings/vod
http://wiki.webmproject.org/ffmpeg/vp9-encoding-guide
https://www.reddit.com/r/AV1/comments/k7colv/encoder_tuning_part_1_tuning_libvpxvp9_be_more/
2022-02-28 22:02:56 -06:00
evazion
0def039015 Merge pull request #5023 from 9suika/fix/favgroup-updated-at
favorite_groups: make search[order]=updated_at work
2022-02-28 20:23:07 -06:00
evazion
8ef05ec69b Fix #5021: Downvoting/upvoting a revealed hidden comment will hide it again.
Fix it so that upvoting or downvoting a revealed thresholded comment
doesn't hide it again.

The fix is to explicitly store a `data-show-thresholded` flag on the
comment, instead of manually hiding elements with jQuery, and to morph
the comment HTML instead of replacing it so that the state isn't lost
after voting. Alpine.js is used for this, which isn't strictly necessary,
but is useful to test the library before adopting it on a wider scale.

https://alpinejs.dev/start-here
2022-02-28 20:11:22 -06:00
user
a41f291971 favorite_groups: make search[order]=updated_at work. 2022-02-28 23:50:22 +01:00
evazion
8ab0803e00 Merge pull request #5022 from 9suika/fix/nijie-urls
nijie: extract post ID from new image URL
2022-02-28 16:09:06 -06:00
evazion
b538a60582 Merge pull request #5018 from nonamethanks/deprecate-report-topic
Moderation Reports: deprecate the forum topic
2022-02-28 14:43:49 -06:00
user
2600dcdbfa nijie: extract post ID from new image URL. 2022-02-28 21:14:47 +01:00
user
550e0bef93 nijie: fix pattern for new image URL. 2022-02-28 21:14:31 +01:00
evazion
1609059bf4 sources: factor out Source::URL::Fanbox.
Also fix it so that we grab the full image for cover URLs like this:

* Sample: https://pixiv.pximg.net/c/1620x580_90_a2_g5/fanbox/public/images/creator/1566167/cover/QqxYtuWdy4XWQx1ZLIqr4wvA.jpeg
* Full: https://pixiv.pximg.net/fanbox/public/images/creator/1566167/cover/QqxYtuWdy4XWQx1ZLIqr4wvA.jpeg
2022-02-28 06:25:06 -06:00
evazion
317ec886bc sources: factor out Source::URL::Nijie.
Also fixes the uploader uploading all images when trying to upload only a
single image in a multi-image work. Caused by `image_urls` incorrectly
returning all images when the source strategy was given a url for a
single image.
2022-02-27 02:27:35 -06:00
evazion
926a8fa81f Danbooru::URL: add #basename, #filename, and #file_ext utility methods.
Add `#basename`, `#filename`, and `#file_ext` utility methods to
Danbooru::URL and change a few places to use them. Simplifies parsing
filenames in source URLs in various places.
2022-02-27 02:27:21 -06:00
evazion
fcf517834d sources: factor out Source::URL::ArtStation. 2022-02-26 21:03:49 -06:00
evazion
9169f00e80 sources: factor out Source::URL::Moebooru. 2022-02-26 17:46:44 -06:00
evazion
74fdeef10c sources: factor out Source::URL::Mastodon. 2022-02-26 15:08:27 -06:00
evazion
86d8e2d13d sources: factor out Source::URL::Lofter. 2022-02-25 23:43:10 -06:00
evazion
f062f2d145 sources: factor out Source::URL::Newgrounds.
Also fix it so that the image URL is set as the source for Newgrounds
posts, not the page URL. It's possible to generate the page URL from the
image URL (except for images after the first in multi-image posts).

* Page: https://www.newgrounds.com/art/view/natthelich/weaver
* Image: https://art.ngfiles.com/images/1520000/1520217_natthelich_weaver.jpg?f1606365031
2022-02-25 23:04:03 -06:00
evazion
64472a7b7e sources: factor out Source::URL::HentaiFoundry.
Add support for these URL types:

* http://pictures.hentai-foundry.com//s/soranamae/363663.jpg
* http://www.hentai-foundry.com/piccies/d/dmitrys/1183.jpg
* http://www.hentai-foundry.com/pic-149160.php
* http://www.hentai-foundry.com/user-RockCandy.php
* http://www.hentai-foundry.com/profile-sawao.php

These URL types are obsolete, but still present in some old posts.
2022-02-25 22:01:17 -06:00
evazion
5837b614d4 artists: fix exception on show page when artist has invalid URLs.
Fix an exception on the artist show page when the artist entry contained invalid URLs such
as `http://ttp://album.yahoo.co.jp/photos/my/8027988`. Caused by `ArtistUrl#domain`
returning nil for certain invalid URLs, which caused `Artist#sorted_urls` to blow up.

ref: https://danbooru.donmai.us/forum_posts/206488
2022-02-25 02:06:57 -06:00
evazion
e6ded89f85 sources: factor out Source::URL::Plurk.
Also fix it so that for adult works, we get the images posted by the
artist in the replies. Example: https://www.plurk.com/p/omc64y (nsfw).
2022-02-25 02:06:57 -06:00
evazion
26f4cf1ebd sources: factor out Source::URL::Skeb. 2022-02-25 02:06:57 -06:00
evazion
6523cd14cb Merge pull request #5019 from NamelessContributor/patch-1
Fix minor typo on Account Upgrade page
2022-02-24 01:58:03 -06:00
evazion
ffe52f5ead sources: factor out Source::URL::Foundation.
Add support for a couple more URL types:

* https://foundation.app/@asuka111art/dinner-with-cats-82426
* https://f8n-production-collection-assets.imgix.net/0x3B3ee1931Dc30C1957379FAc9aba94D1C48a5405/128711/QmcBfbeCMSxqYB3L1owPAxFencFx3jLzCPFx6xUBxgSCkH/nft.png

Also include these URLs in the list of profile URLs:

* https://foundation.app/0x7E2ef75C0C09b2fc6BCd1C68B6D409720CcD58d2 (for https://foundation.app/@mochiiimo)

These URLs should be stable even if the user changes their name.
2022-02-23 23:49:31 -06:00
evazion
043c08eb05 sources: factor out Source::URL::TwitPic. 2022-02-23 23:49:31 -06:00
evazion
7ed8f95a8e sources: add Source::URL class; factor out Source::URL::Twitter.
Introduce a Source::URL class for parsing URLs from source sites. Refactor the Twitter
source strategy to use it.

This is the first step towards factoring all the URL parsing logic out of source
strategies and moving it to subclasses of Source::URL. Each site will have a subclass
of Source::URL dedicated to parsing URLs from that site. Source strategies will use
these classes to extract information from URLs.

This is to simplify source strategies. Most sites have many different URL formats we have
to parse or rewrite, and handling all these different cases tends to make source
strategies very complex. Isolating the URL parsing logic from the site scraping logic
should make source strategies easier to maintain.
2022-02-23 23:46:04 -06:00
NamelessContributor
9ca47ad29e Fix minor typo on Account Upgrade page 2022-02-23 19:14:17 +01:00
evazion
6f5aef1cef uploads: fix being redirected to blank page when rating is not selected.
Fix the upload page redirecting you to a blank page if you forgot to
select the rating on a multi-asset upload.

ref: https://danbooru.donmai.us/forum_posts/206365
2022-02-23 02:50:40 -06:00
evazion
4c652cf3ec ugoiras: increase quality of webm samples.
Fix certain ugoiras having very low quality webm samples. This was
because we had a target bitrate of 5 Mbps, but this wasn't enough for
videos that were high resolution or that had choppy, hard-to-compress
motion, such as post 5081776 (nsfw).
2022-02-23 02:50:14 -06:00
nonamethanks
1a66feddcf Moderation Reports: deprecate the forum topic 2022-02-22 23:47:36 +01:00
evazion
112b323f01 foundation: fix exception when uploading new Foundation url format.
Fix 'null value in column "source_url"' exception when uploading urls like this:

* https://foundation.app/@KILLERGF/kgfgen/4
* https://foundation.app/@mochiiimo/foundation/97376
2022-02-22 13:29:28 -06:00
evazion
7b009cc893 nicoseiga: fix inability to login to nicoseiga.
NicoSeiga changed it so that on every login, you must enter a 2FA code
sent by email. This broke the NicoSeiga strategy. The fix is to just use
a static session cookie instead (and hope it doesn't expire, and isn't
tied to an IP).

The `nico_seiga_login` and `nico_seiga_password` config settings have
been removed from config/danbooru_default_config.rb and replaced by
`nico_seiga_user_session`. If you run your own Danbooru instance, you
will have to update your config file manually.
2022-02-22 12:23:01 -06:00
evazion
c5777f360e artist urls: normalize trailing slashes and missing http://.
* Remove unnecessary trailing slashes when artist URLs are saved.
* Automatically add `http://` to new artist URLs if it's missing (before
  this was an error; now it's automatically fixed).
2022-02-22 00:17:53 -06:00
evazion
7d49ab6130 Add Danbooru::URL class.
Introduce a Danbooru::URL class for dealing with URLs. This is a wrapper
around Addressable::URI that adds some additional helper methods. Most
significantly, the `parse` method only allows valid http/https URLs, and
it returns nil instead of raising an exception when the URL is invalid.
2022-02-22 00:17:53 -06:00
evazion
60a26af6e3 rails: add 'URL' inflection.
Make it so we can write `ArtistURL` instead of `ArtistUrl`.
2022-02-22 00:17:53 -06:00
evazion
fbab273c81 Upgrade http.rb gem to 5.0.4.
Fixes a bug where the Foundation source strategy failed because http.rb
automatically sent a `Content-Length: 0` header with all GET requests,
which caused Foundation to return a 400 Bad Request error. This behavior
was fixed in http.rb 5.x.

http.rb 5.x has a breaking change where it now includes the request object
inside the response object, which we have to handle in a few places.
2022-02-22 00:17:05 -06:00
evazion
35dca33abc uploads: remove batch upload link from navbar. 2022-02-21 00:29:46 -06:00
evazion
68ba447494 uploads: remove batch upload page.
* Make /uploads/batch redirect to /uploads/new.
* Remove /uploads/image_proxy.
2022-02-21 00:03:43 -06:00
evazion
3da9c1574f Merge pull request #5010 from nonamethanks/add-mod-report-link
Moderation Reports: add link for moderators to navbar
2022-02-20 22:45:30 -06:00
evazion
a916fd9e7b Fix #5015: 'Rating not selected' error leads to an empty page. 2022-02-19 17:25:14 -06:00
evazion
202dfe5d87 uploads: allow uploading multiple files from your computer at once.
Allow uploading multiple files from your computer at once.

The maximum limit is 100 files at once. There is still a 50MB size limit
that applies to the whole upload. This limit is at the Nginx level.

The upload widget no longer shows a thumbnail preview of the uploaded
file. This is because there isn't room for it in a multi-file upload,
and because the next page will show a preview anyway after the files are
uploaded.

Direct file uploads are processed synchronously, so they may be slow.

API change: the `POST /uploads` endpoint now expects the param to be
`upload[files][]`, not `upload[file]`.
2022-02-19 00:00:56 -06:00