Commit Graph

12031 Commits

Author SHA1 Message Date
evazion
f9b8e83bb0 emails: fix deliverability check when MX lookup returns nonexistent domain.
Fix the email deliverability check to return undeliverable when the MX lookup returns a domain or IP
address that is invalid or that refuses connections. For example, `yeah.com` returns the invalid IP
address 0.0.0.0 for its MX record.
2022-11-20 22:39:12 -06:00
evazion
80faee67db forum: fix mod action when forum post is deleted.
Fix forum posts logging an "updated forum #1234" mod action instead of a "deleted forum #1234" mod
action when a forum post was deleted.
2022-11-20 22:24:55 -06:00
evazion
1e478ab1b5 favgroups: add stricter favgroup naming rules.
Don't allow favgroup names that:

* Start or end with underscores.
* Contain multiple underscores in a row.
* Contain asterisks or non-printable characters.
* Consist of only underscores.
* Consist of only digits (conflicts with `favgroup:1234` syntax).

Add a fix script that fixes favgroups that violate these rules and notifies the user.
2022-11-20 22:09:57 -06:00
evazion
4fd028a5ce artists: fix ban/unban actions.
Fix the ban! and unban! methods to:

* Lock the artist while it is being banned or unbanned.
* Perform the edits as a mass update, so that the posts are updated in parallel.
* Edit the artist as the banner rather than as the current user.
* Soft delete the banned_artist implication when an artist is unbanned instead of hard deleting it.
* Ignore the banned_artist implication if it's deleted.
2022-11-20 19:25:20 -06:00
evazion
01c6d11253 bans: validate ban duration is standard length. 2022-11-20 17:10:20 -06:00
evazion
001ce99dd3 feedbacks: don't send redundant dmails when user is banned or loses approver status.
When a user is banned, send them a "You have been banned" dmail instead of a "Your user record has
been updated" dmail.

When a user loses approver status due to inactivity, don't seen them a "Your user record has been
updated" dmail for the "Lost approver privileges" neutral feedback they receive.
2022-11-20 17:10:20 -06:00
evazion
bbe32f5e5d feedbacks: update messaging about negative feedbacks.
Update wording about negative feedbacks on the new feedback page, and on the Dmail you receive when
you receive a negative feedback.
2022-11-20 17:10:20 -06:00
evazion
cc13c5cd57 feedbacks: fix mod action when feedback is deleted.
Fix feedbacks logging an "updated user feedback" mod action instead of a "deleted user feedback"
mod action when a feedback was deleted.
2022-11-20 15:29:46 -06:00
evazion
d99da72744 4chan: fix commentary linebreaks. 2022-11-20 14:34:25 -06:00
evazion
d63d0b1adc uploads: add archives filetypes to file upload component. 2022-11-20 14:30:02 -06:00
evazion
468754702e docker: upgrade openresty and exiftool versions. 2022-11-19 21:34:38 -06:00
evazion
f8d2758f8b uploads: fix sources for 4chan uploads.
Fix a bug where, if you were uploading an entire 4chan thread, then the source of each post would
get set to the 4chan thread, rather than to the individual 4chan post.
2022-11-18 21:27:10 -06:00
evazion
902cd0bbdf sources: add 4chan.org upload support.
Add upload support for 4chan.org. You can upload either individual posts or entire threads at a time.
2022-11-18 01:23:15 -06:00
evazion
94824fb171 bilibili: fix API calls returning HTTP 412.
Fix API calls returning HTTP 412 (Precondition Failed) errors. It seems we need to spoof the Referer as
`https://bilibili.com` for API calls to succeed.
2022-11-16 22:26:01 -06:00
evazion
96e1b6a512 Fix #5063: Support downloading ArtStation video clips. 2022-11-16 22:26:01 -06:00
evazion
869fddbb1a tests: fix broken tests. 2022-11-16 22:26:01 -06:00
evazion
a747911df8 foundation: fix foundation.app/@name/~/$id type URLs.
Fix URLs like this not working:

    https://foundation.app/@mochiiimo/~/97376

The correct URL is now:

    https://foundation.app/@mochiiimo/foundation/97376
2022-11-16 22:26:01 -06:00
evazion
491cce2f8c Update Ruby gems and Yarn packages. 2022-11-16 22:26:01 -06:00
evazion
6798e2ccf5 Merge pull request #5348 from nottalulah/patch-2
fix category text formatting in CategorizedTagList template
2022-11-16 17:21:16 -06:00
evazion
2deae38a4e uploads: allow uploading .zip, .rar., and .7z files from disk.
Allow uploading .zip, .rar, and .7z files from disk. The archive will be extracted and the images
inside will be uploaded.

This only works for archive files uploaded from disk, not from a source URL.

Post source URLs will look something like this: "file://foo.zip/1.jpg", "file://foo.zip/2.jpg", etc.
Sometimes artists uses Shift JIS or other encodings instead of UTF-8 for filenames. In these cases
we just assume the filename is UTF-8 and replace invalid characters with '?', so filenames might be
wrong in some cases.

There are various protections to prevent uploading malicious archive files:

* Archives with more than 100 files aren't allowed.
* Archives that decompress to more than 100MB aren't allowed.
* Archives with filenames containing '..' components aren't allowed (e.g. '../../../../../etc/passwd').
* Archives with filenames containing absolute paths aren't allowed (e.g. '/etc/passwd').
* Archives containing symlinks aren't allowed (e.g. 'foo -> /etc/passwd').
* Archive types other than .zip, .rar, and .7z aren't allowed (e.g. .tar.gz, .cpio).
* File permissions, owners, and other metadata are ignored.

Partial fix for #5340: Add support for extracting archive attachments from certain sources
2022-11-16 16:47:37 -06:00
evazion
d791924aad posts: fix /posts/random?tags=... redirect.
Fix /posts/random?tags=touhou to redirect to /posts/1234?q=touhou instead of /posts/1234?tags=touhou.

Fixes the search bar on random posts not containing the current search.
2022-11-15 19:38:43 -06:00
evazion
bc169fc98c posts: fix exception in random:1 filetype:png. 2022-11-15 19:33:25 -06:00
evazion
e935f01358 uploads: fix temp files not being cleaned up quickly enough.
Fix temp files generated during the upload process not being cleaned up quickly enough. This included
downloaded files, generated preview images, and Ugoira video conversions.

Before we relied on `Tempfile` cleaning up files automatically. But this only happened when the
Tempfile object was garbage collected, which could take a long time. In the meantime we could have
hundreds of megabytes of temp files hanging around.

The fix is to explicitly close temp files when we're done with them. But the standard `Tempfile`
class doesn't immediately delete the file when it's closed. So we also have to introduce a
Danbooru::Tempfile wrapper that deletes the tempfile as soon as it's closed.
2022-11-15 18:50:50 -06:00
evazion
21a779455f discord: fix /tagme command to work with html page URLs. 2022-11-15 15:22:56 -06:00
evazion
aa21f002be tests: fix cleanup of upload temp files.
Fix upload temp files sometimes not being cleaned up when tests exit.
2022-11-15 15:16:50 -06:00
evazion
53f40b3331 gems: add back pry gem.
The `pry` gem was removed in e698bf91 because we replaced `pry-byebug` with the standard debugger.
Add it back because `pry` is better than `irb` and we still can use the standard debugger with it.
2022-11-15 14:18:22 -06:00
Lily
4bc4365cca fix category text formatting in CategorizedTagList template 2022-11-15 04:43:34 -04:00
evazion
f128c48485 archives: add code for detecting .rar and .7z files. 2022-11-14 20:14:37 -06:00
evazion
5f92f452fe media file: factor out file type detection code.
Factor out the file type detection code from MediaFile into a FileTypeDetector class so we can use
it to detect archive files (.zip, .rar, .7z) too.
2022-11-14 20:14:37 -06:00
evazion
0c1e9a1618 Add Danbooru::Archive library for handling .zip and .rar files.
Introduce a new Danbooru::Archive library. This is a wrapper around libarchive that lets us extract
.zip, .rar, .7z, and other archive formats. Replace the rubyzip library in MediaFile::Ugoira with
the new Danbooru::Archive library.

This is a step towards fixing #5340: Add support for extracting archive attachments from certain sources.

This adds a new dependency on libarchive. Downstream users should `apt-get install libarchive13` if
they're not using Docker.

https://github.com/chef/ffi-libarchive
https://github.com/libarchive/libarchive
https://www.rubydoc.info/gems/ffi-libarchive/0.4.2
https://github.com/libarchive/libarchive/wiki/Examples#a-complete-extractor
2022-11-14 20:14:37 -06:00
evazion
f942768ce8 Fix #5347: Don't use exception template for post validation errors
Also fixes #5173: Parenting a post to itself using the "parent id" box leads to an error.
2022-11-13 02:01:18 -06:00
evazion
33e4cbeb8d Fix #5179: Add wiki page versions to wiki page secondary links plus standardization
* Call it "Changes" when referring to the global list of changes (pool changes, artist changes, etc).
* Call it "History" when referring to the history of a single item (pool history, artist history, etc).
* Put the artist history search form on the /artist_versions page, and remove the /artist_versions/search endpoint.
2022-11-13 00:58:11 -06:00
evazion
d053dc7356 Fix #5143: Allow name changes with same name but changed case. 2022-11-12 23:04:45 -06:00
evazion
22f232f4e9 Fix #5209: dtext link search doesn't work right. 2022-11-12 22:18:11 -06:00
evazion
d7005c6b3d Fix #5195: Export ArtistCommentary Javascript Functions. 2022-11-12 22:11:04 -06:00
evazion
e12ce9b472 Fix #5343: Add icons for websites linked to at least 9 artists. 2022-11-12 19:49:20 -06:00
evazion
220db642e1 Fix #5346: Can add tags beginning with newpool:, causing the next edit to add the post to a pool. 2022-11-12 17:01:52 -06:00
evazion
86669fa605 Fix #5344: populate.rb creates users with invalid names. 2022-11-12 16:44:30 -06:00
evazion
d5676bf0fd Merge pull request #5345 from nottalulah/patch-1
wiki pages: don't show create links to anonymous users on non-existent pages
2022-11-12 16:41:27 -06:00
Lily
32fbc4e65f wiki pages: don't show create links to anonymous users on non-existent pages 2022-11-12 09:21:20 -04:00
evazion
a167091bf9 Danbooru::Http: fix bug when using proxy option.
Fix external HTTP requests not working when the HTTP proxy was enabled. Caused by the `public_only`
option (which prevents SSRF attacks by validating that the URL doesn't resolve to a local IP) being
incompatible with the `proxy` option.
2022-11-12 02:44:05 -06:00
evazion
215df49050 config: rework http proxy options.
Replace the http_proxy_host, http_proxy_port, http_proxy_username, http_proxy_password options with
a single `Danbooru.config.http_proxy` option.
2022-11-12 02:44:01 -06:00
evazion
4c3b988d15 sources: fix site icons. 2022-11-11 14:03:06 -06:00
evazion
81cba9b6f3 posts: autodetect more AI-generated images. 2022-11-11 14:03:06 -06:00
nonamethanks
d525df9ad5 Newgrounds: fix exception for deleted videos
Fixup for 8c0f2255f9
2022-11-11 12:28:23 +01:00
nonamethanks
8c0f2255f9 Newgrounds: fix support for some old videos 2022-11-11 11:01:13 +01:00
evazion
0fd95375fa sources: add more site icons.
Add site icons for the following sites:

* 4chan
* Catbox
* Danbooru
* Drawcrowd
* E-Hentai
* Enty
* Gelbooru
* Hitomi
* Imgur
* Joyreactor
* Kemono Party
* Konachan
* Luscious.net
* Mega.nz
* Monappy
* Mihoyo
* Overdoll
* Pinterest
* Rule34.us
* Sankaku Complex
* Steam
* Webmshare
* Yande.re
* Zerochan

These are mainly used for source links on the https://danbooru.donmai.us/post_events?search[category]=Replacement page.
2022-11-11 00:54:50 -06:00
evazion
0a792fbb8a replacements: link to old and new media assets.
On the /post_events and /post_replacements pages, include links to the old and new media assets
pages so you can compare the two images.
2022-11-11 00:54:10 -06:00
evazion
6f61abc6a7 users: don't log mod action for user deletions.
Don't log a mod action when a user deletes their own account. This isn't a moderator action, so it
doesn't belong here. Account deletions are still logged on the /user_events page (visible to mods only).

A mod action is still logged when the Owner-level user deletes someone else's account.
2022-11-10 14:30:23 -06:00
evazion
15f611ca8b Add Rule34.us support. 2022-11-10 12:53:06 -06:00