Commit Graph

11033 Commits

Author SHA1 Message Date
evazion
892c3899a8 uploads: fix more tests. 2022-01-29 23:23:29 -06:00
evazion
810c3da939 favgroups: fix typo in new favgroup dialog. 2022-01-29 15:59:44 -06:00
evazion
09d6c602be uploads: fix long filenames not word-wrapping.
Fix a bug where, when uploading a file from disk, if the filename was
too long, it wouldn't get word-wrapped and could break out of the
containing element.
2022-01-29 05:14:49 -06:00
evazion
e1d4de1c22 uploads: fix double paste bug.
Fix a bug where, when pasting an URL directly into the URL field, the
URL would be pasted twice, which would cause the upload to fail because
the URL was invalid.
2022-01-29 05:14:49 -06:00
evazion
11b7bcac91 uploads: fix broken tests.
* Fix broken upload tests.
* Fix uploads to return an error if both a file and a source are given
  at the same time, or if neither are given. Also fix the error message
  in this case so that it doesn't include "base" at the start of the string.
* Fix uploads to percent-encode any Unicode characters in the source URL.
* Add a max filesize validation to media assets.
2022-01-29 05:14:49 -06:00
evazion
5d0c14d2bd uploads: fix file upload component colors in dark mode.
Fix the file upload component blending into the background in dark mode.
2022-01-28 21:15:08 -06:00
evazion
21dcf53dcb uploads: show similar images for disk uploads.
Fix the upload page so that it shows similar images (IQDB matches) for
files uploaded from your computer. Before this only worked for files
uploaded from a source.
2022-01-28 21:07:06 -06:00
evazion
c0730630af uploads: fixup issues causing Rails to fail to boot.
* Fix `UploadService is not a class` error.
* Update list of available job classes (remove UploadPreprocessorDelayedStartJob,
  UploadServiceDelayedStartJob, add ProcessUploadJob).
2022-01-28 04:32:20 -06:00
evazion
abdab7a0a8 uploads: rework upload process.
Rework the upload process so that files are saved to Danbooru first
before the user starts tagging the upload.

The main user-visible change is that you have to select the file first
before you can start tagging it. Saving the file first lets us fix a
number of problems:

* We can check for dupes before the user tags the upload.
* We can perform dupe checks and show preview images for users not using the bookmarklet.
* We can show preview images without having to proxy images through Danbooru.
* We can show previews of videos and ugoira files.
* We can reliably show the filesize and resolution of the image.
* We can let the user save files to upload later.
* We can get rid of a lot of spaghetti code related to preprocessing
  uploads. This was the cause of most weird "md5 confirmation doesn't
  match md5" errors.

(Not all of these are implemented yet.)

Internally, uploading is now a two-step process: first we create an upload
object, then we create a post from the upload. This is how it works:

* The user goes to /uploads/new and chooses a file or pastes an URL into
  the file upload component.
* The file upload component calls `POST /uploads` to create an upload.
* `POST /uploads` immediately returns a new upload object in the `pending` state.
* Danbooru starts processing the upload in a background job (downloading,
  resizing, and transferring the image to the image servers).
* The file upload component polls `/uploads/$id.json`, checking the
  upload `status` until it returns `completed` or `error`.
* When the upload status is `completed`, the user is redirected to /uploads/$id.
* On the /uploads/$id page, the user can tag the upload and submit it.
* The upload form calls `POST /posts` to create a new post from the upload.
* The user is redirected to the new post.

This is the data model:

* An upload represents a set of files uploaded to Danbooru by a user.
  Uploaded files don't have to belong to a post. An upload has an
  uploader, a status (pending, processing, completed, or error), a
  source (unless uploading from a file), and a list of media assets
  (image or video files).

* There is a has-and-belongs-to-many relationship between uploads and
  media assets. An upload can have many media assets, and a media asset
  can belong to multiple uploads. Uploads are joined to media assets
  through a upload_media_assets table.

  An upload could potentially have multiple media assets if it's a Pixiv
  or Twitter gallery. This is not yet implemented (at the moment all
  uploads have one media asset).

  A media asset can belong to multiple uploads if multiple people try
  to upload the same file, or if the same user tries to upload the same
  file more than once.

New features:

* On the upload page, you can press Ctrl+V to paste an URL and immediately upload it.
* You can save files for upload later. Your saved files are at /uploads.

Fixes:

* Improved error messages when uploading invalid files, bad URLs, and
  when forgetting the rating.
2022-01-28 04:13:22 -06:00
evazion
f11c46b4f8 uploads: stop pruning uploads. 2022-01-28 04:13:22 -06:00
evazion
6fbca01a23 css: make footer sticky. 2022-01-28 04:13:22 -06:00
evazion
c292c457ca Merge pull request #4977 from ZipFile/svg-favicon
Add svg favicon
2022-01-26 01:33:39 -06:00
evazion
c4775d96a9 uploads: add upload_media_assets table.
Add a join table that allows multiple media assets (images or videos) to
be attached to uploads. This is for a future ability to upload multiple
files at once.
2022-01-25 16:10:53 -06:00
ZipFile
f926956697 Add svg favicon 2022-01-25 19:42:24 +00:00
evazion
a1d650848b forum: fix undelete option not appearing in menu.
Fix the option to undelete forum posts not appearing in the context menu
for Moderators.
2022-01-25 12:43:55 -06:00
evazion
5c97595c6a posts: fix post view counts not being recorded.
Broken by the upgrade to webpacker-6.0.0.rc.6. Webpacker now defaults to
loading the Javascript bundle with `<script src="..." defer>`, which
means Javascript won't be loaded until after the page loads, which broke
the code that records view counts because it uses jQuery and jQuery
wasn't loaded yet.
2022-01-24 01:53:30 -06:00
evazion
0213f8d76a css: remove image-rendering: smooth.
Contrary to its name, `image-rendering: smooth` resulted in pixelated
edges when images were downscaled to fit the browser window. This only
affected Firefox because other browsers don't support `smooth`.
2022-01-22 22:04:30 -06:00
evazion
f02f3fcc6f css: fix text sizes in desktop mode on mobile.
Disable font boosting on mobile. By default, when desktop mode is
enabled on mobile, mobile browsers will automagically increase the size
of text. Usually they do so poorly, making things like headers smaller
than body text, which breaks the layout.

Fixes regression in d6b1302e0.
2022-01-22 21:33:34 -06:00
evazion
893fa1e948 css: fix <input> margins.
Fix regression in d6b1302e0. Fixes the related tags list having extra
space between tags because of default margins around the <input> checkboxes.
2022-01-22 18:29:33 -06:00
evazion
d2a24e6b10 Fix #4971: NoMethodError when trying to display some modreports.
Delete modreports for hard-deleted comments. There were a total of six
invalid modreports for deleted comments.
2022-01-22 18:12:07 -06:00
evazion
90be15e0b5 Fix #4973: Wiki pages json index returns 404.
Fix regression introduced in 0db20e0ca. Setting `format: false` on the
wiki pages resource disabled format negotiation on all wiki page routes,
not just the show page, which meant /wiki_pages.json no longer worked.

The fix to monkey patch the internal Rails method that parses the file
extension from the URL, and have it ignore everything but the .html,
.json, .js, and .xml extensions. This is really hacky and may break in
future Rails releases.
2022-01-22 16:52:20 -06:00
evazion
a4279ceff2 css: hide "*" next to required form fields.
Fix regression in d6b1302e.
2022-01-22 00:08:14 -06:00
evazion
56722df753 forum: delete posts when topic is deleted.
Fix it so that when a forum topic is deleted, all posts in the topic are
deleted too. Also make it so that when a forum topic is undeleted, all
posts in it are undeleted too.

Before when a topic was deleted, only the topic itself was marked as
deleted, not the posts inside the topic. This meant that when a spam
topic was deleted, the OP wouldn't be marked as deleted, so any
modreports against it wouldn't be marked as handled.

Also change it so that it's not possible to undelete a post in a deleted
topic, or to delete the OP of a topic without deleting the topic itself.

Finally, add a fix script to delete all active posts in deleted topics,
and to undelete all deleted OPs in active topics.
2022-01-21 22:35:20 -06:00
evazion
befdb87bd5 Merge pull request #4970 from GiantFrog/master
Fix bug with configurable upload limits
2022-01-21 18:40:21 -06:00
GiantFrog
d69256cf75 Fix bug where env variables need to be ints 2022-01-21 16:55:08 -07:00
evazion
2fff563027 css: fix <li> line-height. 2022-01-20 21:50:48 -06:00
evazion
8d9cee5ad0 modreports: don't allow reporting content more than 1 year old.
Don't allow users to report comments, forum posts, or dmails that are
more than 1 year old.
2022-01-20 21:50:31 -06:00
evazion
5fd0d498a4 modreports: log modaction when report is handled or rejected. 2022-01-20 21:28:29 -06:00
evazion
c8d27c2719 Fix #4669: Track moderation report status.
* Add ability to mark moderation reports as 'handled' or 'rejected'.
* Automatically mark reports as handled when the comment or forum post
  is deleted.
* Send a dmail to the reporter when their report is handled.
* Don't show the report notice on comments or forum posts when all
  reports against it have been handled or rejected.
* Add a fix script to mark all existing reports for deleted comments,
  forum posts, or dmails as handled.
2022-01-20 20:50:23 -06:00
evazion
98aee048f2 artists: fix old artists with invalid names.
There are a lot of old artist entries with Japanese names. These names
are now invalid and these artist entries can't be edited because they
fail validation checks.

Add a fix script to delete all artist entries with non-ASCII names, and
rename them to `artist_1234`.
2022-01-20 16:01:31 -06:00
evazion
57a7b19180 css: fix <ul>, <menu> margins. 2022-01-20 01:43:50 -06:00
evazion
02c9498860 artists: normalize group names.
Normalize artist group names following the same rules as artist other names.

This means artist group names now use underscores instead of spaces.
It also means extra space characters at the beginning and end of names
is stripped, and Unicode characters are normalized.

Fixes #4647, which was caused by users accidentally replacing group
names with a single space character when trying to remove a group.
2022-01-20 00:17:06 -06:00
evazion
0376765847 views: standardize timestamp format across history pages.
Standardize the timestamp format across various history pages to look like this:

    DanbooruBot »
    2022-01-19 23:28
2022-01-19 23:33:06 -06:00
evazion
00f4fbe2d1 artists: don't show unchanged URLs in artist history.
Only show changed URLs in the artist history, not unchanged URLs. Makes
the edit history more compact and easier to read by showing only the
things that changed on every edit.
2022-01-19 23:24:20 -06:00
evazion
b9d4be16c8 Fix #4609: Searching for users failed flags fails. 2022-01-19 22:56:51 -06:00
evazion
0db20e0cab Fix #4591: Wiki pages with filename-like name are broken by default.
Fix wiki pages like this returning 406 errors:

* https://danbooru.donmai.us/wiki_pages/rnd.jpg

Caused by Rails parsing the .jpg part as a file extension and trying to
return a JPEG in response. This happens deep in Rails' MIME negotiation
code, so it's hard to override. The fix is to pass `format: false` in
the route to disable all special handling of file extensions by Rails,
and then handle it ourselves in the controller. Ugly.

This only affected two tags: `rnd.jpg` and `haru.jpg`.
2022-01-19 21:44:40 -06:00
evazion
d6b1302e0b css: clean up css reset stylesheet.
Remove all unnecessary rules from our CSS reset stylesheet.

Our CSS reset was based on a combination of Eric Meyer's CSS reset [1]
and Nicholas Gallagher's normalize.css [2]. Neither had been updated in
over 10 years, so they both contained a lot of unnecessary cruft. This
included workarounds for bugs in ancient versions of IE, and rules for
elements we don't use. Some of these rules had already been removed
upstream, but we never synchronized our copy of normalize.css with it.

This fixes some minor issues with vertical alignment of form inputs,
caused by unnecessary `vertical-align: middle` rules.

[1]: https://meyerweb.com/eric/tools/css/reset/
[2]: https://nicolasgallagher.com/about-normalize-css/
2022-01-19 18:05:36 -06:00
evazion
f39b0bacf4 dtext: adjust space between headers and paragraphs.
Adjust heading tags (h1, h2, h3, h4, h5, h6) to use margins instead of
padding so that margins collapse together. Adjust margin sizes to reduce
gaps between headers and paragraphs, and headers and subheaders.

Also adjust paragraph margins so that there's slightly less space
between paragraphs.
2022-01-19 17:02:01 -06:00
evazion
ae11a59c37 Merge pull request #4964 from GiantFrog/master
Add configurable upload limits
2022-01-19 00:11:06 -06:00
GiantFrog
b1b706aaff Add configurable upload limits 2022-01-17 13:28:24 -07:00
evazion
d0ce511465 css: fix stylelint errors. 2022-01-17 11:58:19 -06:00
evazion
54dbf1a436 Update Ruby gems and Yarn packages. 2022-01-17 10:49:04 -06:00
evazion
35b03799a5 jobs: increase job timeout to 24 hours.
Should fix BigQuery exports failing for the posts table. Apparently even
an 8 hour timeout wasn't enough.
2022-01-16 15:14:48 -06:00
evazion
acf565be7b Fix #4678: Validate custom CSS.
* Make it an error to add invalid custom CSS to your account.
* Add a fix script to remove custom CSS from all accounts with invalid CSS.
2022-01-15 23:20:49 -06:00
evazion
c455c08b2c tests: fix broken tests.
Fixups for c3c4f5a2a.
2022-01-15 22:02:47 -06:00
evazion
8a4faf7741 Fix #4960: Cosplay wikis link to char:x instead of just x. 2022-01-15 21:30:32 -06:00
evazion
5bec0aa147 Fix #4629: Tag renames/aliases should also modify pool descriptions. 2022-01-15 20:34:08 -06:00
evazion
33103f6dc4 pools: add ability to search for pools linking to given tag.
Add ability to search for pools linking to a given tag in the pool
description. Example:

    https://danbooru.donmai.us/pools?search[linked_to]=touhou

(This isn't actually exposed in the UI to avoid cluttering the pool
search form with rarely used options.)

Pools with broken links can be found here:

    https://danbooru.donmai.us/dtext_links?search[has_linked_tag]=No&search[has_linked_wiki]=No&search[model_type]=Pool

Lays the groundwork for fixing #4629.
2022-01-15 20:26:30 -06:00
evazion
c3c4f5a2a7 Fix #4957: Autotag non-web_source.
Autotag non-web_source on posts that have a non-http:// or https:// URL.
Add a fix script to backfill old posts.

Syntactically invalid URLs are still considered web sources. For
example, `https://google,com` technically isn't a valid URL, but it's
not considered a non-web source.
2022-01-14 22:58:27 -06:00
evazion
adc5bbf906 posts: fix paginator not appearing when all posts are hidden.
Fix the paginator not appearing when all posts on the page are hidden,
because of deleted posts, banned artists, censored tags, or non-safe
posts in safe mode. This prevented navigating to the next or previous
page.
2022-01-14 22:07:38 -06:00