Commit Graph

9296 Commits

Author SHA1 Message Date
nonamethanks
1a66feddcf Moderation Reports: deprecate the forum topic 2022-02-22 23:47:36 +01:00
evazion
112b323f01 foundation: fix exception when uploading new Foundation url format.
Fix 'null value in column "source_url"' exception when uploading urls like this:

* https://foundation.app/@KILLERGF/kgfgen/4
* https://foundation.app/@mochiiimo/foundation/97376
2022-02-22 13:29:28 -06:00
evazion
7b009cc893 nicoseiga: fix inability to login to nicoseiga.
NicoSeiga changed it so that on every login, you must enter a 2FA code
sent by email. This broke the NicoSeiga strategy. The fix is to just use
a static session cookie instead (and hope it doesn't expire, and isn't
tied to an IP).

The `nico_seiga_login` and `nico_seiga_password` config settings have
been removed from config/danbooru_default_config.rb and replaced by
`nico_seiga_user_session`. If you run your own Danbooru instance, you
will have to update your config file manually.
2022-02-22 12:23:01 -06:00
evazion
c5777f360e artist urls: normalize trailing slashes and missing http://.
* Remove unnecessary trailing slashes when artist URLs are saved.
* Automatically add `http://` to new artist URLs if it's missing (before
  this was an error; now it's automatically fixed).
2022-02-22 00:17:53 -06:00
evazion
7d49ab6130 Add Danbooru::URL class.
Introduce a Danbooru::URL class for dealing with URLs. This is a wrapper
around Addressable::URI that adds some additional helper methods. Most
significantly, the `parse` method only allows valid http/https URLs, and
it returns nil instead of raising an exception when the URL is invalid.
2022-02-22 00:17:53 -06:00
evazion
60a26af6e3 rails: add 'URL' inflection.
Make it so we can write `ArtistURL` instead of `ArtistUrl`.
2022-02-22 00:17:53 -06:00
evazion
fbab273c81 Upgrade http.rb gem to 5.0.4.
Fixes a bug where the Foundation source strategy failed because http.rb
automatically sent a `Content-Length: 0` header with all GET requests,
which caused Foundation to return a 400 Bad Request error. This behavior
was fixed in http.rb 5.x.

http.rb 5.x has a breaking change where it now includes the request object
inside the response object, which we have to handle in a few places.
2022-02-22 00:17:05 -06:00
evazion
35dca33abc uploads: remove batch upload link from navbar. 2022-02-21 00:29:46 -06:00
evazion
68ba447494 uploads: remove batch upload page.
* Make /uploads/batch redirect to /uploads/new.
* Remove /uploads/image_proxy.
2022-02-21 00:03:43 -06:00
evazion
3da9c1574f Merge pull request #5010 from nonamethanks/add-mod-report-link
Moderation Reports: add link for moderators to navbar
2022-02-20 22:45:30 -06:00
evazion
a916fd9e7b Fix #5015: 'Rating not selected' error leads to an empty page. 2022-02-19 17:25:14 -06:00
evazion
202dfe5d87 uploads: allow uploading multiple files from your computer at once.
Allow uploading multiple files from your computer at once.

The maximum limit is 100 files at once. There is still a 50MB size limit
that applies to the whole upload. This limit is at the Nginx level.

The upload widget no longer shows a thumbnail preview of the uploaded
file. This is because there isn't room for it in a multi-file upload,
and because the next page will show a preview anyway after the files are
uploaded.

Direct file uploads are processed synchronously, so they may be slow.

API change: the `POST /uploads` endpoint now expects the param to be
`upload[files][]`, not `upload[file]`.
2022-02-19 00:00:56 -06:00
evazion
e37dd3a6d0 uploads: change loading indicator for thumbnails.
Use a spinner icon instead of the word "Loading" for thumbnails that are
being processed in the background in a batch upload.

Also use morphdom to update thumbnails so we only update the parts of
the DOM that actually changed.
2022-02-18 16:03:43 -06:00
evazion
a1d2572bad uploads: add data attributes to thumbnail html.
Add data attributes to thumbnails on the /uploads, /upload_media_assets,
and /media_assets pages. Add a `data-is-posted` attribute for styling
thumbnails based on whether they've already been posted.
2022-02-18 15:44:12 -06:00
evazion
fc5aec7de0 media assets: optimize /media_assets?search[is_posted] query.
Followup to 093a808a3. Using a NOT EXISTS clause is much faster than the
`LEFT OUTER JOIN posts WHERE posts.id IS NULL` clause generated by
`.where.missing(:post)`.
2022-02-18 04:24:33 -06:00
evazion
77515915a4 uploads: change loading indicator for upload widget.
Change the loading indicator from a progress bar to a spinner. Fixes
issue with the <progress> element having a different appearance on
different browsers.
2022-02-18 03:39:08 -06:00
evazion
093a808a36 Fix #4986: Add ability to filter images in /media_assets and /uploads depending on if they have become posts 2022-02-18 03:39:08 -06:00
evazion
f54fac74c8 Add image icon next to thumbnail size dropdown menu. 2022-02-17 20:31:59 -06:00
evazion
0b8d042c06 uploads: allow changing thumbnail size on My Uploads / All Uploads pages.
* Add a "Size" menu to the My Uploads / All Uploads pages to allow
  changing the thumbnail size.
* Make the My Uploads / All Uploads pages use the same thumbnail size as
  the post index page.
* Change the "Gallery | Table" links on the My Uploads page to icons.
2022-02-16 16:39:28 -06:00
evazion
f890e8595e uploads: respect user's posts-per-page setting on My Uploads page. 2022-02-16 16:38:47 -06:00
evazion
7c3586f985 uploads: make gallery mode the default on the My Uploads page. 2022-02-16 16:38:47 -06:00
evazion
c7c326b552 uploads: fix exception on My Uploads page when upload is in pending state.
Fix an exception when an upload is in the pending state and it doesn't
have any upload media assets yet.
2022-02-16 16:38:47 -06:00
evazion
049750e512 uploads: fix My Uploads page showing Admins uploads for other users.
Fix the "My Uploads" page showing Admins all uploads, not just their own
uploads.

Changes the URL of the My Uploads page from /uploads to /users/:id/uploads.
2022-02-16 14:11:40 -06:00
evazion
70f8d3d0c0 uploads: autorefresh /uploads/:id page if upload is still processing.
Fixes an issue where if you were uploading a multi-image source, and you
clicked on a thumbnail that was still processing, then the page wouldn't
refresh when the processing was complete.
2022-02-16 01:18:37 -06:00
evazion
9a5a04d74e nijie: fix uploads not working for new image URL format.
Fix uploads not working for image URLs like this:

    https://pic.nijie.net/07/nijie/17/95/728995/illust/0_0_403fdd541191110c_c25585.jpg
2022-02-15 20:45:28 -06:00
evazion
a14cd3cd8b uploads: increase upload status polling frequency. 2022-02-15 20:45:28 -06:00
evazion
9114161bca uploads: remove obsolete attributes from permitted_attributes. 2022-02-15 20:45:28 -06:00
evazion
7cfbd891ae pixiv: avoid unnecessary API call when uploading Pixiv posts.
Do one less API call when fetching the image URLs for a Pixiv post. The
`is_ugoira?` check in `image_urls` caused us to do an extra API call
when fetching the image URLs for a non-ugoira post.

API calls to Pixiv take around ~800ms, so this reduces minimum upload
time for Pixiv posts from ~1.6 seconds (two calls) to ~0.8 seconds.
2022-02-15 18:55:12 -06:00
evazion
ccaf5398f6 nico seiga: fix error when login fails.
Fix an undefined variable exception being thrown when login to NicoSeiga
failed (the `url` variable wasn't defined).
2022-02-15 18:55:12 -06:00
evazion
6b56b6a122 uploads: fix error when source doesn't have any images.
Fix an error when trying to upload a source that doesn't have any
images, for example a Twitter post with no images.
2022-02-15 18:55:12 -06:00
evazion
e4d7453180 uploads: improve error messages.
Improve upload error messages when downloading an URL fails, or it isn't
an image or video file.
2022-02-15 18:54:55 -06:00
evazion
87a00a1182 uploads: fix "ArgumentError: string contains null byte" error
Fix an error when trying to upload a file larger than the file size
limit. In this case we tried to dump the whole HTTP response into the
error message, which included the binary file itself, which caused this
exception because it contained null bytes.
2022-02-15 18:16:47 -06:00
evazion
16b8d4b607 uploads: consider uploads as failed when all assets fail.
Make the "completed" status for an upload mean "at least one file in the
upload successfully completed". The "error" status means "all files in
the upload failed".

This means that when an upload has multiple assets and some succeed and
some fail, the whole upload is considered completed. This can happen
when uploading multiple files and some files are over the size limit,
for example. The upload is considered failed only if all files in the
upload fail.

This fixes an issue where, if uploading a single file and that file
failed because it was over the size limit, then the upload wouldn't be
marked as failed.
2022-02-15 17:12:02 -06:00
evazion
b6538fde38 uploads: fix NicoSeiga sources not working.
Fix uploads for NicoSeiga sources not working because the strategy
returned URLs like the one below in the list of image_urls, which
require a login to download:

    https://seiga.nicovideo.jp/image/source/10315315

Also fix certain URLs like https://dic.nicovideo.jp/oekaki/52833.png not
working, because they didn't contain an image ID and the image_urls
method returned an empty list in this case.
2022-02-15 17:12:02 -06:00
evazion
347c37864a uploads: fix direct file uploads not working. 2022-02-15 17:12:02 -06:00
evazion
e0ed6391f5 uploads: fix related tags error on multi-file upload page.
Fix the related tags Javascript trying to run on the multi-file upload
page. It should only run on the single-file upload page.
2022-02-15 01:00:06 -06:00
evazion
36265dbff0 uploads: autorefresh thumbnails on multi-file uploads page.
Automatically refresh thumbnails on the multi-file upload page as images
are processed in the background.
2022-02-15 00:59:30 -06:00
evazion
37075988ce uploads: fix page_url for null strategy.
Fix the null source strategy setting the page URL. The page URL is
expected to be nil when we can't determine the page containing the image URL.

Fixes the upload_media_assets.page_url field being filled for uploads
from unknown sites.
2022-02-15 00:59:22 -06:00
evazion
27d71f2727 uploads: raise download timeout.
Raise the timeout for downloading files from the source to 60 seconds globally.

Previously had a lower timeout because uploads were processed in the
foreground when not using the bookmarklet, and we didn't want to tie up
Puma worker processes with slow downloads. Now that all uploads are
processed in the background, we can have a higher timeout.
2022-02-15 00:56:51 -06:00
evazion
02edb52569 uploads: enable multi-file uploads when uploading from source.
Make the upload page automatically detect when a source URL has multiple images
and let the user choose which images to post.

For example, when uploading a Twitter or Pixiv post with more than one image, we
direct the user to a page showing a thumbnail for each image and letting
them choose which ones to post.

This is similar to the batch upload page, except we actually download each image
in the background, instead of just hotlinking or proxying the thumbnails through
our servers. This avoids various problems with proxying and makes new features
possible, like showing which images in the batch have already been posted.
2022-02-14 16:13:55 -06:00
evazion
5b2fcaaafc uploads: default to only showing completed uploads.
Default to only showing completed uploads on the My Uploads page, not
failed or processing uploads.
2022-02-14 16:00:22 -06:00
evazion
2d4055ec1f uploads: show placeholder thumbnail for failed uploads. 2022-02-14 15:58:52 -06:00
nonamethanks
9ff01c1a4b Moderation Reports: add link for moderators to navbar 2022-02-14 17:22:03 +01:00
evazion
bdf83d1ffd uploads: refactor /uploads/:id page for multi-file uploads. 2022-02-14 00:41:08 -06:00
evazion
229759cc72 uploads: add /upload_media_assets index page.
This page shows each individual file you've uploaded. This is different
from the regular uploads page because files in multi-file uploads are
not grouped together.
2022-02-14 00:41:08 -06:00
evazion
53a3beee35 uploads: refactor "My Uploads" page for multi-file uploads.
* Make thumbnails on the "My Uploads" page show an icon with an image
  count when an upload contains multiple files.

* Make the "My Uploads" page show each upload, not each individual file.
  If an upload contains multiple files, they're shown grouped together
  under a single upload. This does mean that failed or duplicate uploads
  will show up on this page now. This is because this page shows each
  upload attempt, not each uniquely uploaded file.
2022-02-14 00:41:07 -06:00
evazion
879363b585 media assets: show placeholder thumbnail when image is missing.
Make media assets show a placeholder thumbnail when the image is
missing. This can happen if the upload is still processing, or if the
media asset's image was expunged, or if the asset failed during upload
(usually because of some temporary network failure when trying to
distribute thumbnails to the backend image servers).

Fixes a problem where new images on the My Uploads or All Uploads pages
could have broken thumbnails if they were still in the uploading phase.
2022-02-14 00:41:07 -06:00
evazion
eb032d54c1 uploads: set upload_media_asset.status to active.
Fix the status being set to pending instead of active for new upload
media assets.
2022-02-14 00:40:40 -06:00
evazion
26da728a07 deviant art: fix new image URLs not being recognized.
Partial fix for #5008. DeviantArt now returns https://wixmp-ed30a86b8c4ca887773594c2.wixmp.com
URLs instead of https://images-wixmp-ed30a86b8c4ca887773594c2.wixmp.com for images in the
API. Fix these URLs not being recognized by the DeviantArt strategy.
2022-02-14 00:33:50 -06:00
evazion
d7ac57f1e5 artists: sort URLs in edit form. 2022-02-14 00:33:50 -06:00