Lock the post during replacement to ensure we have the latest version of
the tags and to ensure nobody else can modify the post until after the
replacement is finished.
Perform the replacement in a before_create callback so that it runs in a
transaction and if it fails, the transaction will rollback and the
replacement record won't be created.
Doing the replacement in a transaction isn't great because, for one
thing, it could hold the transaction open a long time, which isn't good
for the database. And two, if the transaction rolls back, the database
changes will be undone, but if the replacement file has already been saved
to disk, then it won't be undone, which could result in a dangling file.
* Move replacement tests from test/unit/upload_service_test.rb to
test/functional/post_replacement_controller_test.rb
* Move UploadService::Replacer to PostReplacementProcessor.
* Fix a minor bug where if you used the API to replace a post with a file,
the replacement would fail unless you passed an empty string for the
replacement_url.
Don't delete replaced files after 30 days. There are only about 30k
replacements in total, so the cost of keeping replaced files is
negligible. It was also wrong because the media asset wasn't destroyed
too, so there were active media assets with missing files.
Rename the following post replacement attributes:
* file_size_was -> old_file_size
* file_ext_was -> old_file_ext
* image_width_was -> old_image_width
* image_height_was -> old_image_height
* md5_was -> old_md5
In Rails 6.1, having attributes named `file_size` and `file_size_was` on
the same model breaks things because it conflicts with Rails' dirty
attribute tracking.
- The only string works much the same as before with its comma separation
-- Nested includes are indicated with square brackets "[ ]"
-- The nested include is the value immediately preceding the square brackets
-- The only string is the comma separated string inside those brackets
- Default includes are split between format types when necessary
-- This prevents unnecessary includes from being added on page load
- Available includes are those items which are allowed to be accessible to the user
-- Some aren't because they are sensitive, such as the creator of a flag
-- Some aren't because the number of associated items is too large
- The amount of times the same model can be included to prevent recursions
-- One exception is the root model may include the same model once
--- e.g. the user model can include the inviter which is also the user model
-- Another exception is if the include is a has_many association
--- e.g. artist urls can include the artist, and then artist urls again
Refactor various post_tag_match methods to use subqueries instead of joins.
This simplifies things inside PostQueryBuilder, since now we can assume
we're always dealing with a Post relation, rather than some other table
joined with the posts table.
* Allow using ApplicationRecord#attribute_matches to search text attributes,
and standardize models on using this instead of duplicating code.
* Remove restrictions that limited wildcard searches to Builders only in various places.
Fail loudly if we forget to whitelist a param instead of silently
ignoring it.
misc models: convert to strong params.
artist commentaries: convert to strong params.
* Disallow changing or setting post_id to a nonexistent post.
artists: convert to strong params.
* Disallow setting `is_banned` in create/update actions. Changing it
this way instead of with the ban/unban actions would leave the artist in
a partially banned state.
bans: convert to strong params.
* Disallow changing the user_id after the ban has been created.
comments: convert to strong params.
favorite groups: convert to strong params.
news updates: convert to strong params.
post appeals: convert to strong params.
post flags: convert to strong params.
* Disallow users from setting the `is_deleted` / `is_resolved` flags.
ip bans: convert to strong params.
user feedbacks: convert to strong params.
* Disallow users from setting `disable_dmail_notification` when creating feedbacks.
* Disallow changing the user_id after the feedback has been created.
notes: convert to strong params.
wiki pages: convert to strong params.
* Also fix non-Builders being able to delete wiki pages.
saved searches: convert to strong params.
pools: convert to strong params.
* Disallow setting `post_count` or `is_deleted` in create/update actions.
janitor trials: convert to strong params.
post disapprovals: convert to strong params.
* Factor out quick-mod bar to shared partial.
* Fix quick-mod bar to use `Post#is_approvable?` to determine visibility
of Approve button.
dmail filters: convert to strong params.
password resets: convert to strong params.
user name change requests: convert to strong params.
posts: convert to strong params.
users: convert to strong params.
* Disallow setting password_hash, last_logged_in_at, last_forum_read_at,
has_mail, and dmail_filter_attributes[user_id].
* Remove initialize_default_image_size (dead code).
uploads: convert to strong params.
* Remove `initialize_status` because status already defaults to pending
in the database.
tag aliases/implications: convert to strong params.
tags: convert to strong params.
forum posts: convert to strong params.
* Disallow changing the topic_id after creating the post.
* Disallow setting is_deleted (destroy/undelete actions should be used instead).
* Remove is_sticky / is_locked (nonexistent attributes).
forum topics: convert to strong params.
* merges https://github.com/evazion/danbooru/tree/wip-rails-5.1
* lock pg gem to 0.21 (1.0.0 is incompatible with rails 5.1.4)
* switch to factorybot and change all references
Co-authored-by: r888888888 <r888888888@gmail.com>
Co-authored-by: evazion <noizave@gmail.com>
add diffs
* Perform backups synchronously inside `distribute_files` instead of
asynchronously in `queue_backup`. Asynchronous backups assumed that
files are stored on the local filesystem, which isn't true in general.
* Remove obsolete backup service classes.
Refactors the upload process to pass around temp files, rather than
passing around file paths and directly writing output to the local
filesystem. This way we can pass the storage manager the preview /
sample / original temp files, so it can deal with storage itself.
* Change Download::File#download! to return a temp file.
* Change DanbooruImageResizer and PixivUgoiraConverter to accept/return
temp files instead of file paths.
* Change Upload#generate_resizes to return temp files for previews and samples.
* Change Upload#generate_resizes to generate ugoira .webm samples
synchronously instead of asynchronously.
* Allow every controller to take the `search[id]` param.
* Parse the `search[id]` param the same way that the `id:<N>` metatag is
parsed. So `search[id]=1,2,3`, `search[id]=<42`, `search[id]=1..10`, for
example, are all accepted.
When replacing a post with the same file, don't queue a deletion and
don't leave a comment or modaction. There's no need to do these things
when we're restoring a missing or corrupted image with the original file.
* Adds a "Tags" field to the post replacement dialog box. The given tags
are added to the post after replacement.
* Prefills the Tags field with certain tags that usually need to be
removed after replacement: replaceme, image_sample, jpeg_artifacts, etc.
* Record "file://#{filename}" as the replacement url when the
replacement comes from an uploaded file.
* Record the actual url downloaded by the upload process otherwise. This
may be different from the url given by the user, since the upload
process may rewrite the url.
Adds a "Final Source" field to the post replacement dialog. If
specified, the post's source field will be changed to this value after
replacement.
This makes fixing the source back to the HTML page after
replacement easier.
