Remove various associated fields that were included by default on
certain endpoints. API users can use the only param to include the
full association if they need these fields.
* /artists.json: urls.
* /artist_urls.json: artist.
* /comments.json: creator_name and updater_name.
* /notes.json: creator_name.
* /pools.json: creator_name.
* /posts.json: uploader_name, children_ids, pixiv_ugoira_frame_data.
* /post_appeals.json: is_resolved.
* /post_versions.json: updater_name.
* /uploads.json: uploader_name.
- The only string works much the same as before with its comma separation
-- Nested includes are indicated with square brackets "[ ]"
-- The nested include is the value immediately preceding the square brackets
-- The only string is the comma separated string inside those brackets
- Default includes are split between format types when necessary
-- This prevents unnecessary includes from being added on page load
- Available includes are those items which are allowed to be accessible to the user
-- Some aren't because they are sensitive, such as the creator of a flag
-- Some aren't because the number of associated items is too large
- The amount of times the same model can be included to prevent recursions
-- One exception is the root model may include the same model once
--- e.g. the user model can include the inviter which is also the user model
-- Another exception is if the include is a has_many association
--- e.g. artist urls can include the artist, and then artist urls again
* Add ability to report dmails.
* Enable reports for comments, forum posts, and dmails.
* Allow Members to send reports.
* Don't allow users to report the same thing twice.
The belongs_to_creator macro was used to initialize the creator_id field
to the CurrentUser. This made tests complicated because it meant you had
to create and set the current user every time you wanted to create an
object, when lead to the current user being set over and over again. It
also meant you had to constantly be aware of what the CurrentUser was in
many different contexts, which was often confusing. Setting creators
explicitly simplifies everything greatly.
- Posts and topics have an added moderation_reports function
-- This is so all moderation reports can be loaded in a single query
- Those moderation reports are passed into the render functions separately
-- This is so the individual comments/posts don't have to be queried
Unify the `name_to_id`, `named`, and `find_by_name` methods into a
single `find_by_name_or_id` method that has consistent behavior in how
names are normalized.
* Convert notices from helpers to partials.
* Eliminate PostSets::PostRelationship class in favor of post_sets/posts template.
* Eliminate COUNT(*) queries when calculating the number of child posts.
* Eliminate redundant parent load and parent exists queries.
Fix regression caused by 3d3f61559. This is what happened:
* The posts controller calls post.visible? before post.update.
* post.visible? calls post.tag_array.
* The call to tag_array causes the current value of tag_string to be
cached in @tag_array.
* post.update calls post.merge_old_changes.
* post.merge_old_changes accesses tag_array, which contains the old
cached version of tag_string rather than the new version from the
update.
* post.merge_old_changes detects no changes, so the update does nothing.
* This only happens for Members because for Gold+ users the call to
post.visible? short circuits before accessing tag_array.
Moral of the story: cache invalidation is hard. Don't cache unless you have to.
Bug: Post.fast_count failed when Danbooru.config.estimate_post_counts
was enabled but the database didn't have any posts. This normally
happened only during testing.
* Switch CloudflareService from HttpartyCache to Danbooru::Http.
* Purge cached urls from Cloudflare when a post is replaced and the md5
doesn't change. This happens when a corrupted image is replaced or
thumbnails are regenerated. Before we purged urls when a post was
expunged, which was unneeded because those urls can expire naturally.
It was also wrong because the subdomains were hardcoded, the urls used
http:// instead of https://, and we didn't account for tagged urls.
https://danbooru.donmai.us/forum_topics/9127?page=283#forum_post_160508
There was a recent outage that was caused by the read replica
(yukinoshita.donmai.us) being temporarily unavailable. The pg driver in
rails got hardstuck trying to connect to the replica, which brought down
the whole site. The app servers stopped responding and could only be
brought down with SIGKILL. Even try to boot the rails console didn't
work.
We only really used this to calculate tag counts inside Post.fast_count,
which wasn't really beneficial since the read replica is slower than the
main database.
Stop maintaining pool category pseudo tags (pool:series, pool:collection)
in pool strings. They're no longer used and the changes to the
`Post#pools` method in dc4d2e54b caused issues with this.
Also allow Members to change the category of large pools again. This was
only restricted because maintaining these pseudotags forced us to update
every post in the pool whenever a pool's category was changed.
Replace the `method_attributes` and `hidden_attributes` methods with
`api_attributes`. `api_attributes` can be used as a class macro:
# include only the given attributes.
api_attributes :id, :created_at, :creator_name, ...
# include all default attributes plus the `creator_name` method.
api_attributes including: [:creator_name]
or as an instance method:
def api_attributes
[:id, :created_at, :creator_name, ...]
end
By default, all attributes are included except for IP addresses and
tsvector columns.
Calling set_pool_category_pseudo_tags before saving the post doesn't
work on new posts because they don't have an id yet, so trying to get
the pools by post id in `Post#pools` fails.
Stop using the pool_string field internally, but keep maintaining it
until we can drop it later.
* Stop using the pool_string for `pool:<name>` metatag searches.
* Stop using the pool_string in the `Post#pools` method. This is used to
get the list of pools on post show pages.
Revert optimization from a6163258b. Turns out that we have to resolve
aliases in fast_count, otherwise for aliased tags we'll return an empty
count.
Fixes#4156.
Certain parts of comment rendering triggered sql queries that we didn't
really need to do. Rework things to avoid this.
* Preload comment creators in order to display commenter names with link_to_user.
* Preload comment votes in order to display "undo vote" links. Only preload
votes for members since anonymous users can't vote and don't have "undo
vote" links.
* Rework various conditionals to do the filtering in Ruby so that we
avoid issuing any extra queries in sql.
* Avoid issuing any queries at all when the post doesn't have any
comments (when last_commented_at is blank).
Changes:
* Drop Users.id_to_name.
* Don't cache Users.name_to_id.
* Replace calls to name_to_id with find_by_name when possible.
* Don't autodefine creator_name in belongs_to_creator.
* Don't autodefine updater_name in belongs_to_updater.
* Instead manually define creator_name / updater_name only on models that need
to return these fields in the api.
id_to_name was cached to reduce the impact of N+1 query patterns in
certain places, especially in api responses that return creator_name /
updater_name fields. But it still meant we were doing N calls to
memcache. Using `includes` to prefetch users avoids this N+1 pattern.
name_to_id had no need be cached, it was never used in any performance-
sensitive contexts.
Avoiding caching also avoids the need to keep these caches consistent.
* Drop /posts?ro=true param (broken).
* Clean up tag_match (rescuing PG::ConnectionBad didn't do anything, we
just build the query here, we don't run it).
* Don't allow adding tags with invalid names when they already exist in
the tags table.
* If an invalid tag is added, show an warning and ignore the tag instead
of failing with a hard error.
* Move the _(cosplay) tag validation into the tag name validator.
During single-tag searches we look up the tag once in Post.fast_count
and later on when rendering the wiki excerpt. Doing a raw query here
meant that Rails couldn't cache the query, so it got ran again when
rendering the excerpt.
Post.fast_count calls Tag.normalize_query to normalize the tag string
used for the pfc cache key. This slightly improves cache sharing at the
expense of an extra query during tag searches. The extra query isn't
worth it.
