Remove various associated fields that were included by default on
certain endpoints. API users can use the only param to include the
full association if they need these fields.
* /artists.json: urls.
* /artist_urls.json: artist.
* /comments.json: creator_name and updater_name.
* /notes.json: creator_name.
* /pools.json: creator_name.
* /posts.json: uploader_name, children_ids, pixiv_ugoira_frame_data.
* /post_appeals.json: is_resolved.
* /post_versions.json: updater_name.
* /uploads.json: uploader_name.
- The only string works much the same as before with its comma separation
-- Nested includes are indicated with square brackets "[ ]"
-- The nested include is the value immediately preceding the square brackets
-- The only string is the comma separated string inside those brackets
- Default includes are split between format types when necessary
-- This prevents unnecessary includes from being added on page load
- Available includes are those items which are allowed to be accessible to the user
-- Some aren't because they are sensitive, such as the creator of a flag
-- Some aren't because the number of associated items is too large
- The amount of times the same model can be included to prevent recursions
-- One exception is the root model may include the same model once
--- e.g. the user model can include the inviter which is also the user model
-- Another exception is if the include is a has_many association
--- e.g. artist urls can include the artist, and then artist urls again
Fix a couple security issues related to dmail permalinks. Dmails have a
permalink that you can give to a Mod to let them read the dmail. This is
done with a key param that grants access when the dmail is opened by
another user. The key param had several problems:
* The key contained a full copy of the message's title and body encoded in
base64. This meant that anyone given a dmail permalink could read the
full dmail just by decoding the key in the link, without even having
to open the link.
* The key was derived from the dmail's title and body. If you knew or
could guess a dmail's title and body you could open the dmail. One
case when this was possible was when sending dmails. You could send
someone a dmail, take the permalink from your sent copy of the dmail,
then increment the dmail id to open the receiver's copy of the dmail.
Since the sent copy and the received copy both had the same title and
body, they both had the same dmail key. This let you check whether a
person had read your dmail, and what time they read it at.
* The key verification was done with an insecure string comparison
rather than a secure constant-time comparison. This was potentially
vulnerable to timing attacks.
* Opening a dmail belonging to another user would mark it as read for them.
The fix to all this is to use the dmail's id as the key instead of the
dmail's title and body. This means that old permalinks no longer work.
This is unavoidable given the issues above.
Other changes:
* The name of the 'Permalink' link is now 'Share'.
* Anyone with the 'Share' link can view the dmail, not just Mods.
* Add ability to mark dmails as unread.
* Fix users.unread_dmail_count to not count deleted dmails.
* Fix show action so that API calls don't mark dmails as read.
* Don't show the unread dmail notice on the /dmails page itself.
* Stop using users.has_mail flag.
* Add unread and deleted dmail folders.
* Remove dmail_folder cookie (wasn't used).
* Default to the received folder so that we don't show sent messages by default.
Turn deletions into soft deletions (set the is_deleted flag) instead of
hard deletions (remove from database). The is_deleted flag actually
already existed, but it was never used before.
Also remove options to configure names of wiki notice pages. These names
generally don't need to be changed and we already hardcode links to wiki
pages in other places anyway.
* Add ability to report dmails.
* Enable reports for comments, forum posts, and dmails.
* Allow Members to send reports.
* Don't allow users to report the same thing twice.
Avoid doing one SQL query per topic when checking for new topics on the
forum index.
This also changes it so that forum topics aren't always marked as new
for anonymous users.
* Rarely used (only used ~15 times in total, not used at all since 2015-2016).
* Merging topics didn't properly bump the new topic.
* Merging topics didn't log a modaction when the old topic was deleted.
* Merging topics broke the old topic. Moving all the posts from one topic
to another leaves the old topic with zero posts. This normally can't
happen and it causes exceptions when you try to view the empty topic.
* It was technically possible to merge a topic with itself. This would
break the response_count.
* It was technically possible for a mod to merge a topic into an
admin-only topic.
Few people used forum subscriptions (only around 100), and even fewer
people were subscribed to active threads. Most subscriptions were for
old threads that will never be bumped again. The implementation also had
a few problems:
* Unsubscribe links in emails didn't work (they unset the user's
receive_email_notifications flag, but forum subscriptions didn't
respect this flag).
* Some users had invalid email addresses, which caused notifications to
bounce. There was no mechanism for preventing bounces.
* The implementation wasn't scalable. It involved a daily linear scan
over _all_ forum subscriptions looking for any topics that had been updated.
Few people used dmail filters (~900 users in 5 years) and even fewer
used them correctly. Most people used them to try to block dmail spam,
but usually they either blocked too much (by adding common words that
are present in nearly all dmails, causing all mails to them to be
filtered) or too little (blocking specific email addresses or urls,
which usually are never seen again after the spammer is banned).
Nowadays the spam detection system does a better job of filtering spam.
The belongs_to_creator macro was used to initialize the creator_id field
to the CurrentUser. This made tests complicated because it meant you had
to create and set the current user every time you wanted to create an
object, when lead to the current user being set over and over again. It
also meant you had to constantly be aware of what the CurrentUser was in
many different contexts, which was often confusing. Setting creators
explicitly simplifies everything greatly.
- Limited to Builders+
-- Moderator+ can also use as they may be too busy ATM
- Only on users, comments, and forum posts
- Multiple reports can be generated per instance
- Primarily posts to a moderator-only topic for viewability
- Secondarily has a table for searchability
-- Viewable only by moderators
* Rename 'privacy mode' to 'private favorites'.
* Make the private favorites setting only hide favorites, not favgroups
and not the user's uploads on their profile page.
* Make the favgroup is_public flag default to true instead of false and
fix existing favgroups to be public if the user didn't have privacy mode
enabled before.
* List _all_ public favgroups on the /favorite_groups index, not just
favgroups belonging to the current user.
* Add a /users/<id>/favorite_groups endpoint.
