Commit Graph

11711 Commits

Author SHA1 Message Date
evazion
3d092bfd37 site map: reorganize layout.
Reorganize the /static/site_map page to try to group things together in
a more logical fashion, especially the Admin section.
2022-09-16 06:01:44 -05:00
evazion
a62e844a1a forum: fix visibility of forum post votes.
Make all forum post votes visible to everyone.

When forum votes were first introduced, it was technically possible to
vote on any forum post, including on posts in mod-only threads.
Accordingly, forum post votes were only visible if the forum post itself
was visible. However, there doesn't actually exist any votes on private
forum posts, and trying to filter them out makes the /user_actions page
much slower, so just make them visible to everyone.
2022-09-16 06:01:44 -05:00
evazion
bd73090b4c user events: make all events visible to moderators.
Allow moderators to see all events on the /user_events page. Before only
admins could see when a user changed their email, changed their
password, or had a failed login attempt. Now moderators can see these
events too.

Filtering these events out made the /user_actions page slower, and it
wasn't really necessary since merely knowing that a user changed their
email or password isn't that much more sensitive than knowing when they
logged in or out.
2022-09-16 06:01:44 -05:00
evazion
ee638f976f Add /user_actions page.
Add a /user_actions page. This page shows you a global timeline of
(almost) all activity on the site, including uploads, comments, votes,
edits, forum posts, and so on.

The main things it doesn't include are post edits, pool edits, and
favorites (posts and pools live in a separate database, and favorites
don't have the timestamps we need for ordering).

This page is useful for moderation purposes because it lets you see a
history of almost all of a user's activity on a single page.

Currently this page is mod-only. In the future it will be open to all
users, so you can view the history of your own site activity, or the
activity of others.
2022-09-16 05:39:25 -05:00
evazion
0830af49a7 db: add user_actions view.
Add a user_actions view. This view unions together a bunch of tables to
produce an event log of every action taken by a user.

Also add a bunch of indexes to make queries on this table efficient.
Even though the view is an enormous query combining together about 30
different tables, queries are very efficient as long as every table has
`created_at` and `(user_id, created)` indexes.
2022-09-16 04:20:19 -05:00
evazion
abf493794f twitter: fix misparsing of https://twitter.com/i/status/:id urls.
Fix URLs like `https://twitter.com/i/status/943446161586733056` parsing
the username as `i`. This led to the new artist page recommending the
tag name `i` when creating an artist for a source like this.

Also fix these URLs not being normalized to `https://twitter.com/:username/status/:id` after upload.
2022-09-15 19:57:12 -05:00
evazion
cfe567b649 uploads: fix exception in UploadMediaAsset.visible.
Fix `UploadMediaAsset.visible(user).count` failing when we weren't
joined on the uploads table.
2022-09-15 19:19:44 -05:00
evazion
0a5ebcc69d uploads: refactor media asset validation logic.
Refactor the upload validation logic to not depend on the current user.
Fixes several broken upload tests.
2022-09-15 05:09:07 -05:00
evazion
bfe2eabc6d db: change ids from bigint to integer on various tables.
Change ID columns from `bigint` (64-bits) to `integer` (32-bits) on various tables.

Rails 6.0 switched the default from bigint to integer for IDs on new
tables, so now we have a mix of tables with integer IDs and bigint IDs.
Switch back to integer IDs on certain tables because we're going to
build a view that unions a bunch of tables together to build a user
activity timeline, and for this purpose all the tables need to have IDs
of the same type in order for Postgres to optimize the query effectively.
2022-09-15 03:47:05 -05:00
evazion
86b2d25f21 views: fix search forms to retain limit param.
Fix search forms to retain the `limit` URL param if it was manually given in the URL.
2022-09-15 03:47:05 -05:00
evazion
25739d9e29 Merge pull request #5226 from kidonng/favorite-groups
Add keyboard shortcuts for favorite groups
2022-09-15 03:46:50 -05:00
evazion
9e16de13ef Merge pull request #5220 from nonamethanks/duration-validation
Uploads: allow admins to bypass duration limits again
2022-09-15 03:46:21 -05:00
evazion
04f98d3b8c Merge pull request #5246 from NamelessContributor/fix-long-tag-overflow
Fix some long tags / wiki links overflowing their container
2022-09-15 03:44:15 -05:00
evazion
ab900beffd Merge pull request #5250 from jwood7423/Add-api-to-the-list-of-meta-wikis
Added `api:` to the list of META_WIKIS
2022-09-15 03:43:42 -05:00
evazion
f504f752d0 Merge pull request #5243 from nonamethanks/remove-self-upload-from-highlights
Modqueue: remove self_upload from highlighted tags
2022-09-15 03:42:17 -05:00
nonamethanks
425a905b83 tests: update tumblr tests 2022-09-15 09:48:28 +02:00
evazion
e2a3265daf mod dashboard: remove ip address search.
Remove the IP address search option from the /moderator/dashboard page.
This was an obsolete way of searching for sockpuppet accounts by IP.
The /user_events page should be used instead.
2022-09-13 00:14:19 -05:00
evazion
fb980d4a16 notes: merge versions when note is deleted.
Unlike other models, notes had a special rule where if you deleted or
undeleted a note, it would always create a new version instead of
merging it into the previous version. Remove this rule since it didn't
have a purpose and it was inconsistent with other versioned models.
2022-09-12 22:10:14 -05:00
evazion
0267e4e3d7 wikis: fix exception when wiki doesn't have a tag.
Fix a nil reference exception in the sidebar when a wiki page doesn't
belong to a tag.

Also hide the options sidebar on the new wiki page since none of the
options are relevant when creating a new wiki.
2022-09-12 21:24:03 -05:00
evazion
e1fb219738 aliases: retire artist aliases after 2 years.
Automatically retire artist aliases after they're more than 2 years old.
Before the rule was that artist aliases were only retired if they didn't
have any new posts in the last 2 years.
2022-09-12 02:31:12 -05:00
evazion
2c37fdf9e8 tags: don't create mod action when tag is deprecated.
Not needed anymore since deprecations are now tracked in the tag history.
2022-09-12 02:06:16 -05:00
evazion
600e75e324 /tags: update tag actions menu.
Replace the "Edit | History | Related | Similar" links on the /tags
index with a popup menu containing more actions, including links to the
tag's version history, links to the tag's aliases and implications,
links to comments and forum discussions mentioning the tag, and links to
find untagged and mistagged posts using AI tags.
2022-09-12 00:28:26 -05:00
evazion
bb728ecebf tags: add /tag_versions page. 2022-09-11 18:41:16 -05:00
evazion
54a45a3021 tags: track tag histories.
Track the history of the tag `category` and `is_deprecated` fields in
the `tag_versions` table.

Adds generic Versionable and VersionFor concerns that encapsulate most
of the history tracking logic. These concerns are designed to make it
easy to add history to any model.

There are a couple notable differences between tag versions and other versions:

* There is no 1 hour edit merge window. All changes to the `category`
  and `is_deprecated` fields produce a new version in the tag history.

* New versions aren't created when a tag is created. Versions are only
  created when a tag is edited for the first time. The tag's initial
  version isn't created until *after* the tag is edited for the first time.

For example, if you change the category of a tag that was last updated
10 years ago, that will create an initial version of the tag backdated
to 10 years ago, plus a new version for your edit.

This is for a few reasons:

* So that we don't have to create new tag versions every time a new tag
  is created. This would be wasteful because most tags never have their
  category or deprecation status change.
* So that if you make a typo tag, your name isn't recorded in the tag's
  history forever.
* So that we can create new tags in various places without having to know
  who created the tag (which may be unknown if the current user isn't set).
* Because we don't know the full history of most tags, so we have to
  deal with incomplete histories anyway.

This has a few important consequences:

* Most tags won't have any tag versions. They only gain tag versions if
  they're edited.
* You can't track /tag_versions to see newly created tags. It only
  shows changes to already existing tags.
* Tag version IDs won't be in strict chronological order. Higher IDs may
  have created_at timestamps before lower IDs. For example, if you
  change the category of a tag that is 10 years old, that will create an
  initial version with a high ID, but with a created_at timestamp dated
  to 10 years ago.

Fixes #4402: Track tag category changes
2022-09-11 17:47:44 -05:00
evazion
0c327a2228 tags: add tag_versions table.
Add a tag_versions table for tracking the history of tags.

A couple notable differences from other version tables:

* There is a previous_version_id column that points to the previous
  version. This allows finding the first, last, previous, or next
  version efficiently in SQL.

* There is a `version` column that tracks the revision number (1, 2, 3, etc).
  Post versions and note versions have this, but other version tables don't.

* The `updater_id` column is optional. This is because we don't know who
  the last updater was before we started tracking the history of tags,
  so the initial updater will be NULL in the first version of the tag.
2022-09-11 17:35:53 -05:00
jwood7423
f55c4525e2 Added api: to the list of META_WIKIS
As of right now, api wiki pages are categorized as "general" when they should be categorized as "meta".
2022-09-11 19:34:13 +01:00
evazion
10cb97dbd5 Fix #5200: non-web_source, bad_source, etc. not removed when using source: metatag 2022-09-11 03:03:57 -05:00
evazion
2eead46ad4 tags: remove dead code. 2022-09-10 14:39:17 -05:00
evazion
22bfa44183 posts: fix exception when tagging post with char:copy:foo.
Fixup for 015c6dc7d. Show a warning about failure to add a tag instead
of raising an exception when trying to tag a post with `char:copy:foo`.
This tries to create a tag named `copy:foo` then set the category to
character, which doesn't work because `copy:foo` isn't a valid tag name.
2022-09-10 14:39:17 -05:00
evazion
015c6dc7db Fix #4965: Account for metatag prefixes when searching/linking
Drop the ability to write e.g. `create alias foo -> char:bar` in a BUR
to change the tag's type as a side effect. You can only use these
tag type prefixes in tag edits now.

This feature was only intended to be used in tag edits. The fact it
worked elsewhere was unintended behavior.

This feature was problematic because it relied on `Tag.find_or_create_by_name`
automagically changing the tag's category when the tag name contained a
tag category prefix, e.g. `char:hatsune_miku`. This meant that merely
looking up a tag could have the side effect of changing its category.
It was also bad because `find_or_create_by_name` had a hidden dependency
on the current user, which may not be set or available in all contexts.
2022-09-10 04:49:24 -05:00
evazion
f36f1ff37b tags: drop is_locked column.
This column was deprecated in 208b6189. Finish removing it.
2022-09-09 15:58:48 -05:00
evazion
c96bdd1766 autocomplete: fix ranking of exact matches.
Fix a bug where searching for `sakana~` ranked `sakana~_(meme)` beneath
random artist tags containing the word `sakana`. Now, if the search contains
punctuation, we rank exact matches first, even for small tags. Before we
ranked exact matches for small tags lower than inexact matches for large
tags. If the search contains punctuation, it's a strong signal the user
is looking for an exact match.
2022-09-09 15:58:48 -05:00
evazion
0cc76625eb Update Ruby gems and Yarn packages. 2022-09-07 03:13:13 -05:00
evazion
a80ee22746 autocomplete: fix incorrect highlighting when tag contains repeated words.
Fix tags like `short_shorts` or `hunter_x_hunter` being highlighted
incorrectly. Typing `short_sh` would highlight it as SHort_SHorts
instead of as SHORT_SHorts.
2022-09-06 18:01:30 -05:00
evazion
d2147eca80 tumblr: fix exception when fetching data for video urls.
Fix an exception when trying to fetch source data for URLs like
https://va.media.tumblr.com/tumblr_pgohk0TjhS1u7mrsl.mp4.

For these URLs it's not possible to use the trick where we try to open
the URL as a HTML page and scrape the post id from the HTML. Instead we
get the raw video if we try to to this.
2022-09-05 16:15:47 -05:00
evazion
f55951ab58 tumblr: fix exception when parsing mangled image urls.
Fix a nil exception when trying to parse invalid URLs like `https://25.media.tumblr.com/91719d337b218681abc48cdc24e`.
2022-09-05 16:15:46 -05:00
evazion
3d5b201e0d tests: fix upload tests. 2022-09-03 23:54:28 -05:00
evazion
1968c9c69d autocomplete: lowercase searches clientside.
Lowercase searches clientside before sending them to the server. This is
to improve cache hit rates when users accidentally type in uppercase,
especially on mobile where the first letter often gets capitalized.
2022-09-03 23:54:28 -05:00
evazion
306f3d8f1a autocomplete: add tests. 2022-09-03 20:34:34 -05:00
evazion
4a77d67d1f autocomplete: fix completion of ai: metatag.
Fix it so we don't show the `ai:` prefix in front of results in the
autocomplete menu when completing the `ai: metatag.
2022-09-03 20:34:34 -05:00
evazion
f83af31a00 autocomplete: fix usernames not being highlighted in @mentions.
Fix usernames not being highlighted when completing @mentions.

This also changes it so the autocomplete results don't include the '@'
in front of the name.

Minor breaking change to the /autocomplete.json API. Autocomplete
results for mentions now have the type `mention` instead of `user`.
2022-09-03 20:34:26 -05:00
evazion
03c02ef78e autocomplete: fix accidental API breakage.
Fix /autocomplete.json returning results like this:

    [
      {
        "table": {
          "type": "tag-word",
          "label": "long hair",
          "value": "long_hair",
          "category": 0,
          "post_count": 2818211,
          "antecedent": null
        }
      },
    ]

instead of like this:

    [
      {
        "type": "tag-word",
        "label": "long hair",
        "value": "long_hair",
        "category": 0,
        "post_count": 2818211,
      },
    ]

Also change it so that optional attributes like `antecedent` aren't
returned if they're null. This could be a minor breaking change if users
rely on these attributes always being present.

Lastly change XML results to look like this:

    <autocomplete-service-results type="array">
      <autocomplete-service-result>
        <type>tag-word</type>
        <label>long hair</label>
        <value>long_hair</value>
        <category type="integer">0</category>
        <post-count type="integer">2801117</post-count>
      </autocomplete-service-result>
    </autocomplete-service-results>

instead of like this:

    <objects type="array">
      <object>
        <type>tag</type>
        <label>hair ornament</label>
        <value>hair_ornament</value>
        <category type="integer">0</category>
        <post-count type="integer">883340</post-count>
        <antecedent nil="true"/>
      </object>
    </objects>
2022-09-03 18:53:29 -05:00
evazion
0274dbde10 autocomplete: fix highlighting of words surrounded by punctuation.
Fix a bug where the word `lazy` wasn't highlighted in `don't_say_"lazy"`
when searching for `lazy`. The bug was that punctuation surrounding
words wasn't properly split from the word.
2022-09-03 18:21:56 -05:00
NamelessContributor
9ea3ffb10e sidebar: fix overflow of some long links.
136e2777cb didn't account for the
wiki_pages sidebar.
2022-09-03 12:37:24 +02:00
NamelessContributor
743989262d post tooltips: fix long tags causing tag list to overflow.
The default min-width for flex items is auto instead of 0.
2022-09-03 11:40:49 +02:00
evazion
f20e1d5bc9 autocomplete: fix highlighting of short searches.
Fix a bug where tags weren't highlighted correctly when the search only contained one character.
2022-09-02 14:15:25 -05:00
evazion
6235965da0 autocomplete: fix labels for static metatags.
When completing a static metatag, such as rating: or order:,
don't include the metatag name in the autocomplete menu.

For example, when completing `rating:g`, show `general` in the
autocomplete menu, not `rating:general`.

This makes static metatags consistent with other metatags.
2022-09-02 13:56:53 -05:00
evazion
6b6c452ea2 autocomplete: remove unused html data attributes.
Remove the `data-autocomplete-antecedent`, `data-autocomplete-category`,
and `data-autocomplete-post-count` data attributes from the autocomplete
menu. These weren't used by Danbooru itself and they bloat the HTML.
2022-09-02 13:56:53 -05:00
evazion
3d831f7e1c autocomplete: limit width of autocomplete menu.
Limit the width of the autocomplete menu to 480px. This is so that
stupidly long isekai titles don't blow out the size of the menu when
searching for common words.

For example, when searching for "look", one of the results is:

   no_matter_how_you_look_at_it_it's_your_fault_that_i'm_not_popular! -> watashi_ga_motenai_no_wa_dou_kangaetemo_omaera_ga_warui!
2022-09-02 13:56:53 -05:00
evazion
350009fd15 autocomplete: increase number of results returned from 10 to 20.
The new autocomplete system tends to return more results, so increase
the number of tags shown from 10 to 20. This is useful so you can do things
like search for 'skirt' during tagging to find all skirt-related tags.
2022-09-02 13:56:52 -05:00