Commit Graph
789 Commits
Author SHA1 Message Date
evazion 075199cd1e Remove /ip_addresses page.
Remove the /ip_addresses page. This page allowed moderators to search
users by IP, and to see recent activity tied to an IP. However, it was
limited to IPs tied to uploads, comments, dmails, artist edits, note
edits, and wiki edits.

Remove this page because it was limited in scope and because there are
better ways of doing what it did. The /user_events page is better at
catching sockpuppets because it tracks IPs for every login, not just for
certain types of edits. And the /user_actions page is better at
monitoring user activity because it shows all activity associated with
an account, not just for certain types of edits.

Removing this allows us to drop IP addresses from all tables besides the
user_events table. This is good because these IPs are no longer necessary
for any purpose, and because storing them forever is a liability.
2022-09-17 21:32:26 -05:00
evazion bd73090b4c user events: make all events visible to moderators.
Allow moderators to see all events on the /user_events page. Before only
admins could see when a user changed their email, changed their
password, or had a failed login attempt. Now moderators can see these
events too.

Filtering these events out made the /user_actions page slower, and it
wasn't really necessary since merely knowing that a user changed their
email or password isn't that much more sensitive than knowing when they
logged in or out.
2022-09-16 06:01:44 -05:00
evazion ee638f976f Add /user_actions page.
Add a /user_actions page. This page shows you a global timeline of
(almost) all activity on the site, including uploads, comments, votes,
edits, forum posts, and so on.

The main things it doesn't include are post edits, pool edits, and
favorites (posts and pools live in a separate database, and favorites
don't have the timestamps we need for ordering).

This page is useful for moderation purposes because it lets you see a
history of almost all of a user's activity on a single page.

Currently this page is mod-only. In the future it will be open to all
users, so you can view the history of your own site activity, or the
activity of others.
2022-09-16 05:39:25 -05:00
evazion e2a3265daf mod dashboard: remove ip address search.
Remove the IP address search option from the /moderator/dashboard page.
This was an obsolete way of searching for sockpuppet accounts by IP.
The /user_events page should be used instead.
2022-09-13 00:14:19 -05:00
evazion 0267e4e3d7 wikis: fix exception when wiki doesn't have a tag.
Fix a nil reference exception in the sidebar when a wiki page doesn't
belong to a tag.

Also hide the options sidebar on the new wiki page since none of the
options are relevant when creating a new wiki.
2022-09-12 21:24:03 -05:00
evazion bb728ecebf tags: add /tag_versions page. 2022-09-11 18:41:16 -05:00
evazion 54a45a3021 tags: track tag histories.
Track the history of the tag `category` and `is_deprecated` fields in
the `tag_versions` table.

Adds generic Versionable and VersionFor concerns that encapsulate most
of the history tracking logic. These concerns are designed to make it
easy to add history to any model.

There are a couple notable differences between tag versions and other versions:

* There is no 1 hour edit merge window. All changes to the `category`
  and `is_deprecated` fields produce a new version in the tag history.

* New versions aren't created when a tag is created. Versions are only
  created when a tag is edited for the first time. The tag's initial
  version isn't created until *after* the tag is edited for the first time.

For example, if you change the category of a tag that was last updated
10 years ago, that will create an initial version of the tag backdated
to 10 years ago, plus a new version for your edit.

This is for a few reasons:

* So that we don't have to create new tag versions every time a new tag
  is created. This would be wasteful because most tags never have their
  category or deprecation status change.
* So that if you make a typo tag, your name isn't recorded in the tag's
  history forever.
* So that we can create new tags in various places without having to know
  who created the tag (which may be unknown if the current user isn't set).
* Because we don't know the full history of most tags, so we have to
  deal with incomplete histories anyway.

This has a few important consequences:

* Most tags won't have any tag versions. They only gain tag versions if
  they're edited.
* You can't track /tag_versions to see newly created tags. It only
  shows changes to already existing tags.
* Tag version IDs won't be in strict chronological order. Higher IDs may
  have created_at timestamps before lower IDs. For example, if you
  change the category of a tag that is 10 years old, that will create an
  initial version with a high ID, but with a created_at timestamp dated
  to 10 years ago.

Fixes #4402: Track tag category changes
2022-09-11 17:47:44 -05:00
evazion a80ee22746 autocomplete: fix incorrect highlighting when tag contains repeated words.
Fix tags like `short_shorts` or `hunter_x_hunter` being highlighted
incorrectly. Typing `short_sh` would highlight it as SHort_SHorts
instead of as SHORT_SHorts.
2022-09-06 18:01:30 -05:00
evazion 306f3d8f1a autocomplete: add tests. 2022-09-03 20:34:34 -05:00
evazion cf13ab1540 autocomplete: render html server-side.
Render the HTML for autocomplete results server-side instead of in
Javascript. This is cleaner than building HTML in Javascript, but it may
hurt caching because the HTTP responses are larger.

Fixes #4698: user autocomplete contains links to /posts

Also fixes a bug where tag counts in the autocomplete menu were different
from tag counts displayed elsewhere because of differences in rounding.
2022-08-30 01:26:02 -05:00
evazion 18c949ff34 Fix #5208: The posts version of create or update artist commentary endpoint does not work without the post ID parameter being passed in the body
Fix /posts/:post_id/artist_commentary/create_or_update not working
without passing `artist_commentary[post_id]` in the form body.
2022-08-25 21:50:33 -05:00
evazion 91b3a4c37a Fix #5205: Creating posts via API fails if you pass any commentary field but not all. 2022-08-25 21:35:29 -05:00
evazion 1ee2d22b97 Fix #5184: undefined method `to_model' for []:Array searching for IQDB queries with JSON. 2022-08-25 21:28:21 -05:00
evazion 115085006e Fix #5194: AND/OR no longer trigger autocomplete.
Also change the /autocomplete.json API to no longer strip '-' and '~'
from the start of the tag. This may be a breaking change if third-party
scripts relied on this behavior.
2022-08-25 20:45:22 -05:00
evazion 28ee901192 Fix #5211: Using tab to autocomplete tags in tag implication/alias search adds a space making it return no results.
Fix the alias and implication search forms not normalizing tags before search.
2022-08-24 22:01:03 -05:00
evazion c2597ff299 Fix #5231: "Reason is too long" notice does not appear when editing an appeal 2022-08-24 14:29:19 -05:00
evazion d01b3c0637 wiki pages: remove /wiki_pages/search page.
This page is unnecessary since you can already search wiki pages from the /wiki_pages listing.
2022-08-24 14:27:47 -05:00
evazion d9cf7f7443 users: allow anonymous users to use /profile.json.
Allow logged out users to call https://danbooru.donmai.us/profile.json.
This allows getting information on default settings and limits for
anonymous users.

May be a breaking API change if users were using the HTTP response code
from /profile.json to check if they were successfully logged in.
2022-08-24 02:24:05 -05:00
evazion e3af738371 tests: fix broken tests. 2022-08-24 02:03:37 -05:00
evazion c8917684b8 jobs: fix failures in /jobs controller.
Fix errors with searching for and retrying jobs on the /jobs page caused
by the upgrade to GoodJob 3.0.
2022-08-23 18:07:04 -05:00
evazion 67798c9ece Fix #5221: Trying to upload an unsupported url shows ai tags error. 2022-07-01 18:13:36 -05:00
evazion 907194fc6f commentaries: move search form to /artist_commentaries page.
* Move the commentary search form to the /artist_commentaries page.
* Add Order field for changing the sort order.
2022-06-05 14:28:44 -05:00
evazion 173e43b192 user upgrades: add upgrade code system.
Add a system for upgrading accounts using upgrade codes. Users purchase
an upgrade code off-site then redeem it on-site to upgrade their account
to Gold. Upgrade codes are randomly pre-generated and are one time use
only. Codes have enough randomness that guessing a code is infeasible.
2022-06-01 18:31:46 -05:00
evazion 81bd86d202 posts: add "general" rating; rename "safe" rating to "sensitive".
* Add "general" rating.
* Rename "safe" rating to "sensitive".
* Change safe mode to include both rating:s and rating:g.
* Treat rating:safe as a synonym for rating:sensitive.
* Link "howto:rate" in the post edit form.
2022-05-22 13:38:45 -05:00
evazion 4b65e96abc upgrades: rename stripe_id to transaction_id
* Rename the stripe_id column to transaction_id.
* Add a new payment_processor column to identity the processor used for
  this transaction (and hence, which backend system the transaction_id is for).
2022-05-15 01:05:24 -05:00
evazion 638c928c8a Add 18 USC 2257 disclaimer.
Add a 2257 disclaimer and link to it in the site footer.
2022-05-09 02:26:19 -05:00
evazion 449fd6c49c upgrades: factor out Stripe integration.
Factor out the Stripe code from the UserUpgrade class. Introduce a new
PaymentTransaction abstract class that represents a payment with some
payment processor, and a PaymentTransaction::Stripe class that
implements transactions with Stripe.

Note that we can't completely eliminate Stripe even though we no longer
accept payments with it because we still need to be able to look up old
payments in Stripe.
2022-05-06 22:52:33 -05:00
evazion 2219a64f47 posts: show takedown page for banned artists and posts.
Show a "This page has been removed because of a takedown request" error when
an unauthorized user searches for a banned tag, or tries to view a banned post.
2022-05-03 05:56:14 -05:00
evazion 1a89071f2d posts: fix error on post index page when search count times out in safe mode
Fix a nil deference error on the post index page. This happened when
performing a single tag search in safe mode and calculating the number
of search results timed out.
2022-05-02 22:03:33 -05:00
evazion 17ffe3590a Fix #4982: Add route to remove a post from a favorite group 2022-05-02 15:56:16 -05:00
evazion 93352b318e Fix #5146: Adding an existing favorite to favorite groups leads to an error.
Show "Favgroup already contains post XXX" error when trying to add a
post to a favgroup that already contains that post.
2022-05-02 15:56:16 -05:00
evazion f117049750 users: remove 'hide deleted posts' account setting.
This setting automatically added the `-status:deleted` metatag to all searches. This meant deleted
posts were filtered out at the database level, rather than at the html level. This way searches
wouldn't have less-than-full pages.

The cost was that searches were slower, mainly because post counts weren't cached. Normally when you
search for a tag, we can get the post count from the tags table. If the search is actually like
`touhou -status:deleted`, then we don't know the count and we have to calculate it on demand.

This option is being removed because it did the opposite of what people thought it did. People
thought it made deleted posts visible, when actually it made them more hidden.
2022-05-01 00:47:46 -05:00
evazion ccd0dde081 Fix #5013: BUR model doesn't validate tags.
Don't allow users to request aliases, implications, or renames for invalid tag names.

As a side effect, it's no longer possible to request shortcut aliases like
`/hr -> hakurei_reimu` (slash abbreviations still exist, but they can't
be overridden with aliases). Tests involving these types of aliases are
removed.
2022-04-30 20:03:04 -05:00
evazion 918f32c554 Fix #4461: Improve posts/index page titles. 2022-04-30 01:52:33 -05:00
evazion bbe748bd2b posts: factor out post edit logic.
Factor out most of the tag edit logic from the Post class to a new
PostEdit class. The PostEdit class contains the logic for parsing tags
and metatags from the tag edit string, and for determining which tags
were added or removed by the edit.

Fixes various bugs caused by not calculating the set of added or removed
tags correctly, for example when tag category prefixes were used (e.g.
`copy:touhou`) or when the same tag was added and removed in the same
edit (e.g. `touhou -touhou`).

Fixes #5123: Tag categorization prefixes bypass deprecation check
Fixes #5126: Negating a deprecated tag will still cause the warning to show
Fixes #3477: Remove tag validator triggering on tag category changes
Fixes #4848: newpool: metatag doesn't parse correctly
2022-04-29 17:13:33 -05:00
evazion 6c5dd5ffed tests: fix broken tests. 2022-04-18 00:31:31 -05:00
nonamethanks 1a990d5ab9 Allow post disapprovals to be edited 2022-04-11 21:05:44 +02:00
nonamethanks 11281d6f58 Tags: don't allow deprecation of tags without wiki 2022-04-09 20:16:55 +02:00
nonamethanks ea76a889db Add ability to mark tags as deprecated
* Deprecated tags can't be added to posts, but existing deprecated tags
  in a post won't be removed
* Only empty tags can be marked as deprecated manually
* No tags can be manually undeprecated
** These limits don't apply to admins
* Deprecating or undeprecating a tag will create a new mod action to
  prevent people from going rogue
* Added deprecate/undeprecate commands for BURs
* Deprecating a tag via BUR removes all implications to and from it as well
2022-04-08 09:07:14 +02:00
evazion 0d480eb832 artist urls: stop using normalized_url.
Stop the last remaining uses of the `artist_urls.normalized_url` column.
It's already no longer used by the artist finder. The only remaining
uses were by API users. Those users should use the `url` column instead.
2022-04-02 23:58:01 -05:00
evazion a272c19b98 Fix #5078: Pixiv booth upload broken.
Allow image URLs from https://booth.pximg.net to be uploaded. Fix bug
where Booth.pm URLs were incorrectly caught by the Pixiv extractor.
2022-03-30 03:25:42 -05:00
evazion d9d3c1dfe4 sources: rename Sources::Strategies to Source::Extractor.
Rename Sources::Strategies to Source::Extractor. A Source::Extractor
represents a thing that extracts information from a given URL.
2022-03-24 03:49:44 -05:00
evazion 56f47c60e1 posts: fix exception when viewing post with source Blog..
Fix a PublicSuffix::DomainNotAllowed exception raised with viewing or editing a post
with a source like `Blog.`.

This happened when parsing the post's source. `Danbooru::URL.parse("Blog.")` would
heuristically parse the source into `http://blog`. Calling any methods related to the
URL's hostname or domain would lead to calling `PublicSuffix.parse("blog")`, which
would fail with PublicSuffix::DomainNotAllowed.
2022-03-21 03:24:50 -05:00
evazion defea08084 posts: fix exception in random:1 searches.
Fix regression in 1ad0e8688. Caused by `relation.order_values` returning
an array of Arel nodes instead of an array of strings when doing a
`random:1` search.
2022-03-21 01:29:10 -05:00
evazion 01b683798e sources: add Tinami support. 2022-03-19 00:50:36 -05:00
evazion 912e996027 Fix #4470: Check URLs for duplicates when creating artists
Show a warning when creating a duplicate artist; that is, when adding a
URL that already belongs to another artist.

This is a soft warning rather than a hard error because there are some
cases where multiple artists legitimately share the same site or account.
2022-03-18 17:10:23 -05:00
evazion 42144eaa4b Fix #5012: Fc2 image link paste not uploading.
Fix referer spoofing not working for certain fc2.com image URLs.

Spoofing the referer like this redirects to an HTML error page:

* curl -H "Referer: http://wwwew.web.fc2.com" http://wwwew.web.fc2.com/e/405.jpg

Spoofing it like this works:

* curl -H "Referer: http://wwwew.web.fc2.com/e/405.jpg" http://wwwew.web.fc2.com/e/405.jpg
2022-03-18 04:39:13 -05:00
evazion 10dac3ee51 artists: normalize urls added to artist entries.
When a URL is added to an artist entry, normalize it to a standard form.

Artist URLs have both a `url` column and a `normalized_url` column. The
`normalized_url` is used for artist finding and the `url` is the raw URL
entered by the user. Previously only the `normalized_url` field was
normalized; now the URL entered by the user is also converted to a
normalized form.

This means that if an URL like this is added to an artist entry:

* http://www.pixiv.net/member.php?id=1234
* http://www.pixiv.net/en/users/1234
* http://www.twitter.com/DanbooruBot/
* http://mobile.twitter.com/DanbooruBot/

It will get normalized to this:

* https://www.pixiv.net/users/1234
* https://twitter.com/DanbooruBot

This fixes problems with duplicate URLs being added to artist entries
because URLs weren't normalized to a single form.
2022-03-18 02:06:50 -05:00
nonamethanks a6549bc6fe Add Fantia support
Also fixes a regression in 74fdeef10c
that stopped mastodon urls from being given the right priority.
2022-03-10 17:43:32 +01:00
evazion 80939f5e22 users: fix name changes not being validated.
Fix names not being validated when a user changes their name. Regression in 5ce724f1e.
2022-03-09 01:14:09 -06:00