Commit Graph

9139 Commits

Author SHA1 Message Date
evazion
54dbf1a436 Update Ruby gems and Yarn packages. 2022-01-17 10:49:04 -06:00
evazion
35b03799a5 jobs: increase job timeout to 24 hours.
Should fix BigQuery exports failing for the posts table. Apparently even
an 8 hour timeout wasn't enough.
2022-01-16 15:14:48 -06:00
evazion
acf565be7b Fix #4678: Validate custom CSS.
* Make it an error to add invalid custom CSS to your account.
* Add a fix script to remove custom CSS from all accounts with invalid CSS.
2022-01-15 23:20:49 -06:00
evazion
8a4faf7741 Fix #4960: Cosplay wikis link to char:x instead of just x. 2022-01-15 21:30:32 -06:00
evazion
5bec0aa147 Fix #4629: Tag renames/aliases should also modify pool descriptions. 2022-01-15 20:34:08 -06:00
evazion
33103f6dc4 pools: add ability to search for pools linking to given tag.
Add ability to search for pools linking to a given tag in the pool
description. Example:

    https://danbooru.donmai.us/pools?search[linked_to]=touhou

(This isn't actually exposed in the UI to avoid cluttering the pool
search form with rarely used options.)

Pools with broken links can be found here:

    https://danbooru.donmai.us/dtext_links?search[has_linked_tag]=No&search[has_linked_wiki]=No&search[model_type]=Pool

Lays the groundwork for fixing #4629.
2022-01-15 20:26:30 -06:00
evazion
c3c4f5a2a7 Fix #4957: Autotag non-web_source.
Autotag non-web_source on posts that have a non-http:// or https:// URL.
Add a fix script to backfill old posts.

Syntactically invalid URLs are still considered web sources. For
example, `https://google,com` technically isn't a valid URL, but it's
not considered a non-web source.
2022-01-14 22:58:27 -06:00
evazion
adc5bbf906 posts: fix paginator not appearing when all posts are hidden.
Fix the paginator not appearing when all posts on the page are hidden,
because of deleted posts, banned artists, censored tags, or non-safe
posts in safe mode. This prevented navigating to the next or previous
page.
2022-01-14 22:07:38 -06:00
evazion
00ebd2e13c Merge pull request #4956 from nonamethanks/fix-skeb
Skeb: fix several issues with the strategy
2022-01-14 22:04:44 -06:00
nonamethanks
33db1a2761 Skeb: fix several issues with the strategy
* Fix fetching of videos
* Fix fetching of original commentary
* Fix images being returned out of order in bookmarklet
2022-01-14 21:24:48 +01:00
evazion
fe18e37566 Fix #4954: BigQuery exports not updating.
Fix BigQuery export jobs failing with:

    Google::Cloud::InvalidArgumentError: required: Bucket is requester pays
    bucket but no user project provided.

Caused by changing the storage bucket to requester pays. The
`user_project` param must be set to true on requester pays buckets to
bill usage to the current project.
2022-01-14 00:12:20 -06:00
evazion
ba29794e89 comments, forum posts: add 'Copy Link' action to '...' menu.
Add a 'Copy Link' action to forum posts and comments. This copies the
full link to the clipboard. The 'Copy ID' action copies just the DText
shortlink (comment #XXX or forum #XXX).
2022-01-13 13:10:41 -06:00
evazion
5adeedcf01 forum: add "..." menu to forum posts, like comments have.
Adds the ability to copy forum post IDs, like with comments.

Fixes #4768: Display DText shortlink for forum posts/topics.
2022-01-13 12:49:20 -06:00
evazion
5ba71f2a79 Fix #4840: Forum post interface is not updated correctly after deletion.
Re-render and replace the whole forum post with AJAX when a forum post
is deleted or undeleted. This is easier now that forum posts are
componentized.
2022-01-12 12:59:06 -06:00
evazion
7ad24ef231 Fix #4953: Add is_deleted attribute to forum posts.
* Add data-is-deleted, data-updater-id, data-topic-id, and
  data-topic-is-deleted data attributes to forum posts.
* Change data-forum-post-id to data-id.
* Dim deleted forum posts like comments.
2022-01-12 12:48:54 -06:00
evazion
c1a1741d20 posts: add z=N URL param to searches and tag lists.
Add a `z=N` param to the URL when doing a search from the search box, or
when clicking a link in the tag list. This is to gather data on how
often the links in the tag list are used, and whether it's more common
to click tags in the tag list or to perform searches manually.

This is temporary and will be removed when enough data is collected.
2022-01-12 10:36:14 -06:00
evazion
7c32b8b503 Fix #4466: Replacing a post does not correctly set the best source.
Don't default the "Final source" field to the current source. This way
the default will be to set the source to the normalized replacement URL.
The "final source" field can be set to override this.
2022-01-11 19:48:20 -06:00
evazion
8bb923cf9b Fix #4807: Change creator to creator_id on forum post attributes. 2022-01-11 18:55:11 -06:00
evazion
df09bb239b Fix #4906: Trying to replace a sourceless post that matches md5 returns error and doesn't fix source.
Allow replacing a post with itself. Skip uploading the file and just
update the post's source with the given source.
2022-01-11 16:09:14 -06:00
evazion
17fb34922b nijie: fix failure to fetch source data due to change in login system.
Nijie changed their login system so that now there are two cookies that
need to be remembered: NIJIEIJIEID, and nijie_tok.
2022-01-11 15:14:54 -06:00
evazion
1518c3c4be posts: fix search queries not being logged to NewRelic in some cases (#4900)
Fix the /posts index controller not logging the normalized search query
to NewRelic when the search failed, either because of a tag limit error,
a search timeout, or a RSS feed rate limit error.

Also don't log the number of search results when it's an API request or
failed search. This is to avoid doing a potentially slow full post count
when it's not otherwise needed.
2022-01-11 13:39:30 -06:00
evazion
698d732667 Fix #4878: Disapproval messages appear out of order.
Display disapproval messages in oldest-to-newest order.
2022-01-11 12:27:56 -06:00
evazion
f8768fd6b7 BURs: move the is_banned flag when aliasing artists.
When a banned artist tag is aliased into a nonbanned artist tag, move
the is_banned flag from the old artist entry to the new artist_entry.

Related to #4940. Fixes a case where a banned artist could lose the
banned status when it was moved.
2022-01-11 12:17:56 -06:00
evazion
40d6351249 Fix #4940: DanbooruBot retiring banned_artist implications. 2022-01-11 11:53:35 -06:00
evazion
422dbbc72e Fix #4942: Missing Blacklist Controls. 2022-01-11 10:16:23 -06:00
evazion
33828ec8a4 posts: remove set_tag_string method. 2022-01-11 10:06:46 -06:00
evazion
2e1c7ce6d3 Fix #4951: chartags:0 returning posts with chartags.
* Add fix script to fix posts with incorrect tag_count_* fields.
* Simplify the code for updating tag_count_* fields (no functional change).
2022-01-10 13:33:56 -06:00
evazion
fd2db2ff23 Update Ruby gems and Yarn packages. 2022-01-10 11:32:59 -06:00
evazion
104234126f robots.txt: add more static pages.
Let Google index a few more static pages.
2022-01-10 11:00:13 -06:00
evazion
aedc09f301 bigquery: exclude GoodJob::Job from BigQuery. 2022-01-10 00:12:31 -06:00
evazion
323882dbc4 upgrades: fix unsafe redirect errors in receipt/payment actions. 2022-01-10 00:11:10 -06:00
evazion
c09cd9e9fd users: fix incorrect count columns on users table.
Fix incorrect post_upload_count, note_update_count, and
unread_dmail_count columns on the users table.
2022-01-09 12:51:10 -06:00
evazion
04d3a04836 favorites: fix users with incorrect favorite counts.
Fix users that have a non-zero favorite count, but no favorites.
Follow-up to 69facb22b.
2022-01-08 14:18:29 -06:00
evazion
69facb22b1 Fix #4265: Negative favorite counts.
There were 2176 users with incorrect favorite counts, and it stopped
after user id 522393, so this must have been an old bug.
2022-01-08 13:42:49 -06:00
evazion
d900e15dcc css: fix stylelint complaints. 2022-01-08 12:30:21 -06:00
evazion
349f8e098f posts: prevent dragging of thumbnails.
Disable the ability to click and drag thumbnails. This usually happened
by accident when you clicked on a thumbnail and accidentally dragged it,
especially during tag scripting.
2022-01-07 21:33:26 -06:00
evazion
37f2d5925f sessions: fix open redirect in login page.
Fix an open redirect exploit where if you went to <https://danbooru.donmai.us/login?url=//fakebooru.com>,
then after you logged in you would be redirected to https://fakebooru.com.

This was actually fixed by the upgrade to Rails 7.0. `redirect_to` now
raises an `UnsafeRedirectError` on redirect to an offsite URL. Before we
tried to prevent offsite redirects by checking that the URL started with
a slash, but this was insufficient - it allowed protocol-relative URLs
like `//fakebooru.com`.

Add a test case for protocol-relative URLs and return a 403 error on an
offsite redirect.
2022-01-07 21:11:04 -06:00
evazion
87dfc66073 rails: update framework files and settings to 7.0.
* Update framework files with `bin/rails app:update`.
* Update to use new Rails 7.0 default settings, except for a couple
  things regarding new cookie and cache formats that would prevent us
  from rolling back to Rails 6.1 if necessary.
2022-01-07 21:10:55 -06:00
evazion
346aeca791 rails: remove sprockets references.
Remove a dummy Sprockets config file needed to get `bin/derailed` to
run. This is no longer necessary since Rails 7.0 no longer depends on
Sprockets.
2022-01-07 14:49:14 -06:00
evazion
cae6599631 pagination: fix paginator regression caused by Rails 7.
Fix the paginator not detecting the first or last page correctly during
sequential pagination.

Caused by the fact that we fetch one more record than needed to detect
whether we're on the last page, then throw that record away by
overriding Rails' internal `records` method. An upstream refactoring
meant that the `size` method now counts the number of records *after*
the extra record is thrown away, where before it counted *before* the
extra record was thrown away.
2022-01-07 14:24:57 -06:00
evazion
72ea78e697 searchable: replace find_ordered with in_order_of.
Rails 7 added an `in_order_of` method that does what our `find_ordered`
method did before.
2022-01-07 14:24:57 -06:00
evazion
a26ed9d389 rails: fix #to_s(:human_size) deprecation warnings.
Fix this deprecation warning:

  DEPRECATION WARNING: Integer#to_s(:human_size) is deprecated. Please use Integer#to_formatted_s(:human_size) instead.
2022-01-07 14:24:54 -06:00
evazion
ea069c7b0d models: remove hack to ignore tsvector columns.
Previously we overrode `ActiveRecord::Base#columns` to make it ignore
tsvector columns so they wouldn't show up in API responses. These
columns have been removed, so this is no longer necessary.
2022-01-07 11:23:53 -06:00
evazion
53527b9b29 posts: remove pool_string, fav_string from ignored columns.
These columns have been removed from the database.
2022-01-07 11:22:10 -06:00
evazion
72d5291a27 bigquery: exclude more GoodJobs classes from BigQuery. 2022-01-06 11:13:55 -06:00
evazion
123edc63a1 bigquery: don't dump good_jobs table to bigquery. 2022-01-06 00:41:26 -06:00
evazion
b254d6d9d9 jobs: fix searching by name not finding certain jobs.
Fix a bug where /jobs?search[name]=Prune+Posts didn't find jobs named PrunePostsJob.
`"Prune Posts".tr(" ", "_").classify` was wrong because it returned `"PrunePost"`.
2022-01-05 11:14:56 -06:00
evazion
090125e239 Revert "Temp disable dumping favorites table to BigQuery."
This reverts commit 788dcbd87b.
2022-01-04 18:08:54 -06:00
evazion
9000facaf7 Revert "bigquery: temp disable dumping the posts table."
This reverts commit f02b437085.
2022-01-04 18:08:47 -06:00
evazion
7ce81ceccb emails: fix exception when user signs up without an email.
Fix an `ActionView::Template::Error: undefined method 'verification_key'
for nil` error in the welcome_user mailer when a user signs up without
an email address.

Caused by the fact that we now render mail templates regardless of
whether the user has an email address, and then skip sending the email
only after the mail template is rendered.
2022-01-04 17:57:16 -06:00