Commit Graph

10990 Commits

Author SHA1 Message Date
evazion
cae6599631 pagination: fix paginator regression caused by Rails 7.
Fix the paginator not detecting the first or last page correctly during
sequential pagination.

Caused by the fact that we fetch one more record than needed to detect
whether we're on the last page, then throw that record away by
overriding Rails' internal `records` method. An upstream refactoring
meant that the `size` method now counts the number of records *after*
the extra record is thrown away, where before it counted *before* the
extra record was thrown away.
2022-01-07 14:24:57 -06:00
evazion
72ea78e697 searchable: replace find_ordered with in_order_of.
Rails 7 added an `in_order_of` method that does what our `find_ordered`
method did before.
2022-01-07 14:24:57 -06:00
evazion
a26ed9d389 rails: fix #to_s(:human_size) deprecation warnings.
Fix this deprecation warning:

  DEPRECATION WARNING: Integer#to_s(:human_size) is deprecated. Please use Integer#to_formatted_s(:human_size) instead.
2022-01-07 14:24:54 -06:00
evazion
3f25ad6dce rails: upgrade to Rails 7.0.1. 2022-01-07 14:24:45 -06:00
evazion
41a095abfe gems: temp disable meta_request gem.
It's not yet compatible with Rails 7.0.
2022-01-07 12:43:39 -06:00
evazion
ea069c7b0d models: remove hack to ignore tsvector columns.
Previously we overrode `ActiveRecord::Base#columns` to make it ignore
tsvector columns so they wouldn't show up in API responses. These
columns have been removed, so this is no longer necessary.
2022-01-07 11:23:53 -06:00
evazion
53527b9b29 posts: remove pool_string, fav_string from ignored columns.
These columns have been removed from the database.
2022-01-07 11:22:10 -06:00
evazion
dbf4e1e98e db: remove unused tsvector triggers.
This was forgotten in 080dbf5a8.
2022-01-06 20:03:50 -06:00
evazion
0f3fc0d697 Fix Twitter widget overlapping Discord widget on Downbooru page. 2022-01-06 12:43:37 -06:00
evazion
e8c52432a4 db: remove unused columns on posts table.
is_note_locked, is_rating_locked, and is_status_locked have been unused
since 126046cb6.

tag_index has been unused since 37a8dc5db.

fav_string has been unused since 165339236.

pool_string has been unused since 7d503f088.
2022-01-06 11:39:18 -06:00
evazion
080dbf5a8c db: remove unused tsvector columns.
These columns have been unused since e3b836b50.
2022-01-06 11:25:51 -06:00
evazion
72d5291a27 bigquery: exclude more GoodJobs classes from BigQuery. 2022-01-06 11:13:55 -06:00
evazion
123edc63a1 bigquery: don't dump good_jobs table to bigquery. 2022-01-06 00:41:26 -06:00
evazion
edd0656b73 tests: fix broken tests. 2022-01-06 00:41:18 -06:00
evazion
b254d6d9d9 jobs: fix searching by name not finding certain jobs.
Fix a bug where /jobs?search[name]=Prune+Posts didn't find jobs named PrunePostsJob.
`"Prune Posts".tr(" ", "_").classify` was wrong because it returned `"PrunePost"`.
2022-01-05 11:14:56 -06:00
evazion
090125e239 Revert "Temp disable dumping favorites table to BigQuery."
This reverts commit 788dcbd87b.
2022-01-04 18:08:54 -06:00
evazion
9000facaf7 Revert "bigquery: temp disable dumping the posts table."
This reverts commit f02b437085.
2022-01-04 18:08:47 -06:00
evazion
7ce81ceccb emails: fix exception when user signs up without an email.
Fix an `ActionView::Template::Error: undefined method 'verification_key'
for nil` error in the welcome_user mailer when a user signs up without
an email address.

Caused by the fact that we now render mail templates regardless of
whether the user has an email address, and then skip sending the email
only after the mail template is rendered.
2022-01-04 17:57:16 -06:00
evazion
1ccc74adca jobs: add ability to order jobs in /jobs listing. 2022-01-04 17:53:35 -06:00
evazion
dc159ed135 jobs: show retried jobs in /jobs listing.
Fix the /jobs listing to show retried jobs. GoodJob::ActiveJobJob has a
default scope that filters out retried jobs; remove it in the index
controller so we can see retried jobs.

Also fix mail delivery jobs not showing up in the dropdown in the /jobs
search form.
2022-01-04 17:46:54 -06:00
evazion
f38910f0a2 jobs: hide job arguments and errors from non-admins.
These can sometimes contain sensitive information, such as IP addresses
or what files a user is trying to upload.
2022-01-04 17:20:43 -06:00
evazion
82211ba935 jobs: add ability to search jobs on /jobs page.
Add ability to search jobs on the /jobs page by job type or by status.

Fixes #2577 (Search filters for delayed jobs). This wasn't possible
before with DelayedJobs because it stored the job data in a YAML string,
which made it difficult to search jobs by type. GoodJobs stores job data
in a JSON object, which is easier to search in Postgres.
2022-01-04 17:18:36 -06:00
evazion
12601e49fd users: fix exception in can_receive_email?. 2022-01-04 16:04:41 -06:00
evazion
3841fba78e jobs: remove DelayedJobs.
Remove the DelayedJobs gem and database table. Completes the transition
to GoodJob started in c06bfa64f and f4953549a.

Downstream users can upgrade as follows:

* Stop the Rails server.
* Stop the DelayedJobs worker (normally running as `bin/delayed_job` or `bin/rails jobs:work`).
* Run `bin/rails jobs:work` to finish any pending delayed jobs.
* Run `bin/rails db:migrate` to create the good_jobs table and drop the delayed_jobs table.
* Start the Rails server again.
* Start the GoodJobs worker with `bin/good_job start`.
2022-01-04 15:58:12 -06:00
evazion
f4953549ae jobs: switch from DelayedJob to GoodJob.
Switch the ActiveJob backend from DelayedJob to GoodJob. Differences:

* The job worker is run with `bin/good_job start` instead of `bin/delayed_job`.
* Jobs have an 8 hour timeout instead of a 4 hour timeout.
* Jobs don't automatically retry on failure.
* Finishing jobs are preserved and pruned after 7 days.
2022-01-04 13:52:08 -06:00
evazion
21a9bb2c63 jobs: rename /delayed_jobs to /jobs.
Rename the /delayed_jobs endpoint to just /jobs since it's no longer
based on DelayedJob.
2022-01-02 21:21:16 -06:00
evazion
f7784d2340 jobs: update jobs dashboard to work with GoodJob.
Update the jobs dashboard at /delayed_jobs to work with GoodJob instead
of DelayedJob.
2022-01-02 21:21:04 -06:00
evazion
c06bfa64f5 Add GoodJob gem.
This is the first step towards replacing DelayedJob with GoodJob. Compared to
DelayedJob:

* GoodJob supports Rails 7 (DelayedJob is currently a blocker for Rails 7
  because it has a version bound on ActiveRecord <6.2).
* GoodJob has a builtin admin dashboard.
* GoodJob supports threaded job workers.
* GoodJob supports scheduled cronjobs.
* GoodJob supports healthchecks for workers.
* GoodJob uses Postgres notifications instead of polling to pick up new
  jobs. This allows jobs to be picked up faster and scales better with
  large numbers of workers.

https://github.com/bensheldon/good_job
2022-01-02 17:13:41 -06:00
evazion
370ed32426 emails: fix exception when disabling dmail notifications.
Fix an `undefined method 'find' for Maintenance::User:Module` exception
when disabling email notifications using the "Disable notifications"
link in dmails.
2022-01-02 17:10:27 -06:00
evazion
e40218d1c8 Fix #4515: Set List-Unsubscribe header on notification emails. 2022-01-02 17:10:27 -06:00
evazion
27ed41437c emails: fix invalid RCPT TO errors.
Fix `501 invalid RCPT TO address provided` errors caused by usernames
containing invalid characters. Example: `To: Foo,Bar <foobar@gmail.com>`.
2022-01-02 16:08:35 -06:00
evazion
751835745b emails: fix email validation regex.
Fix the email validation regex allowing certain invalid emails like `foo@gmail..com`.
2022-01-02 16:08:35 -06:00
evazion
d903f45935 emails: add script to fix invalid emails.
Add a fix script to fix email addresses that are invalid or that contain
common typos. If the email can't be fixed, usually because the fixed
address is already in use by another account, then the email address is
marked undeliverable.
2022-01-02 16:08:35 -06:00
evazion
32613f9bb1 emails: fix sending emails to invalid addresses.
Fix mailers to not attempt deliveries to invalid or nonexistent email
addresses. This usually happened when someone changed their email, and
we tried to send a confirmation email to a nonexistent address.
2022-01-02 16:07:57 -06:00
evazion
9ffc269e43 winter sale: update banner interval.
Reset cookie and set the hide banner interval to 1 hour for the final
three hours of the sale.
2021-12-31 15:12:47 -06:00
evazion
18c08688df Merge pull request #4947 from nonamethanks/fix-duration-twitter
Fix duration check for uploads from twitter
2021-12-29 22:33:59 -06:00
nonamethanks
15bd5f73b3 Fix duration check for uploads from twitter
Some twitter videos near the max duration had some stray
milliseconds that made the check fail.

For example
https://twitter.com/kivo_some_18/status/1152167154059321344?s=20 (nsfw)
has 140.053333 duration.
2021-12-29 14:25:14 +01:00
evazion
f2b5c9f54d winter sale: fix error with promotion codes.
Fix an exception when checking out on Stripe caused by using the
`allow_promotion_codes` and `discounts` parameters at the same time.

   Stripe::InvalidRequestError - You may only specify one of these parameters: allow_promotion_codes, discounts
2021-12-28 16:05:45 -06:00
evazion
b8356c97c1 Add Danbooru Winter Sale. 2021-12-28 14:41:37 -06:00
evazion
572b1b74f7 posts: add hidden 720x720 thumbnail option (#4944).
Allow 720x720 thumbnails with https://danbooru.donmai.us/posts?size=720.
2021-12-28 11:36:14 -06:00
evazion
6d7a139ef1 Fix #4946: Don't Add Comments to Posts When Doing Post Replacements. 2021-12-28 11:16:15 -06:00
evazion
0e778f0972 Update Ruby gems and Yarn packages. 2021-12-28 09:47:34 -06:00
evazion
4d9028f1fd docker: switch Ruby memory allocator to Jemalloc.
Switch the Ruby memory allocator from Glibc malloc to Jemalloc. Jemalloc
supposedly uses less memory than Glibc malloc because it's better at
handling memory fragmentation. It also has detailed internal statistics
to help monitor allocator behavior.

We use the LD_PRELOAD method of loading Jemalloc instead of building it
into Ruby so that we can switch allocators at runtime.
2021-12-19 03:44:59 -06:00
evazion
0ba6dc9ee5 Fix #4945: Search for an artist by URL throws an exception. 2021-12-18 01:55:29 -06:00
evazion
b5b54f2c78 gems: add rbtrace 2021-12-17 22:23:59 -06:00
evazion
993965b654 posts: reduce string allocations during thumbnail generation.
Further micro-optimize thumbnails to reduce string allocations.

`Post#levelblocked?` gets called once per thumbnail. Before it split the
tag string, which meant one string allocation for each tag on each post.
This added up to thousands of string allocations per pageload.
2021-12-16 17:17:06 -06:00
evazion
1c5786d20f posts: remove cropped thumbnails. 2021-12-16 15:58:29 -06:00
evazion
379775d10d Remove HTML data attributes from thumbnails #4943
Remove most data attributes from thumbnails, except the ones used by our
own Javascript.
2021-12-16 15:48:07 -06:00
evazion
c3f61a5888 tests: fix broken tests. 2021-12-16 01:38:06 -06:00
evazion
a62ae69740 dtext: fix frozen string error. 2021-12-16 00:56:46 -06:00