9ba84efc07
Change the way BURs are processed. Before, we spawned a background job for each line of the BUR, then processed each job sequentially. Now, we process the entire BUR sequentially in a single background job. This means that: * BURs are truly sequential now. Before certain things like removing aliases weren't actually performed in a background job, so they were performed out-of-order before everything else in the BUR. * Before, if an alias or implication line failed, then subsequent alias or implication lines would still be processed. This was because each alias or implication line was queued as a separate job, so a failure of one job didn't block another. Now, if any alias or implication fails, the entire BUR will fail and stop processing after that line. This may be good or bad, depending on whether we actually need the BUR to be processed in order or not. * Before, BURs were processed inside a database transaction (except for the actual updating of posts). Now they're not. This is because we can't afford to hold transactions open while processing long-running aliases or implications. This means that if BUR fails in the middle when it is initially approved, it will be left in a half-complete state. Before it would be rolled back and left in a pending state with no changes performed. * Before, only one BUR at a time could be processed. If multiple BURs were approved at the same time, then they would queue up and be processed one at a time. Now, multiple BURs can be processed at the same time. This may be undesirable when processing large BURs, or BURs that must be approved in a specific order. * Before, large tag category changes could time out. This was because they weren't actually performed in a background job. Now they are, so they shouldn't time out.
Jobs
This directory contains background jobs used by Danbooru. Jobs are used to handle slow-running tasks that need to run in the background, such as processing uploads or bulk update requests. They're also used for asynchronous tasks, such as sending emails, that may temporarily fail but can be automatically retried later.
Jobs use the Rails Active Job framework. Active Job is a common framework that allows jobs to be run on different job runner backends.
In the production environment, jobs are run using the Delayed Job backend. Jobs
are stored in the database in the delayed_job table. Worker processes spawned
by bin/delayed_job poll the table for new jobs to work.
In the development environment, jobs are run with an in-process thread pool. This will run jobs in the background, but will drop jobs when the server is restarted.
There are two job queues, the default queue and the bulk_update. The
bulk_update queue handles bulk update requests. It has only one worker so that
bulk update requests are effectively processed sequentially. The default queue
handles everything else.
There is a very minimal admin dashboard for jobs at https://danbooru.donmai.us/delayed_jobs.
Danbooru also has periodic maintenance tasks that run in the background as cron jobs. These are different from the jobs in this directory. See app/logical/danbooru_maintenance.rb.
Usage
Start a pool of job workers:
RAILS_ENV=production bin/delayed_job --pool=default:8 --pool=bulk_update start
Examples
Spawn a job to be worked in the background. It will be worked as soon as a worker is available:
DeleteFavoritesJob.perform_later(user)
See also
- app/logical/danbooru_maintenance.rb
- app/controllers/delayed_jobs_controller.rb
- config/initializers/delayed_jobs.rb
- test/jobs