Using Solid Queue in Development with Docker and on Heroku

I recently started a small project that needed Active Job and thought I’d try out the Solid Queue backend. If you haven’t been keeping up, Solid Queue is a database backend for Active Job. Instead of using Redis or Memcached, you use whatever database you are currently using with Active Record. The operating theory is that modern databases are fast enough to use this way and that doing so simplifies your infrastructure.

Solid Queue, along with Solid Cache, a similar database backed cache, and a database backed Action Cable adapter are all slated to become the default in Rails 8.

So far, Solid Queue is working well for me.

Setup

To use Solid Queue, install the gem:

bundle add solid_queue

Then, use its installer to create the necessary tables and configure it as Active Job’s production adapter:

bin/rails generate solid_queue:install

If you want to use Solid Queue in development, you will need to enable it in your config/environments/development.rb:

module MyApp
  class Application < Rails::Application
    # [...]
    config.active_job.queue_adapter = :solid_queue
  end
end

Finally, you need to run the migrations to create the tables:

bin/rails db:migrate

Once Solid Queue is installed and configured, Active Job will transparently use it. However, as with other Active Job backends, Solid Queue requires a worker process to pull jobs off the queue. Let’s look at some options for running that process.

Local Development

The Rails development environment defaults to the Active Job Async backend. This is fine for development and doesn’t require any addition infrastructure. However, I prefer that my development environment mirrors production as closely as possible.

There are several ways to run the worker process locally.

With Rake

Out of the box, you can open a new tab/window and run:

bundle exec rake solid_queue:start

From a Procfile

Alternatively, you can install the foreman gem and create a Procfile.dev with:

web: env RUBY_DEBUG_OPEN=lazy bin/rails server -p 3000
background_jobs: bin/rails solid_queue:start

Then you can run

foreman start -f Procfile.dev

If you have installed the tailwindcss-rails it has already set up a Procfile.dev and added a script ./bin/dev that uses foreman to start Rails and Tailwind. In that case, use can just add background_jobs: bin/rails solid_queue:start to Procfile.dev

With Docker Compose

My preference for local development is to use Docker Compose. This way everything my application needs can be self contained. Setting up a Docker Compose environment is beyond the scope of this post (and a post I really need to write) For now, I have a README, but if you already have it setup, just copy your Rails service and change the command. It should look something like:

services:
  # [...]
  solid_queue:
    build: .
    stdin_open: true
    tty: true
    command: bundle exec rake solid_queue:start
    depends_on:
      - database
    environment:
      - DATABASE_URL=postgresql://postgres@database
    volumes:
      - .:/usr/src/app
      - gem_cache:/gems

In Production on Heroku

If a Procfile exists in your repo, Heroku will use to configure your dynos. Create the following Procfile:

web: bin/rails server -p ${PORT:-5000} -e $RAILS_ENV
background_jobs: bin/rails solid_queue:start

On your next deploy, a new dyno labeled “background_jobs” will appear. You will need to enable (and pay) for it.

I think Solid Queue is a great option for Active Job. It is easy to setup and use and it simplifies your infrastructure. I am looking forward to seeing it become the default.