Choosing a broker¶
Every broker supports the full feature set (delays, priorities, retries, dead letters, chords and periodic tasks), each implemented with its own native primitives. Choose based on what you already run.
| Broker | Best for | Extra infrastructure | Needs | Transactional enqueue | Results | Throughput¹ |
|---|---|---|---|---|---|---|
| Postgres | most apps | none (your database) | PostgreSQL 13+ | built in | ~7.5k/s | |
| Redis / Valkey | high throughput | Redis | Redis 6.2+, Valkey 7.2+ | on commit | opt-in | ~12k/s |
| RabbitMQ | RabbitMQ shops | RabbitMQ | RabbitMQ 4.0+ | on commit | separate backend | ~7k/s |
| SQLite | development, single host | none (a file) | SQLite 3.35+² | built in | ~6k/s |
"On commit": a transactional enqueue needs the queue in your database, so these brokers
send tasks right after COMMIT instead.
¹ No-op tasks per second, 4 worker processes, broker on localhost (benchmark).
² The SQLite that Python was built with: python -c "import sqlite3; print(sqlite3.sqlite_version)".
Pythons from uv and python.org bundle a recent one; see SQLite.
CI tests every broker on the oldest versions listed here and on the newest (PostgreSQL 18, Redis 8, RabbitMQ 4.3).
Recommendations¶
Already on Postgres? Use it. You get no new infrastructure, tasks that commit and roll back with your data, and results stored in the same transaction as the ack. A single Postgres instance handles thousands of tasks per second, more than most applications ever enqueue. Postgres broker
Need tens of thousands of tasks per second, or already run Redis? Use Redis, with
maxmemory-policy noeviction and AOF persistence.
Redis broker
Already operate RabbitMQ? It works well: quorum queues, publisher confirms, and
broker-side delays. Pair it with a Redis or Postgres result_backend if you need
results.
RabbitMQ broker
Development, CI, a single small server? SQLite is the default because it needs nothing at all. SQLite broker
Tests
For unit tests use memory:// with potatoq.testing.drain.