potatoq¶
The Python task queue with the defaults Celery should have had.
- 🛡️ Production defaults Acks after the task finishes, time limits, dead letters, and ETAs kept by the broker.
- 🧵 Free-threaded Python Tested on 3.14t and 3.15t. Threads run CPU-bound tasks in parallel: 4 threads, 3.4× faster.
- 🎸 Django tasks
A backend for Django 6's
django.tasks, or@shared_taskwith one line inINSTALLED_APPS. Sent on commit.
Alpha software
potatoq is alpha software in limited production use. Use it in production at your own risk for now; APIs and defaults may still change between releases. Please report bugs on GitHub.
from potatoq import Potatoq # or: from potatoq import Celery
app = Potatoq("proj") # broker: $POTATOQ_BROKER_URL, your Django DB, or ./potatoq.sqlite3
@app.task
def send_receipt(order_id):
...
send_receipt.delay(42)
-
Tasks don't get lost
Acks happen after the task finishes. If its process crashes (OOM, segfault), the task is requeued, and dead-lettered after 5 crashes. A task that raises is retried only if you opt in; either way, failures end up in a dead-letter store you can replay.
-
No head-of-line blocking
Each idle process takes exactly one task. ETAs and retries wait in the broker, not in worker memory, so long tasks are never redelivered.
-
Your database is the queue
Postgres and SQLite are first-class brokers. Tasks are written inside your transaction: no
DoesNotExistraces, and no lost tasks on rollback. -
Native on every broker
SKIP LOCKEDandLISTEN/NOTIFYon Postgres, Lua leases on Redis, quorum queues and confirms on RabbitMQ, WAL anddata_versionon SQLite. -
No beat to babysit
Every worker runs the scheduler. Each run is claimed exactly once through the broker, so you can't run zero schedulers or two.
-
A find-and-replace migration
@shared_task,delay,apply_async,chain/group/chord,beat_schedule, signals andCELERY_*settings all keep working.
Defaults, side by side¶
| Celery | potatoq | |
|---|---|---|
| Acknowledgement | before running: a crash loses the task | after it finishes |
| Worker process killed (OOM, segfault) | task lost | requeued; dead-lettered after 5 crashes |
| Task raises an exception | failed and discarded; retries are opt-in | failed and dead-lettered, replayable; retries are opt-in |
| Retries, once you opt in | 3, fixed 180 s apart | 10, exponential backoff with jitter, up to 10 min apart |
| Prefetch | 4 × concurrency | one task per idle process |
| ETA / countdown | held in worker RAM | stored by the broker |
| Long tasks on Redis | re-run every hour | leases renewed while running |
| Time limits | none | 30 min, soft 30 s earlier |
.delay() inside atomic() |
sent immediately | sent on commit |
| Scheduler | separate beat process |
built into every worker, deduplicated |
async def tasks |
unsupported | supported |
Every default, with the reasoning and sources
Fast where it counts¶
5000 no-op tasks, 4 worker processes, every broker on localhost. Tasks per second to
enqueue them (.delay()) and to process them (workers running them):
| Broker | Enqueue: potatoq | Enqueue: Celery 5.6 | Process: potatoq | Process: Celery 5.6 |
|---|---|---|---|---|
| Redis | 10,200 | 5,400 | 11,900 | 2,700 |
| Postgres | 5,500 | — | 7,500 | — |
| RabbitMQ | 2,700 ¹ | 8,300 ¹ | 6,800 ² | 8,000 ² |
| SQLite | 14,700 | — | 5,700 | — |
¹ potatoq waits for RabbitMQ to confirm every publish, so a task is replicated before
.delay() returns; Celery doesn't wait, which is how its publishes can silently
disappear.
² Celery acks before running the task (a crash loses it) and prefetches; potatoq acks after the task finishes and only takes work for idle processes.
The benchmark and how to run it
potatoq was generated with Claude Opus 5.5 (high effort): the research, design, code, tests and docs. Linux and macOS only. MIT licensed.