Give your AI agent a task queue, not a webhook

The default pattern for getting work to an AI agent is the same one humans have been using since 2010: pick a URL, POST to it, hope the other side is up. That’s a webhook. Webhooks are great when the receiver is a live server. They are not great when the receiver is an agent that may not exist yet, or may be batched, or may be running on a laptop that closed its lid an hour ago.

The right primitive for “give an AI agent some work and let it process when it’s ready” is a task queue. SaaS engineers have known this since SQS shipped in 2006. The reason AI coding agents don’t use task queues today isn’t that queues are wrong for them — it’s that every queue service in production requires an AWS account, an IAM role, and a boto3.client('sqs', region_name='us-east-1', aws_access_key_id=…) call that the agent has no way to populate.

gopigeon ships a task queue rung with the same property the form rung has: anonymous create, single curl. The agent stands up a durable inbox before any human has to log in.

One curl, one queue

curl -X POST https://api.gopigeon.dev/q/create

Returns a queue id and a plaintext queue_api_key — exactly once, never again. The key is the Bearer credential for every subsequent call. The agent stashes it, hands the queue URL to whatever needs to produce or consume work, and moves on.

The publish/pull/ack cycle

Produce a message:

curl -X POST https://api.gopigeon.dev/q/{id}/publish \
  -H "Authorization: Bearer $QUEUE_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"task": "process-screenshot", "url": "https://..."}'

Pull from it as a worker:

curl "https://api.gopigeon.dev/q/{id}/pull?consumer_id=worker-1&cursor=0&limit=10" \
  -H "Authorization: Bearer $QUEUE_KEY"

Ack what you actually processed:

curl -X POST https://api.gopigeon.dev/q/{id}/ack \
  -H "Authorization: Bearer $QUEUE_KEY" \
  -d '{"consumer_id": "worker-1", "cursor": 42}'

Three endpoints. That’s the whole loop. There’s no consumer to register up front, no exchange or routing key to configure, no visibility timeout to tune. Server-assigned monotonic seq per queue, IETF Idempotency-Key dedupe for 24 hours, cursors that are exclusive on read and only move when you ack. UPSERT MAX semantics on the ack — re-acking with an older cursor is a safe no-op.

Crash your worker mid-batch and restart it: it pulls again with its last-known cursor and picks up exactly where it left off. Have two workers? Give them different consumer_ids and they each get their own cursor. Want to scale out a workpool? Same pattern, more consumer_ids.

Why a task queue is the right shape for AI agents

A few things break in the webhook model that quietly go away when there’s a queue in the middle:

Producers aren’t blocked on consumers. When an agent generates twenty work items, it shouldn’t have to keep them in memory while it round-trips them one at a time to a worker that might or might not respond. Put them on a queue, return, done. The worker pulls when it’s ready.

Retries are idempotent by construction. Agents are stochastic. They retry. Without an idempotency mechanism, retries either drop work or duplicate it — both are bad. The Idempotency-Key header gives the producer a way to retry safely for 24 hours without thinking. The queue dedupes for them.

Workers can fail and recover. An agent worker that crashes between pulling and processing doesn’t lose work. The cursor isn’t advanced by pull — only by ack. Restart the worker with the last cursor it acked, and it sees the unprocessed messages again.

Poison messages have a place to live. Sometimes a message will never succeed — bad JSON, a missing resource, a model the agent doesn’t recognize. The Dead Letter Queue catches them. GET /q/{id}/dlq lists what’s stuck. POST /q/{id}/dlq/replay puts a single message back into the main queue if you’ve fixed the underlying problem. Replay is per-row and not idempotent on purpose — the new envelope carries a replayed_from field so your application can dedupe if it wants to. See error semantics for the full list of failure modes.

None of these are new ideas. They’re table-stakes for any queue you’ve used at a real job. They are conspicuously absent from “POST to a webhook,” which is what most agent-to-anything plumbing looks like today.

Forms compose with queues

The thing this enables, that I find most interesting, is that gopigeon’s form rung and queue rung interlock cleanly. The MCP tool subscribe_form_to_queue takes a form id and a queue id, and from that moment forward every submission to the form is published as a message on the queue.

So the full agent pattern becomes:

  1. Agent creates a contact form (anonymous, one curl) for the user’s landing page.
  2. Agent creates a queue (anonymous, one curl).
  3. Agent calls subscribe_form_to_queue(form_id=…, queue_id=…).
  4. Agent writes the worker that consumes the queue (this part can be another agent on a cron, or a human reviewing in Slack, or a Lambda — doesn’t matter, the queue doesn’t care).

The landing page form now flows into a real durable work system, and at no point did the agent have to stop and ask the user for credentials or a Stripe checkout. The user finds out a queue exists when their first form submission produces an inbox they can claim.

Pricing, and the part that surprised me when I built it

Free tier is 100 messages per month per user across all owned queues. Pro is 10,000, and quota behavior at the boundary is documented separately. When you cross the free limit, the publish doesn’t fail in the way you’d expect — it returns 402 Payment Required with auto_enqueued: true, which means the message was persisted server-side anyway. When you upgrade, an async job drains every stranded envelope back to the queue in original FIFO order within 60 seconds. No re-publishes, no manual replay, no lost work.

I built it that way because the alternative — silently dropping messages at the quota line — was unacceptable when the producer is an autonomous agent. An agent that hits a 402 and drops the message doesn’t have a human in the loop to notice and re-send. Persisting the message and replaying it on upgrade was the only behavior that preserved the property “publish once and forget.”

That’s the most non-obvious thing about the design, and it’s the part you don’t have to think about, which is the point.

Try it

curl -X POST https://api.gopigeon.dev/q/create

Full surface — publish, pull, ack, DLQ, consumers, origins — is in the queue docs. The MCP server exposes create_queue, publish_to_queue, pull_queue, ack_queue, and subscribe_form_to_queue so an agent can do the whole flow without leaving its tool calls. Pricing here.

← Back to writing