Crawl job finished event: Sume has none, poll crawl_get

Cloudflare publishes crawl.finished to Queues. Sume's crawl has no such event: wait on the job id, then read it with crawl_get. Steps and idempotency.

4 min readSume
All posts

Sume's site crawl does not publish a "finished" event to a queue. You submit the crawl, get a workspace job id, wait on that same id with jobs_wait, then read the pages with crawl_get. The webhook docs cover generation jobs and agent runs, not crawls.

Cloudflare's Sep 25, 2026 changelog says Browser Run crawl jobs can publish crawl.started, crawl.updated and crawl.finished to Cloudflare Queues; the events carry no crawl data, so you fetch results by job id afterward. The Sume flow has the same last step, with polling instead of a queue.

How do I start a crawl and know it is done?

Over REST, POST /v1/firecrawl/crawl requires an Idempotency-Key, and results come from GET /v1/firecrawl/crawls/{id}. On hosted MCP the flow is crawl_site (write, with idempotency_key, an unbilled utility) then jobs_wait then crawl_get on the same id. The tool description says never to start a second crawl to poll.

Crawl completion signals, from the Cloudflare changelog and Sume docs, read 2026-09-30.
QuestionCloudflare Browser RunSume
Completion signalcrawl.finished on a QueueTerminal status via jobs_wait / crawl_get
Where results come fromGET the job idcrawl_get with the same job id
Webhook eventsNot applicablejob.completed, job.failed, job.canceled for generation jobs

Will a Sume webhook fire when the crawl ends?

Not that the docs state. Webhooks says Sume sends terminal job events only, and describes them for generation jobs and agent runs. Do not build a crawl pipeline that waits for a callback; poll.

How do I poll without paying twice?

Reuse the same idempotency_key when retrying a submit. In the REST contract, the same key adopts the original job, and a different payload with the same key returns 409 idempotency_conflict. Crawl size is bounded: limit is at most 20, default 10. crawl_get returns status and bounded pages, tells you when to read again with poll_after_seconds, and treats failed and canceled as terminal. If truncated is set, narrow the scope.

What if I want queue-style fan-in anyway?

Run your own poller that waits on the job id and pushes a message to your queue once the status is terminal. For the same pattern on video jobs, see Cloudflare Workers Queues and video webhooks.

Sources

Related posts

More in Developers

All Developers posts

Written by Sume