AWS Step Functions wait for callback on an AI video run

To make AWS Step Functions wait for a callback, append .waitForTaskToken to a Task state's Resource and pass the task token, $$.Task.Token, to whatever does the work. The execution pauses until something calls SendTaskSuccess or SendTaskFailure with that token, or until the state times out. For an AI video job, one task starts the job, and the job's webhook hands the token back.

AWS facts come from the Step Functions guide on service integration patterns, Lambda, the Task state, quotas, and error handling, plus the SendTaskSuccess and SendTaskFailure API reference. Sume facts come from Create a run, Runs and results, and Run webhooks. All were read on 2026-09-27. Sume has no Step Functions integration: your Lambda functions call its HTTPS API.

How does the callback pattern fit a Sume video run?

Split it into two states, so the run id is in the execution's data before the wait begins:

• StartRun, a plain Lambda task, creates a Sume Format run whose communication.webhook_url points at your receiver, and returns the run id. Its Idempotency-Key comes from the execution name, so a retried task gets the original run back: same key and same body return 200 with the original receipt, with no second run and no second charge.
• WaitForVideo, a Lambda task with .waitForTaskToken, hands a small function the run id and the token. The function stores the token under the run id in a table you own and returns, and the execution waits.
• Your webhook receiver verifies Sume's signed POST, looks the token up by run_id, and calls SendTaskSuccess or SendTaskFailure.
• Keep SUME_API_KEY server-side; AWS recommends Secrets Manager over Lambda environment variables for API keys.

"StartRun": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke",
  "Parameters": {
    "FunctionName": "arn:aws:lambda:region:account-id:function:start-sume-run",
    "Payload": { "order.$": "$.order", "execution.$": "$$.Execution.Name" }
  },
  "ResultPath": "$.run",
  "Next": "WaitForVideo"
},
"WaitForVideo": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
  "Parameters": {
    "FunctionName": "arn:aws:lambda:region:account-id:function:park-task-token",
    "Payload": { "run_id.$": "$.run.Payload.run_id", "token.$": "$$.Task.Token" }
  },
  "TimeoutSeconds": 6000,
  "ResultPath": "$.video",
  "Catch": [{ "ErrorEquals": ["States.Timeout"], "ResultPath": "$.timeout", "Next": "ReadRun" }],
  "Next": "Publish"
}

Which Step Functions settings does a video run need?

Most defaults assume short tasks. These are the ones to change or check:

Why not put the task token in the webhook URL?

It looks simpler, since the receiver would need no table. Two things break it. The token may not fit: it can be up to 2,048 characters, and Sume caps webhook_url at 2,048 characters, host and path included. And it defeats safe retries: the URL is part of the request body, and Sume answers the same key with a different body with 409 idempotency_conflict. A callback task that times out gets a new random token, so a retry carrying the new token in its URL is exactly that.

What should the webhook receiver send back?

The receiver is an ordinary webhook endpoint; AWS Lambda webhook receiver for Sume covers the function URL and the signature check. After the check:

• Look the token up by the envelope's run_id. If it isn't stored yet, answer 503: Sume retries a failed attempt, up to 10 attempts, and honors your Retry-After.
• status: "OK": call SendTaskSuccess with the token and a short output, such as the run id and payload.primary_output_url. A receipt over 1 MiB arrives with payload: null, but status still reports the outcome.
• status: "ERROR": call SendTaskFailure with error set to Sume's error.code (at most 256 characters) and cause set to its message.
• If Step Functions answers TaskTimedOut, the token expired or its task already closed, for example because an earlier delivery already succeeded. Answer 2xx anyway, or Sume keeps retrying.
• Answer within 10 seconds, the time Sume gives each attempt, and dedupe on request_id, which repeats on every retry.

How long should the state wait if no webhook comes?

Without a timeout, a task waiting for a token waits until the execution reaches the one-year quota. Set TimeoutSeconds past Sume's own deadline: a run in progress is force-finalized as failed 90 minutes after created_at, or sooner when it is older than 25 minutes and has been silent for 10. Long-form video is 15 to 30 minutes of work.

When TimeoutSeconds runs out, the state fails with States.Timeout, which the States.TaskFailed wildcard does not match, so catch it by name, as above, and route it to a ReadRun state that reads GET /v1/format-runs/{run_id} once. That read is how the execution learns about a run that never POSTs: a canceled run delivers no webhook, and an endpoint that refuses all 10 attempts leaves the run as it was, with nothing delivered. Your timeout doesn't stop the run or its spend either.