AI image description generator: alt text and captions by API

An AI image description generator is a model that can see images and writes text about them: alt text for screen readers, a caption, or a product description. To run one from your own code, send the image to an API that accepts images and ask for each piece of text as a named field in a JSON result, so your code can store the alt text and the description separately.

On Sume, that is POST /v1/agent/completions with the image as an input_image part and an output_schema that names your fields; the docs' own example asks the agent to "Describe this product shot." Facts come from the Agent Completions docs and the attachment rules they point to, read on 2026-09-27. The mechanism, images in and JSON out, is covered in Agent API with image input and JSON output.

How do I generate a description for an image with an API?

Create a completion with one input_image part and a schema with one property per field. generation_spend_cap_usd is required and has no default. An Idempotency-Key makes a retry safe: replaying it returns the original receipt with idempotency_hit: true.

• The call returns 202 with an agent.run receipt. Poll GET /v1/agent-runs/{run_id} until next_action stops being poll_status, then read your fields from output. Over the API, a run whose output does not satisfy your schema ends failed, with the reason in output_error.
• The image must be a public HTTPS URL that Sume can fetch without auth when you create the run.
• Webhooks and the refusal codes are covered in Agent API with image input and JSON output.

curl -sS -X POST "https://api.sume.com/v1/agent/completions" \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: describe-sku-4411-v1" \
  -d '{
    "messages": [{
      "role": "user",
      "content": [
        { "type": "input_text", "text": "Describe this product shot: short alt text, a product title, and a two-sentence description." },
        { "type": "input_image", "image_url": "https://example.com/catalog/sku-4411.jpg" }
      ]
    }],
    "output_schema": {
      "name": "image_description",
      "schema": {
        "type": "object",
        "properties": { "alt_text": { "type": "string" }, "title": { "type": "string" }, "description": { "type": "string" } },
        "required": ["alt_text", "title", "description"],
        "additionalProperties": false
      }
    },
    "generation_spend_cap_usd": 1
  }'

Can I describe many images at once?

Yes, in two ways. One run can carry up to 30 images, as input_image parts or top-level attachments; ask for an array, one entry per image, and give each attachment a filename, the label the agent sees. Or send one run per image, each with its own Idempotency-Key, which keeps each description tied to one image and lets one failure affect one image only.

The schema must fit Sume's strict subset, where an array declares its items; the rules are in Sume Format structured output.

What does an image description cost?

Each completion is a real agent turn, not a flat-rate captioning call, so the price is whatever that run spends. A description task asks for text, not media, and the cap bounds generation spend only, so the agent's own turn is billed on top of it; spend caps for unattended AI agents covers caps in general.

To see what one run cost, read debited_usd, as in AI video run cost per run. The figures that matter:

What are the limits?

• Images only: input_image is the only attachment type today, so PDFs and other files cannot be described yet.
• An image over 30 MB, or a set over 500 MB, is refused with 413 attachment_too_large.
• The text is the model's reading of the picture. Review it before it goes on a product page or into alt text.