Image Description

Extract Insights from Your Images

Image description is the most natural use case for Grok’s multimodal vision. You send an image, ask a question, and the model returns a detailed text analysis. Deceptively simple, this capability opens up significant possibilities in production.

General Description

The simplest request is to ask for a free-form description:

response = client.responses.create(
    model="grok-4.5",
    input=[
        {"type": "input_image", "image_url": url_image},
        {"type": "input_text", "text": "Describe this image in detail."}
    ]
)

The model identifies key elements (objects, people, text, colors, composition) and produces a descriptive paragraph. For most images, auto mode is sufficient.

Structured Description

In production, free-form descriptions are rarely usable. You will want to guide the model with a structured prompt:

prompt = """Analyze this image and return a JSON with:
- "subject": the main subject in one sentence
- "objects": list of identified objects
- "dominant_colors": the 3 main colors
- "mood": the general mood (bright, dark, warm, etc.)
- "visible_text": any text readable in the image
"""

response = client.responses.create(
    model="grok-4.5",
    input=[
        {"type": "input_image", "image_url": url_image},
        {"type": "input_text", "text": prompt}
    ]
)

This format is ready to use in your application to populate a database, search engine, or catalog.

Example: E-commerce Product Sheets

A highly requested use case is automatically generating product descriptions from photos:

def generate_product_sheet(image_url):
    prompt = """You are an expert e-commerce copywriter. Based on this
product photo, generate:

1. A catchy product title (max 80 characters)
2. A marketing description (2-3 sentences)
3. Visible features (material, color, shape)
4. Relevant SEO keywords (5 maximum)

Respond in JSON."""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_image", "image_url": image_url, "detail": "high"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

Using detail: "high", the model captures fine product details (textures, labels, finishes) that enrich the description.

Example: Alternative Text (Accessibility)

Automatically generating alt attributes for your site’s images improves accessibility and SEO:

def generate_alt_text(image_url):
    prompt = """Generate alternative text (alt attribute) for this
web image. The text must:
- Be concise (max 125 characters)
- Describe the content, not the style
- Not start with "Image of" or "Photo of"
- Be useful for a screen reader

Return only the alt text, without quotes."""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_image", "image_url": image_url, "detail": "low"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

The low mode is sufficient here because alternative text is a general description, not a fine-grained analysis.

Example: Content Moderation

You can use vision to automatically filter images uploaded by your users:

def moderate_image(image_url):
    prompt = """Analyze this image and classify it:
- "safe": content appropriate for all audiences
- "warning": sensitive content but not prohibited
- "reject": inappropriate content

Return a JSON with "classification" and "reason"."""

    response = client.responses.create(
        model="grok-4.5",
        input=[
            {"type": "input_image", "image_url": image_url, "detail": "auto"},
            {"type": "input_text", "text": prompt}
        ],
        store=False
    )
    return response.output_text

Best Practices for Description

• Be precise in your prompt: “Describe this image” produces vague results. “Identify the products, colors, and visible text” is much more effective
• Request a structured format (JSON, list) to facilitate automatic processing
• Adjust the detail level to the need: low for classification, high for fine analysis
• Use store: false for sensitive images or those subject to GDPR

Key Takeaways

• Image description is the fundamental use case for Grok vision
• Structure your prompts to get usable responses (JSON, lists)
• Adjust the detail parameter to the required precision level
• Common applications include e-commerce, accessibility, and moderation
• Remember store: false for sensitive data