Content Format

Grok’s Two Input Formats

When you send a message to Grok, the content field in each message can take one of two distinct forms. Understanding this distinction matters because it determines what you can send the model: text alone, or a mix of text and images.

Simple Text String

The most direct form: the content is a simple string. You’ll use this for the majority of your text-based interactions.

{
  "role": "user",
  "content": "Explain the advantages of the Rust language compared to C++"
}

This format works for all purely text-based tasks: questions, instructions, providing data, or generation requests. Use this format when you don’t need to include visual elements.

Typed Object Array

For multimodal requests (text + images), Grok accepts an array of objects where each element includes a type field indicating its nature.

Chat Completions Format (Legacy Endpoint)

{
  "role": "user",
  "content": [
    {"type": "text", "text": "Describe this user interface and suggest improvements"},
    {"type": "image_url", "image_url": {"url": "https://exemple.com/screenshot.png", "detail": "high"}}
  ]
}

Responses Format (Primary Endpoint)

{
  "input": [
    {"type": "input_text", "text": "Analyze this chart and extract the main trends"},
    {"type": "input_image", "image_url": "https://exemple.com/graphique.png"}
  ]
}

Note the naming difference between the two endpoints: text and image_url for Chat Completions, input_text and input_image for Responses.

Combining Text and Images

The object array lets you freely mix text and images in any order. You can send multiple images in the same request, with text before, after, or between them.

{
  "role": "user",
  "content": [
    {"type": "text", "text": "Compare these two interface mockups:"},
    {"type": "image_url", "image_url": {"url": "https://exemple.com/maquette-v1.png"}},
    {"type": "text", "text": "Version 1 above, version 2 below:"},
    {"type": "image_url", "image_url": {"url": "https://exemple.com/maquette-v2.png"}},
    {"type": "text", "text": "Identify the main differences and recommend the best version."}
  ]
}

Accepted Image Sources

Grok accepts images in two ways:

• Direct URL: an HTTPS link to the image hosted online
• Base64 encoded: the image embedded directly in the request as a data URI

{"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,/9j/4AAQ..."}}

The Base64 format is useful when your images aren’t hosted online (local screenshots, images generated by your application).

The detail Parameter for Images

Each image can receive a detail parameter that controls the analysis level:

• auto: the model chooses automatically (default behavior)
• low: fast and economical analysis, consumes fewer tokens
• high: detailed analysis, consumes more tokens but captures fine details

{"type": "image_url", "image_url": {"url": "https://...", "detail": "low"}}

Use low for simple tasks (classification, general description) and high for precise analyses (OCR, detail detection, reading charts).

Technical Constraints

Keep these limits in mind when working with images:

• Maximum size: 20 MB per image
• Accepted formats: JPG, JPEG, PNG only
• No limit on the number of images per request
• Compatible models: grok-4.5 and higher
• Images consume between 256 and 1,792 tokens depending on their size and detail level

Hands-On Practice

Test both formats with a simple task:

Simple text format:

{"role": "user", "content": "What are the 5 fundamental principles of UX design?"}

Array format (text only, but structured):

{
  "role": "user",
  "content": [
    {"type": "text", "text": "Context: I'm designing a mobile food delivery application."},
    {"type": "text", "text": "Question: what are the 5 most important UX principles for this type of application?"}
  ]
}

Even without an image, the array format lets you clearly separate context from the question, which can improve response relevance.

Key Takeaways

• Content can be a simple string (plain text) or an object array (multimodal)
• The array format lets you mix text and images in any order
• Two naming formats exist depending on the endpoint: Chat Completions vs Responses
• The detail parameter controls the depth of image analysis (low, auto, high)
• Images consume tokens: adapt the detail level to your needs
• The Base64 format lets you send images without hosting them online