Configuring Computer Use in the API

Lesson rewritten November 3, 2026 based on OpenAI’s Computer use guide and Computer use integration recipes: configuration now uses the openai SDK, the OPENAI_API_KEY environment variable, and the {"type": "computer"} tool in the Responses API. It no longer relies on the Anthropic SDK or the legacy computer_20241022 tool. The OpenAI tool does not require screen dimensions; coordinates refer to the screenshot you provide. Sources: Computer use · Integration recipes.

Prerequisites

Before writing a single line of code, your workstation needs four things: a recent version of Python 3, an OpenAI account with an active API key and access to a model that supports the computer tool, a Chromium browser driven by Playwright for web interactions, and Docker if you plan to isolate the agent. Docker is optional while you are learning, but OpenAI recommends an isolated browser or virtual machine as soon as the agent interacts with anything beyond a demo site.

For the model, OpenAI’s documentation uses gpt-6.1-sol in its computer tool examples. Check the model card (under Tools) to confirm that computer use is listed and that your project has access.

Installing dependencies

Work in a dedicated virtual environment. A Computer Use agent pins specific versions of Playwright and its browsers. Mixing these with your other Python projects will lead to conflicts that are painful to diagnose.

# Create and activate the environment
python -m venv computer-use-env
source computer-use-env/bin/activate  # Linux/Mac
# computer-use-env\Scripts\activate   # Windows

# Install dependencies
pip install openai playwright pillow

# Install the Playwright browser
playwright install chromium

Each package plays a distinct role: openai provides the client that talks to the model, playwright drives the browser and executes actions, and pillow handles image manipulation when needed. The last command is the one people forget most often. Without playwright install chromium, the package is installed but no browser is downloaded, causing the first run to fail.

Setting the API key

Store your key in an environment variable, never hard-coded in a script. A secret written in code eventually ends up in a Git repository.

export OPENAI_API_KEY="your-key"

In production, use a secrets manager. The client reads the variable automatically:

from openai import OpenAI

client = OpenAI()  # reads OPENAI_API_KEY

Declaring the computer tool

A model does not control a screen by default. You must enable the computer tool in the request and describe the expected outcome.

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-6.1-sol",
    tools=[{"type": "computer"}],
    input=(
        "Check if the Filters panel is open. If not, "
        "click Show Filters, then type penguin in the search "
        "field. Use the computer tool to interact with the interface."
    ),
)
print(response.output)

The declaration is simply {"type": "computer"}. Unlike the early preview version (computer_use_preview), it requires neither screen dimensions nor an environment type, and automatic truncation is no longer necessary. The coordinates the model returns refer to the screenshot you send. Capture exactly the area you intend to click, at the same scale. If you resize a screenshot before sending it, convert the coordinates back before acting.

Configuring the browser

On the Playwright side, wrap the launch logic in a function you can reuse across tasks.

from playwright.sync_api import sync_playwright

def launch_browser(width=1280, height=720):
    """Launches an isolated browser for Computer Use."""
    pw = sync_playwright().start()
    browser = pw.chromium.launch(headless=True)  # False to watch the agent
    context = browser.new_context(viewport={"width": width, "height": height})
    page = context.new_page()
    return pw, browser, page

page.screenshot() captures the viewport, so the model’s coordinates apply directly to page.mouse. The headless=True option runs the browser without a UI, which is essential on a server. During development, set this to False to watch the agent work. It is the fastest way to understand why it makes mistakes.

Organizing the project

Separate browser logic, the agent loop, and action execution from the start.

computer-use-project/
├── .env                  # OPENAI_API_KEY (never commit)
├── requirements.txt      # openai, playwright, pillow
├── src/
│   ├── __init__.py
│   ├── browser.py        # Launch and capture
│   ├── agent.py          # Computer Use loop
│   └── actions.py        # Action execution
└── tasks/
    └── example.py        # Task scripts

Verifying the installation

Do not build a complex workflow on an unvalidated configuration. If the first run fails, you won’t know whether the issue lies with the key, the browser, or your prompt. This minimal script tests the entire chain: client, browser, capture, and image transmission to the model.

import base64

from openai import OpenAI
from playwright.sync_api import sync_playwright

def test_installation():
    """Verifies that the key, browser, and image upload work."""
    client = OpenAI()

    pw = sync_playwright().start()
    browser = pw.chromium.launch(headless=True)
    page = browser.new_page(viewport={"width": 1280, "height": 720})
    page.goto("https://example.com")
    capture = base64.b64encode(page.screenshot()).decode("utf-8")

    response = client.responses.create(
        model="gpt-6.1-sol",
        input=[{
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Describe what you see."},
                {"type": "input_image", "image_url": f"data:image/png;base64,{capture}"},
            ],
        }],
    )
    print("Response:", response.output_text)

    browser.close()
    pw.stop()

if __name__ == "__main__":
    test_installation()

If the console displays a description of the example.com page, your configuration is ready, and you can proceed to build a full agent.

Key takeaways

• Install openai, playwright, and pillow in a dedicated virtual environment, then run playwright install chromium
• Keep the OPENAI_API_KEY in an environment variable, never in code
• Declare the tool with tools=[{"type": "computer"}] using a supported model (OpenAI examples use gpt-6.1-sol)
• Model coordinates refer to the sent screenshot: capture the area you act on at the same scale
• Always test with a minimal script before building complex workflows