Fill forms and click — Computer Use and Automation

Lesson updated November 3, 2026 based on OpenAI’s confirmation guidelines and Playwright documentation: typing delay is no longer presented as a way to evade anti-bot detection; selection uses ControlOrMeta+a, which also works on macOS; a page.screenshot() does not show the native file open dialog, contrary to what the previous version claimed; guidelines are passed via the instructions parameter. Sources: Confirmation and consent · Playwright, Keyboard.

The most common interaction

If you only master one gesture, make it this one. Submitting a contact request, filling out a quote, entering a customer record into a CRM without an API: forms are the mandatory checkpoint for most automations. This is also where poorly configured agents fail most often, for very concrete reasons that this lesson addresses one by one.

Click before typing

The type action does not write into a field; it writes into the currently focused field. You only forget this nuance once: without a prior click, your text goes into the void or, more insidiously, appends to the previous field. The sequence is therefore invariant: a click to focus, then typing.

# In your action executor
def execute_type(page, text: str):
    """Types text into the focused field."""
    page.keyboard.type(text, delay=50)

The delay=50 deserves attention. keyboard.type already sends keyboard events character by character, but without a delay it chains them as fast as possible; with fifty milliseconds between keystrokes, the rate drops to twenty characters per second. Fields that react to every keystroke—autocomplete, phone number input masks—thus have time to keep up. This setting is not about passing as human: a site seeking to block bots expresses a will that must be respected (Lesson 14).

Clear a field before filling it

A pre-filled form—session resume, default value, browser autocomplete—turns your input into absurd concatenation: typing “Dupont” after “Martin” yields “MartinDupont”, and the form accepts it without error. Select all before typing.

def clear_and_type(page, text: str):
    """Selects all content in the field then types."""
    page.keyboard.press("ControlOrMeta+a")  # Ctrl+A, or Cmd+A on macOS
    page.keyboard.type(text, delay=50)

The model decides its actions based on the prompt: remind it of the rule, or it will type directly into the occupied field.

task = """
Fill out the contact form:
- Field "Name": Dupont
- Field "Email": [email protected]
- Field "Message": Quote request for 50 licenses

IMPORTANT: Before typing in each field, 
select all existing text (Ctrl+A) to replace it.
After filling all fields, click the Send button.
"""

Menus, checkboxes, and radio buttons

A <select> dropdown requires three steps: a click to open it, a wait for the options to appear, and a second click on the desired option. The good news is that custom JavaScript menus—those that aren’t real <select> elements and give classic scripts so much trouble—are handled exactly the same way, since the agent sees only the visual rendering: it clicks, discovers the options in the next screenshot, and clicks the right one.

Checkboxes and radio buttons seem simpler, with a single click enough to activate them. The trap lies elsewhere: a checkbox that is already checked becomes unchecked on click, and your agent has just revoked the consent it was meant to give. It must therefore read the state before acting, and you must explicitly ask it to do so. Also keep in mind that accepting terms or creating an account commits the person the agent works for: entrust these actions to it only if that person has explicitly requested them.

task = """
In the registration form:
1. Check the "I accept the terms" box if it is not checked
2. Select "Company" in the account type
3. Click "Create account"

Verify the state of each element BEFORE clicking.
"""

Multi-screen forms

A three-page registration tunnel is not three times harder than a simple form, provided you structure the instructions page by page and mark the transitions. The agent then knows when a section is finished and which button moves to the next one. The example handles fictitious personal data; with real data, remember that OpenAI considers entering sensitive information into a form as a transmission, requiring approval from the person concerned.

task = """
Fill out the 3-step registration form:

STEP 1 - Personal information:
- First name: Jean
- Last name: Dupont
- Date of birth: 03/15/1985
→ Click "Next"

STEP 2 - Contact details:
- Email: [email protected]
- Phone: 06 12 34 56 78
- Address: 12 rue de la Paix, 75002 Paris
→ Click "Next"

STEP 3 - Confirmation:
- Verify that all information is correct
- Check "I accept the Terms of Service"
→ Click "Validate"
"""

Reacting to validation errors

A rejected phone format, a missing required field, a password deemed too short: the form returns a message and waits for a correction. This is exactly where Computer Use surpasses a script, since the agent knows how to read the displayed error instead of stumbling over it. You still need to ask it to look for errors and bound its attempts so a definitively rejected field doesn’t make it loop endlessly. These rules go into the instructions parameter, returned with every request (Lesson 5).

INSTRUCTIONS = """
After submitting a form:
1. Take a screenshot to see the result
2. If error messages appear (in red, boxed), 
   read them and correct the relevant fields
3. Resubmit the form
4. Repeat until the form is accepted 
   or you have tried 3 times
"""

File upload, a special case

There is one limit no prompt will cross: the file open dialog belongs to the operating system, not the page. The agent doesn’t even see it—page.screenshot() captures only page content—and the coordinates it produces apply to the page, not a native window. Hand the operation off to Playwright, which bypasses this dialog entirely.

def handle_file_upload(page, file_path: str, selector: str):
    """Uploads a file via Playwright (not via Computer Use)."""
    page.locator(selector).set_input_files(file_path)

You can also let the agent click “Browse” like a user and intercept the file selector the moment it opens: the workflow remains natural end-to-end, with the system part handled behind the scenes. In both cases, your code chooses the file, never the model; and sending files is among the actions OpenAI allows you to pre-approve in the initial request, or confirm otherwise.

# The agent clicks "Browse", then you intercept
page.on("filechooser", lambda fc: fc.set_files("/path/to/file.pdf"))

Key takeaways

Always click a field before typing into it
Use Ctrl+A (ControlOrMeta+a in Playwright) to clear a field before typing
Dropdown menus require a click to open, then a click to select
Plan for validation error handling in the prompt
File uploads must be handled by Playwright, not Computer Use