Test and Debug
Lesson rewritten on November 3, 2026 based on the Connect and test your plugin and Troubleshooting pages from the OpenAI Plugins documentation. The former “development playground” and its commands belonged to the non-existent SDK from the first version of this course. Sources: Connect and test · Troubleshooting.
Test in layers
OpenAI recommends testing each capability before deploying the full installed plugin: first the standalone MCP server connected to ChatGPT, then the packaged plugin with its skills. Keep your evaluation prompts and their results throughout development so you can compare behavior across versions.
The MCP server
Prepare the endpoint
Before testing in ChatGPT, verify that the server is reachable via public HTTPS—or through a Secure MCP Tunnel—that it speaks streamable HTTP (typically on /mcp), that tool names, descriptions, schemas, and annotations are present, and that authentication discovery works for tools that require an account. A development tunnel is fine for trials, but not for submission.
Inspect
MCP Inspector calls your tools directly, without a model:
npx @modelcontextprotocol/inspector@latestPut every tool through its paces: representative inputs, edge cases, missing credentials, and empty results. Check schema validation, authentication errors, annotations, confirmation behavior, and model-readable output.
Evaluate tool selection
Once the server is connected as a personal plugin (Lesson 4), install it, open a new conversation, type @, and select it. Build an evaluation set containing:
- Direct requests that should trigger a specific tool;
- Indirect requests that express the same goal differently;
- Follow-up requests that reuse identifiers obtained earlier;
- Write operations that require authorization or confirmation;
- Out-of-scope requests that should not trigger any tool.
For each, record the selected tool, its arguments, the result, any errors, and confirmation behavior. Replay the entire set whenever you change a name, description, schema, or annotation. If the server returns a UI component, test both the component itself and the model-readable result.
View raw exchanges
To read full requests and responses, the API Playground accepts an MCP server: go to Tools → Add → MCP Server, enter the HTTPS address, connect, then run your test prompts.
Refresh metadata
After any change to a name, description, schema, annotation, authentication method, or UI resource: redeploy or restart the server, open the connection at chatgpt.com/plugins, select Refresh, verify that the advertised metadata has changed, and replay the relevant tests in a new conversation. This refresh applies to directly connected servers; for a published plugin, tool updates fall under continuous review (Lesson 11).
Before packaging, verify that the tool list matches the advertised capabilities, that structured results comply with output schemas, that authentication failures return useful errors, that the right prompts trigger the right tools while wrong ones trigger none, and that the UI renders without console errors while correctly restoring its state.
The full plugin
Once the server is ready—or immediately for a plugin consisting only of skills—package the plugin (Lesson 10), add it to a local marketplace, install it from the plugins directory, then replay the requests from your use-case inventory: direct, indirect, follow-up, negative, and intentionally excluded edge cases. Verify that the plugin follows skill instructions, uses the correct resources, completes each step, and produces a useful result; that skills call the correct tools; and that authentication works after installation.
Diagnose by layer
When something goes wrong, start by isolating the faulty layer: server, component, or ChatGPT client.
| Symptom | Lead |
|---|---|
| No tools listed | Is the server running? Does the address end in /mcp? |
| Structured data, but no component | Does the tool include _meta.ui.resourceUri pointing to a registered resource of type text/html;profile=mcp-app that loads without CSP errors? |
| Schema errors | Do your Python or TypeScript models match the advertised outputSchema? |
| Slow component | Do tool calls exceed a few hundred milliseconds? Profile and cache. |
| Component fails to load | Browser console: CSP violation, incomplete bundle? |
| State lost (drag-and-drop, editing) | Call window.openai.setWidgetState after every update and restore from window.openai.widgetState on mount. |
| Layout broken on mobile | Read window.openai.displayMode and window.openai.maxHeight; avoid fixed heights and hover actions. |
| Tool never selected | Rewrite the description as “Use when…”, review starter prompts. |
| Wrong tool selected | Clarify neighboring tools, specify excluded cases, split overly broad tools. |
| 401 errors | Include WWW-Authenticate in the response; verify issuer and audience. |
invalid_client on an existing connection | Does the registered OAuth client still exist? Restore it rather than creating a new one. |
| ngrok tunnel expires | Restart it, verify the local server is running; in production, use stable hosting with health probes. |
| Streaming broken behind a proxy | The load balancer or CDN must pass HTTP streaming through without buffering. |
If the problem persists, gather server logs, the component console, a transcript of tool calls, and screenshots. Note the prompt used and send everything to your contact at OpenAI.
Key takeaways
- Test in layers: server alone (Inspector, then ChatGPT), then the full plugin
- Build an evaluation set: direct, indirect, follow-up, write, and out-of-scope requests
- The API Playground shows raw MCP exchanges
- Refresh after every metadata change, then start a new conversation
- When issues arise, isolate the layer first: server, component, or client