Skip to main content
The HostRunner class runs prompts with MCP tools enabled. It handles the multi-step prompt loop and returns rich result objects.

Import

Constructor

Parameters

options
HostRunnerOptions
required
Configuration for the test agent.

HostRunnerOptions

Example

MCP Apps, traces, and mcpClientManager: MCP App widgets are backed by HTML served from an MCP resource (ui.resourceUri), not by the tool’s JSON result alone. After each tool call, HostRunner uses the manager’s readResource to fetch that HTML and attach widgetSnapshots to the PromptResult. When you upload results to MCPJam, those snapshots are stored so the Evals trace viewer can replay the widget offline. If you omit mcpClientManager, prompts still run and tools still execute — but widgetSnapshots stay empty and uploaded traces will only contain messages and spans (no embedded app replay). Passing manager is also what enables replay credential capture for authenticated HTTP servers when reporting to MCPJam.

Methods

run()

Sends a prompt to the LLM and returns the result.
Use getWidgetSnapshots() on the returned result (or toEvalResult() when reporting) to access MCP App HTML captured during the run when mcpClientManager was set.

Parameters

PromptOptions

Returns

Promise<PromptResult> - The result object with response and metadata.

Example

run() never throws exceptions. Errors are captured in the PromptResult. Check result.hasError() to detect failures.
stopAfterToolCall is intended for evals that only care about tool selection and arguments. If the model emits multiple tool calls in the same step, non-target sibling tools may still execute before the loop stops.
timeout bounds prompt runtime. The runtime creates an internal abort signal, so tools can stop early if their implementation respects the provided abortSignal. If a tool ignores that signal, its underlying work may continue briefly after the prompt returns an error result.

Model String Format

Models are specified as provider/model:

Custom Providers

Add custom OpenAI or Anthropic-compatible endpoints.

CustomProvider Type

Example


Configuration Properties

tools

The MCP tools available to the agent. Obtained from MCPClientManager.getTools().

model

The LLM model identifier. Format: provider/model-id.

apiKey

The API key for the LLM provider.

systemPrompt

Optional system prompt to guide the LLM’s behavior.

temperature

Controls response randomness. Range: 0.0 (deterministic) to 1.0 (creative).

maxSteps

Maximum iterations of the agentic loop (prompt → tool → result → continue).
Setting maxSteps too low may prevent complex tasks from completing. Setting it too high may allow runaway loops.

injectOpenAiCompat

Controls whether the OpenAI Apps SDK window.openai shim is injected into captured widget HTML snapshots. Defaults to false. Set this to true when your widget is written against the OpenAI Apps SDK (i.e. it targets ChatGPT or Copilot). Leave it false for SEP-1865-native widgets targeting Claude, Cursor, Codex, or other hosts that don’t expose window.openai — snapshots should match what the live host would have produced.
This flag is stamped onto each widget snapshot (injectedOpenAiCompat) so the MCPJam trace viewer can replay the widget faithfully under a different host config without rewriting the cached bytes.

Control Multi-Step Loops with stopWhen

Use stopWhen to control whether the agent starts another step after the current step completes.
stopWhen does not skip tool execution. It controls whether the prompt loop continues after the current step completes, and HostRunner also applies stepCountIs(maxSteps) as a safety guard.

Bound Prompt Runtime with timeout

Use timeout when you want to bound how long HostRunner.run() can run:
chunkMs is accepted for parity, but it is mainly useful for streaming APIs. For HostRunner.run(), number, totalMs, and stepMs are the main settings to focus on.

Complete Example