Skip to main content
The HostRunner class runs prompts with MCP tools enabled. It handles the multi-step prompt loop and returns rich result objects.

Import

Constructor

Parameters

HostRunnerOptions
required
Configuration for the test agent.

HostRunnerOptions

Example

MCP Apps, traces, and mcpClientManager: MCP App widgets are backed by HTML served from an MCP resource (ui.resourceUri), not by the tool’s JSON result alone. After each tool call, HostRunner uses the manager’s readResource to fetch that HTML and attach widgetSnapshots to the PromptResult. When you upload results to MCPJam, those snapshots are stored so the Evals trace viewer can replay the widget offline. If you omit mcpClientManager, prompts still run and tools still execute — but widgetSnapshots stay empty and uploaded traces will only contain messages and spans (no embedded app replay). Passing manager is also what enables replay credential capture for authenticated HTTP servers when reporting to MCPJam.

Methods

run()

Sends a prompt to the LLM and returns the result.
Use getWidgetSnapshots() on the returned result (or toEvalResult() when reporting) to access MCP App HTML captured during the run when mcpClientManager was set.

Parameters

PromptOptions

Returns

Promise<PromptResult> - The result object with response and metadata.

Example

run() never throws exceptions. Errors are captured in the PromptResult. Check result.hasError() to detect failures.
stopAfterToolCall is intended for evals that only care about tool selection and arguments. If the model emits multiple tool calls in the same step, non-target sibling tools may still execute before the loop stops.
timeout bounds prompt runtime. The runtime creates an internal abort signal, so tools can stop early if their implementation respects the provided abortSignal. If a tool ignores that signal, its underlying work may continue briefly after the prompt returns an error result.

Model String Format

Models are specified as provider/model:

Custom Providers

Add custom OpenAI or Anthropic-compatible endpoints.

CustomProvider Type

Example


Configuration Properties

tools

The MCP tools available to the agent. Obtained from MCPClientManager.getTools().

model

The LLM model identifier. Format: provider/model-id.

apiKey

The API key for the LLM provider.

systemPrompt

Optional system prompt to guide the LLM’s behavior. An empty string ("") is treated the same as omitting the field — both fall back to "You are a helpful assistant.". This matters for saved MCPJam clients, which store a missing prompt as "".

temperature

Controls response randomness. Range: 0.0 (deterministic) to 1.0 (creative).

reasoningEffort

Reasoning effort for the model. The runner maps it through the same table the hosted runner uses (reasoningEffortProviderOptions) into the provider’s own providerOptions — OpenAI reasoningEffort, Anthropic effort (with adaptive thinking on models that need it), Google thinkingConfig.thinkingLevel — and also works for mcpjam/… models, which speak Anthropic’s and OpenAI’s own wire formats.
  • Refused, never dropped. A model or provider with no effort control the installed AI SDK provider exposes (for example openai/gpt-4o) makes the constructor throw capability_missing, before any request is made. withOptions carries the effort with the runner and re-checks it against a new model.
  • Replaces the temperature. While an effort is set the runner does not send a temperature, and the recorded context reports what was sent.
  • Saved clients. runWithClient applies the effort on the saved client’s hosted model selection (settings.reasoningEffort) — and its saved settings.temperature — instead of discarding them with the selection. A host snapshot carries no model selection, so pass reasoningEffort yourself when you build a runner from a Host.
  • Which levels a model accepts: supportedReasoningEfforts on the model catalog (GET /v1/models) for hosted models, and supportedReasoningEfforts() from @mcpjam/sdk/host-config for provider models.

maxSteps

Maximum iterations of the agentic loop (prompt → tool → result → continue).
Setting maxSteps too low may prevent complex tasks from completing. Setting it too high may allow runaway loops.

injectOpenAiCompat

Controls whether the OpenAI Apps SDK window.openai shim is injected into captured widget HTML snapshots. Defaults to false. Set this to true when your widget is written against the OpenAI Apps SDK (i.e. it targets ChatGPT or Copilot). Leave it false for SEP-1865-native widgets targeting Claude, Cursor, Codex, or other hosts that don’t expose window.openai — snapshots should match what the live host would have produced.
This flag is stamped onto each widget snapshot (injectedOpenAiCompat) so the MCPJam trace viewer can replay the widget faithfully under a different host config without rewriting the cached bytes.

Control Multi-Step Loops with stopWhen

Use stopWhen to control whether the agent starts another step after the current step completes.
stopWhen does not skip tool execution. It controls whether the prompt loop continues after the current step completes, and HostRunner also applies stepCountIs(maxSteps) as a safety guard.

Bound Prompt Runtime with timeout

Use timeout when you want to bound how long HostRunner.run() can run:
chunkMs is accepted for parity, but it is mainly useful for streaming APIs. For HostRunner.run(), number, totalMs, and stepMs are the main settings to focus on.

Complete Example