> ## Documentation Index > Fetch the complete documentation index at: https://docs.mcpjam.com/llms.txt > Use this file to discover all available pages before exploring further. # Launch one run per target as a group (async) > Launches ONE run per target — attached project environments, or attached named hosts — under a single **server-minted** `runGroupId`, then detaches execution and responds `202` with a per-target receipt. This is the only endpoint with grouped-launch semantics. `POST /eval-runs` also accepts a `runGroupId`, but purely as a display label: it groups sibling rows in the UI and gives N separate launches no group treatment. Here the server bounds the fan-out (max 10 targets), validates EVERY target before launching any of them (an unattached target, a heterogeneous target list, or a target whose host selects an unavailable harness returns `400` with zero runs started), and holds exactly ONE organization concurrency slot for the whole group — released only when the last sibling finishes. Targets fan out along ONE axis: all `environmentId` or all `namedHostId`, never a mix. Duplicates are deduplicated by id, preserving order. A per-target failure does NOT abort its siblings, so read `outcome` rather than treating the `202` as "everything started": `started` means every target launched, `partial` means some did, `failed` means none did. Send `Idempotency-Key` (or `idempotencyKey`) to make the launch replayable: the group id is DERIVED from the key, and each target carries a run key derived from that, so a retry after a crash mid-launch returns the original run ids instead of double-launching the targets that already started. ## OpenAPI ````yaml /reference/openapi.json post /projects/{projectId}/eval-run-groups openapi: 3.1.0 info: title: MCPJam API version: 1.0.0-preview description: >- Programmatic access to MCP servers saved in your MCPJam projects — live diagnostics (validate, inspect, export) and operations: call tools, render prompts, run eval suites asynchronously and poll their results, and import OAuth tokens. **The API is in preview**: the surface may change while we finish the design. Error `code` values are stable; error `message` strings are not. Write clients that ignore unknown response fields. contact: name: MCPJam url: https://github.com/MCPJam/inspector/issues servers: - url: https://app.mcpjam.com/api/v1 description: Hosted MCPJam security: - bearerAuth: [] tags: - name: Hosts description: >- Project hosts: named model + capability profiles you run chats and eval suites against. - name: Environments description: >- Project environments: named, live-editable execution bundles (one host, an optional standalone server group, optionally pinned skills and plugin versions) that eval suites and journeys run against. Distinct from Sandbox images, which are Computer base images. Reads require project membership; every write requires project admin. - name: Plugins description: >- Agent Plugins imported into a project — read-only inventory and version detail. - name: Sandbox images description: >- Custom Computer images: a digest-pinned Dockerfile built into an immutable image your project's computers boot from. - name: Server diagnostics description: Connect-level health checks against a saved MCP server. - name: Primitives description: 'The server''s MCP primitives: tools, prompts, and resources.' - name: Export description: Full-server snapshots for diffing and CI. - name: Execution description: 'Run the server''s primitives: call tools, render prompts.' - name: Eval runs description: >- Asynchronous eval suite runs: create with 202, poll status, iterations, and traces. - name: Server connections description: >- Connect an MCP server URL to a project, authorizing in a browser when the server requires it. - name: OAuth description: 'Bring-your-own OAuth: import externally obtained tokens for a server.' - name: Scenarios description: >- Read-only access to the scenarios published from a project: listing, settings, attached servers, and share links. - name: Catalog description: >- Discover the resources the other routes operate on: your account, projects, servers, eval suites, and chat sessions. - name: Tunnels description: >- Relay tunnels that expose local MCP servers through a public URL, registered as first-class project servers (the `mcpjam tunnel` CLI flow). - name: Agent description: >- Headless agent turns over the public API: send a message history, the server runs one assistant turn with project-scoped workspace tools (eval reads + suite creation) on a pinned hosted model, and returns the reply plus created-resource references. - name: Swarms description: >- Personas, journeys and swarm containers — the authoring half of Swarms — plus the model-backed generation that drafts them. - name: Swarm runs description: >- Launching journeys and reading what they produced. Launching SPENDS — see the per-operation notes. - name: Swarm insights description: >- What a swarm run revealed. The scorecard and findings are deterministic and free; requesting wave insights runs models and draws on your shared daily ledger. - name: User testing description: >- Publishing an environment for real visitors, and controlling who can reach it. Several of these NARROW access and take effect immediately. paths: /projects/{projectId}/eval-run-groups: post: tags: - Eval runs summary: Launch one run per target as a group (async) description: >- Launches ONE run per target — attached project environments, or attached named hosts — under a single **server-minted** `runGroupId`, then detaches execution and responds `202` with a per-target receipt. This is the only endpoint with grouped-launch semantics. `POST /eval-runs` also accepts a `runGroupId`, but purely as a display label: it groups sibling rows in the UI and gives N separate launches no group treatment. Here the server bounds the fan-out (max 10 targets), validates EVERY target before launching any of them (an unattached target, a heterogeneous target list, or a target whose host selects an unavailable harness returns `400` with zero runs started), and holds exactly ONE organization concurrency slot for the whole group — released only when the last sibling finishes. Targets fan out along ONE axis: all `environmentId` or all `namedHostId`, never a mix. Duplicates are deduplicated by id, preserving order. A per-target failure does NOT abort its siblings, so read `outcome` rather than treating the `202` as "everything started": `started` means every target launched, `partial` means some did, `failed` means none did. Send `Idempotency-Key` (or `idempotencyKey`) to make the launch replayable: the group id is DERIVED from the key, and each target carries a run key derived from that, so a retry after a crash mid-launch returns the original run ids instead of double-launching the targets that already started. operationId: createEvalRunGroup parameters: - $ref: '#/components/parameters/projectId' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/EvalRunGroupCreateRequest' example: suiteId: suite_def456 targets: - namedHostId: host_claude - namedHostId: host_chatgpt iterationOverride: 3 responses: '202': description: >- Group accepted. Each target's outcome is in `targets`; execution continues in the background. content: application/json: schema: $ref: '#/components/schemas/EvalRunGroupCreated' example: runGroupId: 6f1a2b3c4d5e6f708192a3b4c5d6e7f8 suiteId: suite_def456 outcome: partial startedCount: 1 failedCount: 1 targets: - target: namedHostId: host_claude name: Claude status: started runId: run_abc123 runStatus: running - target: namedHostId: host_chatgpt name: ChatGPT status: failed error: code: VALIDATION_ERROR message: Suite has no saved server selection to rerun against. runId: run_abc123 status: running '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '502': $ref: '#/components/responses/ServerUnreachable' '504': $ref: '#/components/responses/Timeout' components: parameters: projectId: name: projectId in: path required: true description: ID of the hosted project that contains the server. schema: type: string schemas: EvalRunGroupCreateRequest: type: object required: - suiteId - targets description: >- Launch one run per target under a single server-minted group id. `targets` fan out along ONE axis: every entry is an `environmentId`, or every entry is a `namedHostId`. A mixed list returns `400` with `details.reason: "HETEROGENEOUS_TARGETS"` — mixing them would have to answer whether a host runs inside an environment or beside it, and every answer to that is a cross product nobody asked for. Duplicates are deduplicated by id, preserving order. The knobs below apply UNIFORMLY to every run in the group. `serverIds` and `refreshSnapshot` are deliberately absent: an environment supplies its own closed server set, a host derives one from its config, and last-writer-wins on a frozen suite snapshot is never what a fan-out means. Inline `tests` are absent for the same reason — a group runs a persisted suite. properties: suiteId: type: string description: The suite every target runs. targets: type: array minItems: 1 maxItems: 10 description: >- One entry per run. Every target must be ATTACHED to the suite; an unattached one returns `400` with zero runs started. items: type: object oneOf: - required: - environmentId - required: - namedHostId properties: environmentId: type: string description: A project environment in the suite's `environmentIds`. namedHostId: type: string description: A host attached to the suite. iterationOverride: type: integer minimum: 1 maximum: 10 description: Override the per-case `runs` count for every run in the group. caseIds: type: array minItems: 1 description: Narrow every run in the group to these suite cases. items: type: string matchOptionsOverride: description: >- Match options for every run in the group. Same public-or-internal vocabulary as `POST /eval-runs`. type: object skillsOverride: type: string enum: - exclude description: Run every target as the "without skills" arm. notes: type: string passCriteria: type: object properties: minimumPassRate: type: number idempotencyKey: type: string maxLength: 256 description: >- Makes the whole launch replayable. The `runGroupId` is DERIVED from this key and each target's run key is derived from that, so a retry after a crash mid-launch returns the original run ids rather than double-launching the targets that already started. The `Idempotency-Key` header carries the same value and wins over this field. additionalProperties: false EvalRunGroupCreated: type: object required: - runGroupId - suiteId - outcome - startedCount - failedCount - targets description: >- One entry per target. A per-target failure does NOT abort its siblings, so read `outcome` rather than treating the `202` as "everything started". properties: runGroupId: type: string description: >- Server-minted. Every sibling run carries it, and the hosted UI collapses them into one row. suiteId: type: string outcome: type: string description: >- `started` — every target launched; `partial` — some did and some did not; `failed` — none did (still a `202`: the group itself was valid, and the per-target reasons are in `targets`). enum: - started - partial - failed startedCount: type: integer failedCount: type: integer targets: type: array description: >- Discriminated on `status`, so a target that failed can never be mistaken for one that started with an unread `runId`. items: type: object required: - target - status properties: target: type: object description: Which target this entry is for. Exactly one id is set. properties: environmentId: type: string namedHostId: type: string name: type: string status: type: string enum: - started - failed runId: type: string description: Present when `status` is `started`. runStatus: type: string description: >- The RUN's status. `running` for a target that just launched; on a target whose run was REPLAYED (see `deduped`), that run's own status. Named apart from the entry's own `status` on purpose — two fields called `status` in one object is how a reader ends up branching on the wrong one. deduped: type: boolean description: >- Present and true when this target REPLAYED an existing run instead of starting one. A replayed run is not executed again, so the target costs no further credits. servers: type: array items: type: object properties: id: type: string name: type: string environment: oneOf: - $ref: '#/components/schemas/EvalRunEnvironment' - type: 'null' caseUpsert: type: object error: type: object description: >- Present when `status` is `failed`. The code is preserved so a caller can tell a retryable conflict from a genuine internal error. properties: code: type: string message: type: string runId: type: string deprecated: true description: >- Mirror of the FIRST started run, so readers written against the single-run receipt keep working. Absent when nothing started. Read `targets` instead — this describes one run out of several. status: type: string deprecated: true description: See `runId`. servers: type: array deprecated: true description: See `runId`. items: type: object properties: id: type: string name: type: string environment: deprecated: true description: See `runId`. oneOf: - $ref: '#/components/schemas/EvalRunEnvironment' - type: 'null' caseUpsert: type: object deprecated: true description: See `runId`. EvalRunEnvironment: type: - object - 'null' description: >- The project environment a run is pinned to, at the revision resolved when it launched. `null` for a legacy run that used the suite's saved server selection. required: - id properties: id: type: string name: type: - string - 'null' revision: type: - integer - 'null' description: The environment revision the run executed against. Error: type: object required: - code - message properties: code: type: string description: >- Stable, machine-readable error code. New codes may be added over time; treat unknown codes as non-retryable failures unless the HTTP status says otherwise. enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - CONFLICT - VALIDATION_ERROR - RATE_LIMITED - FEATURE_NOT_SUPPORTED - SERVER_UNREACHABLE - TIMEOUT - OAUTH_REQUIRED - INTERNAL_ERROR message: type: string description: >- Human-readable description. May change between releases — don't match on it. details: type: object description: Optional, unstructured context bag. additionalProperties: true responses: ValidationError: description: Malformed body or parameters. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: VALIDATION_ERROR message: Invalid JSON body Unauthorized: description: >- Missing, invalid, revoked, or orphaned key (`UNAUTHORIZED`) — or the **target MCP server** needs an OAuth grant (`OAUTH_REQUIRED`), which is a property of the server, not your key. content: application/json: schema: $ref: '#/components/schemas/Error' examples: badKey: summary: Invalid or revoked key value: code: UNAUTHORIZED message: Invalid API key oauthRequired: summary: Target server needs an OAuth grant value: code: OAUTH_REQUIRED message: Server requires OAuth authorization Forbidden: description: Key is valid but not allowed to do this. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: FORBIDDEN message: You do not have access to this project NotFound: description: Unknown project, server, or resource. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: NOT_FOUND message: Server not found RateLimited: description: >- Per-key rate limit exceeded (60 requests/minute sustained, bursts up to 10). Honor `Retry-After` and back off with jitter. headers: Retry-After: description: Seconds to wait before retrying. schema: type: integer content: application/json: schema: $ref: '#/components/schemas/Error' example: code: RATE_LIMITED message: API key rate limit exceeded. Slow down and retry. InternalError: description: Something failed on MCPJam's side. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: INTERNAL_ERROR message: Unexpected internal error ServerUnreachable: description: Could not connect to the target MCP server. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: SERVER_UNREACHABLE message: Failed to connect to server Timeout: description: The target MCP server connected but didn't respond in time. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: TIMEOUT message: Request to server timed out securitySchemes: bearerAuth: type: http scheme: bearer description: >- MCPJam API key (`sk_…`). Create one at [Settings → API keys](https://app.mcpjam.com/settings/api-keys). Guest sessions cannot use the API, and API keys cannot manage other API keys. ````