> ## Documentation Index > Fetch the complete documentation index at: https://docs.mcpjam.com/llms.txt > Use this file to discover all available pages before exploring further. # Get a run's decision summary > The canonical, versioned run decision contract: the verdict and where it came from, the counts with the population they count, the run's own verdict decision when it has one, and one page of per-trial diagnostics carrying the user-value chain, the first failed stage, the failure category, evidence scoped to that stage, and one next action. ADDITIVE and composed: it is the same two reads a caller could make by hand (`GET …/eval-runs/{runId}` and `GET …/iterations`), assembled once here so every client shares one reading of a run instead of inventing its own. `cursor` and `limit` page the DIAGNOSTICS using the same cursors the iterations endpoint issues, and `diagnostics.complete` says honestly whether the page you got is the run's whole non-passing set. ## OpenAPI ````yaml /reference/openapi.json get /projects/{projectId}/eval-runs/{runId}/decision-summary openapi: 3.1.0 info: title: MCPJam API version: 1.0.0-preview description: >- Programmatic access to MCP servers saved in your MCPJam projects — live diagnostics (validate, inspect, export) and operations: call tools, render prompts, run eval suites asynchronously and poll their results, and import OAuth tokens. **The API is in preview**: the surface may change while we finish the design. Error `code` values are stable; error `message` strings are not. Write clients that ignore unknown response fields. contact: name: MCPJam url: https://github.com/MCPJam/inspector/issues servers: - url: https://app.mcpjam.com/api/v1 description: Hosted MCPJam security: - bearerAuth: [] tags: - name: Hosts description: >- Project hosts: named model + capability profiles you run chats and eval suites against. - name: Environments description: >- Project environments: named, live-editable execution bundles (one host, an optional standalone server group, optionally pinned skills and plugin versions) that eval suites and journeys run against. Distinct from Sandbox images, which are Computer base images. Reads require project membership; every write requires project admin. - name: Plugins description: >- Agent Plugins imported into a project — read-only inventory and version detail. - name: Sandbox images description: >- Custom Computer images: a digest-pinned Dockerfile built into an immutable image your project's computers boot from. - name: Server diagnostics description: Connect-level health checks against a saved MCP server. - name: Primitives description: 'The server''s MCP primitives: tools, prompts, and resources.' - name: Export description: Full-server snapshots for diffing and CI. - name: Execution description: 'Run the server''s primitives: call tools, render prompts.' - name: Eval runs description: >- Asynchronous eval suite runs: create with 202, poll status, iterations, and traces. - name: Conformance runs description: >- Ingest MCP spec-conformance results from the SDK/CLI into project-owned history. Distinct from Eval runs (authored LLM cases) and from directory readiness. - name: Server connections description: >- Connect an MCP server URL to a project, authorizing in a browser when the server requires it. - name: OAuth description: 'Bring-your-own OAuth: import externally obtained tokens for a server.' - name: Scenarios description: >- Read-only access to the scenarios published from a project: listing, settings, attached servers, and share links. - name: Catalog description: >- Discover the resources the other routes operate on: your account, projects, servers, eval suites, and chat sessions. - name: Tunnels description: >- Relay tunnels that expose local MCP servers through a public URL, registered as first-class project servers (the `mcpjam cloud tunnel` CLI flow). - name: Agent description: >- Headless agent turns over the public API: send a message history, the server runs one assistant turn with project-scoped workspace tools (eval reads + suite creation) on a pinned hosted model, and returns the reply plus created-resource references. - name: Swarms description: >- Personas, journeys and swarm containers — the authoring half of Swarms — plus the model-backed generation that drafts them. - name: Swarm runs description: >- Launching journeys and reading what they produced. Launching SPENDS — see the per-operation notes. - name: Swarm insights description: >- What a swarm run revealed. The scorecard and findings are deterministic and free; requesting wave insights runs models and draws on your shared daily ledger. - name: User testing description: >- Publishing an environment for real visitors, and controlling who can reach it. Several of these NARROW access and take effect immediately. - name: Directory readiness description: >- Grade a saved server against a publisher's listing requirements: Anthropic's connector directory or OpenAI's plugin directory. Reported as lane status and coverage, never as a numeric score, and excluded from `pooledConformanceScore`. Deterministic grading is free; model-backed experience observations are an explicit opt-in that consumes MCPJam credits and can never decide a verdict. - name: Registry description: >- Search the scraped MCP directories (Claude, ChatGPT, and any future source), list curated/org registry cards, and install them into a project. Install writes a `servers` row and provenance — it does not open a live session. There is no catalog-uninstall route: delete the project server instead. Directory reads require a bearer (including minted guest tokens) but do not materialize a user. Card/connection reads and all writes are authed-non-guest. paths: /projects/{projectId}/eval-runs/{runId}/decision-summary: get: tags: - Eval runs summary: Get a run's decision summary description: >- The canonical, versioned run decision contract: the verdict and where it came from, the counts with the population they count, the run's own verdict decision when it has one, and one page of per-trial diagnostics carrying the user-value chain, the first failed stage, the failure category, evidence scoped to that stage, and one next action. ADDITIVE and composed: it is the same two reads a caller could make by hand (`GET …/eval-runs/{runId}` and `GET …/iterations`), assembled once here so every client shares one reading of a run instead of inventing its own. `cursor` and `limit` page the DIAGNOSTICS using the same cursors the iterations endpoint issues, and `diagnostics.complete` says honestly whether the page you got is the run's whole non-passing set. operationId: getEvalRunDecisionSummary parameters: - $ref: '#/components/parameters/projectId' - $ref: '#/components/parameters/runId' - name: limit in: query required: false description: Iterations examined per page, 1–200. Defaults to 50. schema: type: integer minimum: 1 maximum: 200 default: 50 - name: cursor in: query required: false description: >- Opaque cursor from a previous response's `diagnostics.nextCursor`. A page fetched with one is never reported as complete. schema: type: string responses: '200': description: The run's decision summary. content: application/json: schema: $ref: '#/components/schemas/EvalRunDecisionSummary' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' components: parameters: projectId: name: projectId in: path required: true description: ID of the hosted project that contains the server. schema: type: string runId: name: runId in: path required: true description: Eval run ID, as returned by `POST /eval-runs`. schema: type: string schemas: EvalRunDecisionSummary: type: object description: >- The canonical run decision contract (schema version 1): what the run decided, in what unit it counted, and what evidence sits under the non-passing trials. START HERE when a run did not pass. It EXPLAINS the verdict and never DECIDES it. Under verdict policy v2 the authority is the run's own decision — its verdict, rates, validity phase, reasons, per-case stability and mixed-verdict flags are copied after validation and never recomputed from the trials; `counts` is a tally of `decision.cases[].verdict` and nothing more. `verdict` has a fourth value the run verdict vocabulary does not: `notEstablished` means no verdict exists at all (the run is unfinished, stopped early, or its decision could not be read). It is not a failure and not `inconclusive` — `inconclusive` is a decision the validity phase reached, and this is the absence of any decision. required: - schemaVersion - runId - runStatus - verdict - verdictSource - diagnostics properties: schemaVersion: type: integer enum: - 1 runId: type: string runStatus: type: string description: The run's lifecycle status, verbatim. Not a verdict. verdict: type: string enum: - passed - failed - inconclusive - notEstablished verdictSource: type: string enum: - policyV2 - legacy - none description: >- `policyV2` — the run's own decision, carried on `decision`; `verdict` is its verdict. `legacy` — a percent-threshold run predating policy v2; there is no decision object and any counts are trials. `none` — no verdict; `verdict` is `notEstablished` and `undecided` says why. counts: $ref: '#/components/schemas/EvalRunDecisionCounts' decision: type: object description: >- The run's authoritative v2 verdict decision, copied verbatim after validation — the same shape as `EvalRun.verdictSummary`, published at https://mcpjam.com/schemas/eval-verdict-policy/v2.json. Present exactly when `verdictSource` is `policyV2`. undecided: $ref: '#/components/schemas/EvalRunDecisionUndecided' diagnostics: $ref: '#/components/schemas/EvalRunDecisionDiagnostics' EvalRunDecisionCounts: description: >- A tally WITH the population it counted. `measurementUnit` is never omitted and never inferred: under verdict policy v2 the counts are case-execution VARIANTS (repetitions are trials inside one, not members of it), and on a legacy run they are trials. A 3-case suite with 5 repetitions is legitimately 3 under one unit and 15 under the other, so a count reported without its unit is not a fact. On a `caseVariant` count every field is present and the three buckets sum to `total`; on a `trial` count each field is present only if the legacy run recorded it — absence stays absence rather than defaulting to zero. oneOf: - type: object required: - measurementUnit - total - passed - failed - inconclusive properties: measurementUnit: type: string const: caseVariant total: type: integer minimum: 0 passed: type: integer minimum: 0 failed: type: integer minimum: 0 inconclusive: type: integer minimum: 0 description: '`caseVariant` only. A legacy run has no such bucket.' additionalProperties: false - type: object required: - measurementUnit properties: measurementUnit: type: string const: trial total: type: integer minimum: 0 passed: type: integer minimum: 0 failed: type: integer minimum: 0 additionalProperties: false EvalRunDecisionUndecided: type: object description: >- Why no verdict was established. Present exactly when `verdictSource` is `none`. required: - reason properties: reason: type: string enum: - runNotTerminal - runStatusNotAVerdict - runResultNotAVerdict - verdictSummaryUnavailable description: >- `runNotTerminal` — still pending or running; poll it. `runStatusNotAVerdict` — a LEGACY run that stopped at cancelled/timed_out/failed, so its stored counts describe the iterations it happened to record and gating on them is fail-open (a policy-v2 run is deliberately not resolved this way: its validity phase is where lifecycle enters the verdict). `runResultNotAVerdict` — completed with no recognizable result. `verdictSummaryUnavailable` — decided under policy v2 with no readable decision; there is no legacy fallback, because re-deciding a v2 run under percent-threshold semantics would answer a question nobody asked. detail: type: string description: The platform's own message, when it supplied one. Never synthesized. EvalRunDecisionDiagnostics: type: object description: >- One page of diagnostics, with its completeness stated rather than implied. `complete` is true ONLY when `items` is the whole non-passing set for the run: a page reached through a cursor is never complete however few rows follow it. `scannedIterations` is how many trials this page examined, which is what separates "we looked at 50 and none failed" from "we did not look" — both otherwise render as an empty `items`. required: - items - complete - scannedIterations properties: items: type: array items: $ref: '#/components/schemas/EvalRunDecisionDiagnostic' complete: type: boolean nextCursor: type: string description: Pass back as `cursor`. Absent on the last page. scannedIterations: type: integer minimum: 0 Error: type: object required: - code - message properties: code: type: string description: >- Stable, machine-readable error code. New codes may be added over time; treat unknown codes as non-retryable failures unless the HTTP status says otherwise. enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - CONFLICT - VALIDATION_ERROR - RATE_LIMITED - FEATURE_NOT_SUPPORTED - SERVER_UNREACHABLE - TIMEOUT - OAUTH_REQUIRED - INTERNAL_ERROR message: type: string description: >- Human-readable description. May change between releases — don't match on it. details: type: object description: Optional, unstructured context bag. additionalProperties: true EvalRunDecisionDiagnostic: type: object description: >- One non-passing TRIAL, as evidence beneath the run's verdict. These are never counted as cases and never override the run's or a case's verdict: under policy v2 a case can pass with a failing trial in it, and a reader who tallies these rows instead of reading `decision.cases` has re-derived a different verdict from the same run. required: - iterationId - iterationNumber - status - chain - evidence - nextAction properties: iterationId: type: string iterationNumber: type: integer minimum: 0 caseId: type: string description: >- The case's SDK-declared id, when the run recorded one. NOT a join key into `decision.cases[].caseId`. testCaseId: type: string description: The stored case row id. title: type: string status: type: string description: LIFECYCLE, not verdict. result: type: string enum: - passed - failed description: >- Task verdict once terminal. Absent when the trial never produced one. chain: $ref: '#/components/schemas/EvalRunDecisionChain' expected: type: object required: - toolNames properties: toolNames: type: array items: type: string observed: type: object properties: toolNames: type: array items: type: string failure: type: string evidence: $ref: '#/components/schemas/EvalRunDecisionEvidence' nextAction: type: string description: >- One operator action, keyed on the failure category alone. With no category it says to go and look rather than naming a system, because nothing established which one is involved. EvalRunDecisionChain: description: >- Whether this trial's user-value chain can be believed. `verified` — the stored derivation validated, and ONLY this state carries `stages`, `firstFailedStage` or `failureCategory`. `unverified` — a derivation was stored and did not validate; the chain and both claims derived from it are withheld, because they are assertions ABOUT rows that did not validate. `absent` — no derivation was stored at all, which is a different fact from one that was rejected. oneOf: - type: object required: - status - stages - analyzerVersion properties: status: type: string const: verified stages: type: array description: Always six rows, in chain order. `verified` only. items: $ref: '#/components/schemas/StageResultRow' firstFailedStage: type: string enum: - connection - discovery - selection - call - response - userValue failureCategory: type: string enum: - setup - metadata - selection - arguments - serverData - userValue - evaluator description: >- The bucket this trial is grouped under. Present WITHOUT `firstFailedStage` for a setup abort or an evaluator error — both are real answers about a run that never reached a stage. analyzerVersion: type: integer minimum: 0 analyzerVersionAhead: type: object description: >- Present when the rows came from an analyzer NEWER than the reader knows. Flagged, never rejected: blanking the chain every time the platform ships ahead of a pinned client would be worse than showing it with a caveat. required: - reported - known properties: reported: type: integer minimum: 0 known: type: integer minimum: 0 additionalProperties: false additionalProperties: false - type: object required: - status properties: status: type: string const: unverified analyzerVersion: type: integer minimum: 0 analyzerVersionAhead: type: object required: - reported - known properties: reported: type: integer minimum: 0 known: type: integer minimum: 0 additionalProperties: false additionalProperties: false - type: object required: - status properties: status: type: string const: absent additionalProperties: false EvalRunDecisionEvidence: type: object description: >- Where to look. For a measured failure the span ids, prompt indexes and reasons are read from the FIRST FAILED STAGE'S ROW and from nowhere else — unioning the passing stages in would hand back the evidence of everything that worked, labelled as the explanation of the thing that did not. A stage-less outcome (setup abort, evaluator error) keeps a stage-less locator rather than inventing a stage to hang the link on. required: - runId - iterationId - tracePath properties: runId: type: string iterationId: type: string stage: type: string enum: - connection - discovery - selection - call - response - userValue description: >- The stage this evidence was read from. Absent when none was established. spanIds: type: array items: type: string promptIndexes: type: array items: type: integer minimum: 0 reasons: type: array items: type: string tracePath: type: string description: >- The iteration-trace endpoint's path RELATIVE TO THE API ROOT (e.g. `/projects/{projectId}/eval-runs/{runId}/iterations/{iterationId}/trace`), so it resolves against any deployment's base URL rather than baking one host into a stored artifact. StageResultRow: type: object description: One verified D1 user-value stage verdict, in chain order. required: - stage - state properties: stage: type: string enum: - connection - discovery - selection - call - response - userValue state: type: string enum: - passed - failed - notReached - notMeasured - notApplicable reason: type: string enum: - noSpanChannel - noEvidenceCaptured - matchVerdictUnavailable - traceAbsent - executorEmitsNoSpans - blockedByPolicy - evaluatorError - setupAborted - connectFailed - toolsListFailed - egressUnverified - lifecycleStopped - notAuthored - earlierStageFailed - missingToolCall - unexpectedToolCall - argumentMismatch - toolError - protocolError - renderFailed - predicateFailed - observed - impliedByLaterEvidence evidence: type: object properties: spanIds: type: array items: type: string maxItems: 5 promptIndexes: type: array items: type: integer maxItems: 5 predicateReasons: type: array items: type: string maxLength: 500 maxItems: 5 additionalProperties: false responses: Unauthorized: description: >- Missing, invalid, revoked, or orphaned key (`UNAUTHORIZED`) — or the **target MCP server** needs an OAuth grant (`OAUTH_REQUIRED`), which is a property of the server, not your key. content: application/json: schema: $ref: '#/components/schemas/Error' examples: badKey: summary: Invalid or revoked key value: code: UNAUTHORIZED message: Invalid API key oauthRequired: summary: Target server needs an OAuth grant value: code: OAUTH_REQUIRED message: Server requires OAuth authorization Forbidden: description: Key is valid but not allowed to do this. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: FORBIDDEN message: You do not have access to this project NotFound: description: Unknown project, server, or resource. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: NOT_FOUND message: Server not found RateLimited: description: >- Per-key rate limit exceeded (60 requests/minute sustained, bursts up to 10). Honor `Retry-After` and back off with jitter. headers: Retry-After: description: Seconds to wait before retrying. schema: type: integer content: application/json: schema: $ref: '#/components/schemas/Error' example: code: RATE_LIMITED message: API key rate limit exceeded. Slow down and retry. InternalError: description: Something failed on MCPJam's side. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: INTERNAL_ERROR message: Unexpected internal error securitySchemes: bearerAuth: type: http scheme: bearer description: >- MCPJam API key (`sk_…`). Create one at [Settings → API keys](https://app.mcpjam.com/settings/api-keys). Guest sessions cannot use the API, and API keys cannot manage other API keys. ````