Skip to main content
POST
Launch one run per target as a group (async)

Authorizations

Authorization
string
header
required

MCPJam API key (sk_…). Create one at Settings → API keys. Guest sessions cannot use the API, and API keys cannot manage other API keys.

Headers

x-mcpjam-eval-vocabulary
enum<string>

Which vocabulary this request and its response speak. Absent means 1, which is byte-for-byte today's contract: the same request fields, the same refusals, the same response projection. 2 is the canonical vocabulary. Any other value is a 400 with code: "VALIDATION_ERROR".

Today it decides one thing: the spelling of an evaluator's policy role. Vocabulary 1 accepts and returns gating; vocabulary 2 accepts both spellings and returns the canonical required. Sending required without the header is a 400, deliberately: vocabulary 1 is not widened to meet vocabulary 2 half way, because a boundary that accepts a spelling it does not announce is one two implementations can disagree about.

A response that varies by vocabulary sends Vary: x-mcpjam-eval-vocabulary.

Available options:
1,
2
x-mcpjam-launcher
string

The launching process's own DECLARED identity, as compact JSON: {"kind":"cli"|"mcp"|"github_action","client":"…","version":"…"}. Stored on the run as launcher and used only as a display label: source stays api and the audit trail reads the platform-minted attribution instead.

A malformed value, or a kind outside the three above, is DROPPED rather than rejected, so a label never fails a launch. Max 512 bytes.

x-mcpjam-ci
string

The CI job this run was launched from, as compact JSON. Recognized keys: provider, pipelineId (or runId), jobId (or job), runUrl, branch, commitSha. Stored on the run as ciMetadata, which is what makes a run resolvable by commit for baseline comparison. Unrecognized keys are dropped.

Dropped rather than rejected when malformed. Max 2048 bytes.

Path Parameters

projectId
string
required

ID of the hosted project that contains the server.

Body

application/json

Launch one run per target under a single server-minted group id.

targets fan out along ONE axis: every entry is an environmentId, or every entry is a namedHostId. A mixed list returns 400 with details.reason: "HETEROGENEOUS_TARGETS": mixing them would have to answer whether a host runs inside an environment or beside it, and every answer to that is a cross product nobody asked for. Duplicates are deduplicated by id, preserving order.

The knobs below apply UNIFORMLY to every run in the group. serverIds and refreshSnapshot are deliberately absent: an environment supplies its own closed server set, a host derives one from its config, and last-writer-wins on a frozen suite snapshot is never what a fan-out means. Inline tests are absent for the same reason: a group runs a persisted suite.

suiteId
string
required

The suite every target runs.

targets
object[]
required

One entry per run. Every target must be ATTACHED to the suite; an unattached one returns 400 with zero runs started.

Required array length: 1 - 10 elements
iterationOverride
integer

Override the per-case runs count for every run in the group.

Required range: 1 <= x <= 10
caseIds
string[]

Narrow every run in the group to these suite cases.

Minimum array length: 1
matchOptionsOverride
object

Match options for every run in the group. Same public-or-internal vocabulary as POST /eval-runs.

skillsOverride
enum<string>

Run every target as the "without skills" arm.

Available options:
exclude
notes
string
passCriteria
object

The floor a run must clear, as a PERCENT in [0, 100]: 80 means 80%. minimumPassRatePercent is the canonical spelling (the unit is in the name); minimumPassRate is the deprecated alias for it. Send exactly one. THE NAME DISAMBIGUATES THE UNIT: a value in (0, 1) is accepted on minimumPassRatePercent, where it unambiguously means a sub-1% floor the evaluator can act on, and REJECTED on the bare minimumPassRate, where 0.8 cannot be told apart from a fraction sent by mistake and would silently make the gate unfailable. Neither field ever reinterprets a value. A per-case passThreshold IS a fraction; this suite/run floor is not.

idempotencyKey
string

Makes the whole launch replayable. The runGroupId is DERIVED from this key and each target's run key is derived from that, so a retry after a crash mid-launch returns the original run ids rather than double-launching the targets that already started. The Idempotency-Key header carries the same value and wins over this field.

Maximum string length: 256
ephemeralEnvironment
boolean

When true, each environment target may be a project-scoped, non-archived environment that is NOT attached to the suite. The launch never mutates the suite. Absent / false keeps the membership check. Probe GET /environments/capabilities (ephemeralEnvironmentLaunch) before sending: older servers reject the unknown field.

importApprovals
object[]

Approve approximated imported cases for THIS RUN ONLY. The caller supplies a hosted test-case id and a reason and nothing else: the approver's identity and the approval time are derived server-side and frozen into the run's snapshot, because a caller-supplied approver would file one person's approval under another's name and a caller-supplied timestamp could be backdated past the edit that invalidated the claim. Nothing persists on the case: the next run of the same approximation needs a new approval. On a grouped launch the SAME approvals go to every target: a case's approximation is approximated the same way on each of them, so one human decision covers the whole fan-out.

Minimum array length: 1

Response

Group accepted. Each target's outcome is in targets; execution continues in the background.

One entry per target. A per-target failure does NOT abort its siblings, so read outcome rather than treating the 202 as "everything started".

runGroupId
string
required

Server-minted. Every sibling run carries it, and the hosted UI collapses them into one row.

suiteId
string
required
outcome
enum<string>
required

started: every target launched; partial: some did and some did not; failed: none did (still a 202: the group itself was valid, and the per-target reasons are in targets).

Available options:
started,
partial,
failed
startedCount
integer
required
failedCount
integer
required
targets
object[]
required

Discriminated on status, so a target that failed can never be mistaken for one that started with an unread runId.

runId
string
deprecated

Mirror of the FIRST started run, so readers written against the single-run receipt keep working. Absent when nothing started. Read targets instead: this describes one run out of several.

status
string
deprecated

See runId.

servers
object[]
deprecated

See runId.

environment
object · null · null
deprecated

See runId.

caseUpsert
object
deprecated

See runId.