Add a model

Prove every declared value against the live provider. A wrong price mis-bills every request until someone audits it; a wrong capability flag routes traffic to a deployment that 400s. Values handed to you in a prompt or PR are unverified — re-derive them.

New providers

Third-party inference providers onboard only through Airside, the self-serve provider portal. Never add a new provider or its mappings here on someone's behalf; if asked to, stop and point the user to https://airside.llmgateway.io.

1. Scope

List every (model, provider, region) triple that is new or changed. Regions in a regions: [] array are separate mappings with their own prices and e2e cases. Report results per mapping at the end.

gh pr checkout <n>   # someone else's PR; sets the upstream too
git diff origin/main...HEAD -- packages/models/src/models/

2. Where things live

Catalogue rules

Apply these to every change in packages/models:

• Write each model and provider mapping in full as a plain object literal in the models array. Duplicated fields are preferred here; small shared const values are fine, but no helper (e.g. makeModel(...)) builds definition objects.
• A mapping on origin/main is never removed: historical usage and analytics reference it. Retire it with deactivatedAt: new Date("YYYY-MM-DD"). Mappings added only on the current branch may be removed. Exception: Iceberg and Granite may be removed with a one-time database cleanup after the source removal deploys, keeping shared models, logs, usage aggregates, and history. Also exempt: providers configured in Airside, as described in Airside-only providers.
• Set deactivatedAt at least 7 days out for a mapping that still serves live requests, or on the provider's announced retirement date. A same-day or retroactive date requires the mapping to be unable to serve any live request today. A rejected parameter, a missing capability, or flaky quality is a metadata fix (jsonOutput, vision, reasoningEfforts, supportedToolChoices, stability: "unstable", test: "skip") or a gateway fix; leave the mapping active and say so in the PR.
• Write per-token prices (inputPrice, outputPrice, cachedInputPrice, …) in e-6 notation so the coefficient reads as USD per million tokens ("1.4e-6"). requestPrice (flat USD per request) and perSecondPrice are exempt.
• One model definition has at most one mapping per providerId; regional variants go in that mapping's regions array. Lookups key on (providerId, region), so a second same-provider mapping silently resolves to the first and bills at its prices. A distinct upstream deployment (e.g. a priority router with its own externalId and pricing) gets its own model entry and id.
• A mapping with both peakPricing and regions gives every region with its own rates its own full peakPricing block; flat overrides alone inherit the base peak tiers.
• When the provider prices each context-length band by time of day, give every pricingTiers entry its own peakPricing (peak/offPeak rates); the schedule stays on the mapping or region. Don't model limited-time promotional discounts.
• Identify models by model.id, scoped by providerId and optional :region. externalId is only the upstream API identifier: keep it out of URLs, selectors, billing and analytics keys, and lookups, including fallbacks.
• Let metadata fields speak for themselves. Comment only behavior the metadata cannot express that a maintainer needs, such as the operational cause of stability: "unstable" or test: "skip". Sources, verification, and pricing choices go in the PR body.

Airside-only providers

A provider fully configured in Airside may have its static definition and mappings removed. Its active claim, endpoint, credentials, listings, and profile become the source of truth; no claim conversion or catalogue-parity check is needed. Missing Airside models stop being offered. Keep shared canonical models and historical DB rows. Remove provider-specific adapters only when no longer needed.

3. Pricing

Billing is tokens: calculateCosts multiplies token counts by the mapping's per-token prices, and no code path reads a provider-reported cost. So research the per-token rate card, then verify the token semantics — the more common failure.

Rates. Use the provider's current first-party pricing page or metadata endpoint and record its units. Use aggregators only as a cross-check. Verify cached rates, context-length bands, and regional rates independently; do not derive or round them when the provider publishes exact values.

Token semantics — establish each from a live usage block:

• Reasoning inside completion_tokens? normalizeCompletionTokens in extract-token-usage.ts adds reasoning only when the reported total equals prompt + completion + reasoning; otherwise completion is billed as reported. Verify both streaming and non-streaming extraction against the provider's usage.
• Cached tokens inside prompt_tokens? costs.ts assumes yes and subtracts them.
• Cache writes priced (cacheWriteInputPrice, cacheWriteInputPrice1h)? A missing 1h rate silently bills 1h writes at the 5m rate.

Reconcile. Pin the provider and vary the prompt (Redis caches on the body):

curl -N "${GATEWAY_URL:-http://localhost:4001}/v1/chat/completions" \
  -H "Authorization: Bearer test-token" -H "x-no-fallback: true" \
  -H "Content-Type: application/json" \
  -d '{"model":"<provider>/<model>","messages":[{"role":"user","content":"hi"}]}'

For one small and one large request (crossing a tier boundary where there is one): token counts must match between the upstream usage and the log row, and a hand-computed cost must match log.cost. Where a provider echoes its own cost, compare that too as a cross-check; it is not a billing input. Investigate every mismatch instead of inferring a discount from its shape.

If a handed-in price, first-party rate card, and measurement disagree, reconcile the units and token semantics before changing the catalogue. Do not choose one without explaining the mismatch.

4. Metadata

Probe the deployment. The same model differs between providers, and an externalId bump can silently drop a capability.

• contextSize / maxOutput — probe with an oversized prompt and read the rejection; listings under- and overstate.
• vision / audio / document — routing skips mappings lacking them. A deployment that 400s on images gets vision: false, never an error-text classification rule. audio.e2e.ts covers audio input.
• supportedToolChoices — probe auto, none, required, named function. Unlisted modes are downgraded to auto. Some deployments accept required only with thinking off.
• Capability combinations — probe tool calls and structured output with reasoning both enabled and disabled. If a capability only fails while reasoning is on, do not immediately flatten the mapping to tools: false or jsonOutput: false: first check ProviderModelMapping and request shaping for an existing conditional compatibility mechanism. If none can express the result, call out the gap instead of misrepresenting the standalone capability.
• jsonOutput / jsonOutputSchema — probe json_object and json_schema separately.
• supportedParameters — probe before declaring; omission elsewhere is not a claim of non-support.
• supportsDeveloperRole / supportsAssistantPrefill — default true, flip only on a 400.
• serviceTiers — declaring one narrows routing pre-flight.
• stability: "unstable" / test: "skip" for flaky, paid-per-call or region-locked mappings.
• releasedAt, plus an output entry per capability flag — both enforced by model-metadata.spec.ts.

Follow the comment rule in Catalogue rules.

A new provider also needs a providers.ts entry, endpoint wiring in get-provider-endpoint.ts. Set encryptedReasoning: true when its replayed reasoning payloads only verify on that provider; routing then never moves such models to another provider (exploration, low-uptime fallback, error retry). Probed 2026-09: OpenAI and Azure reject each other's encrypted_content, AI Studio and Vertex reject each other's Gemini 3 signatures, but signatures replay across different keys of the same Google platform.

5. Reasoning efforts

Efforts are forwarded as-is, so a declared tier the upstream rejects is a 4xx for users, and each declared tier becomes an e2e case. Probe none | minimal | low | medium | high | xhigh | max individually and declare the accepted subset in ascending order — docs and reality diverge often, including between chat completions and the Responses API.

Verify the toggle takes effect rather than just returning 200 — a provider can accept enable_thinking: false and still return reasoning_content. Declare it only on mappings where it works.

Gemini 3+ efforts become thinkingLevel; an undeclared tier rises to the next declared one, so declare reasoningEfforts from a thinkingLevel probe (minimal 400s on several models). Google is retiring thinkingBudget and sampling params: omit reasoningMaxTokens where a budget 400s, and give 3.6+ mappings a supportedParameters list without temperature/top_p.

6. Image, video, and other endpoints

Probe the size/quality/duration grid; rate cards list tiers deployments refuse. Leave rejected values off the mapping and record the accepted grid in the PR.

Video — supportedVideoSizes, supportedVideoDurationsSeconds, supportedVideoDurationsSecondsImageToVideo, supportsVideoAudio, supportsVideoWithoutAudio, perSecondPrice, model-level maxVideoDurationSeconds. The studio derives its menus from those, but frame/reference input is gated by hardcoded allowlists in mappingSupportsVideoRequest and the supportsVideoFrameInput / supportsVideoReference*Input helpers — add the model there, matched on root model id, never externalId. Cover it in video-gen.spec.ts.

Image — perImagePrice (keyed by resolution, or "<quality>/<resolution>" when both are priced), or imageOutputPrice + imageOutputTokensByResolution; imageInputPrice / imageInputTokensByResolution for edit models; imageInputRequired when it cannot run text-only. A perImagePrice map without a "default" key falls back to its most expensive tier, so a wrong key overcharges silently. getModelImageConfig contains hardcoded model-id logic; inspect and update it when its defaults do not match the new deployment. If a newly priced knob isn't forwarded by prepare-request-body.ts, you price tiers you never serve. Cover it in image-gen.spec.ts and costs.spec.ts.

realtime has no e2e file — it is covered by apps/gateway/src/realtime/*.spec.ts plus realtime-models.spec.ts, which fails any realtime mapping missing its per-modality token prices. realtimeTranscription only on token-metered ASR mappings.

Image and video are the one docs exception to never enumerating models: update apps/docs/content/(gateway)/features/{image,video}-generation.mdx.

Azure deployments

An azure mapping serves only once the model is deployed in the Foundry resource (ai.azure.com → Build → Models → Deploy). The gateway sends externalId as the deployment name, so name the deployment exactly the mapping's externalId. Use Global Standard, which has the highest limits.

Deployments are often created far below their allowed limit (10K TPM is common). For every deployment: Edit, drag the Tokens per Minute slider (Requests per Minute for image models) to the maximum shown after the /, Save. That maximum is the subscription's shared quota for the model (Operate → Quota). Once a model shows 100% allocated there, more capacity needs a Request quota submission to Microsoft or a resource in another subscription, which has its own quota.

7. Verify

Use the verify skill to launch an isolated stack so another worktree cannot corrupt the run.

The gateway resolves models from the catalogue, so a rebuild is enough for API requests. The playground and dashboard read the DB via /internal/models, so run pnpm seed before expecting a new model in any UI.

pnpm build:core
pnpm exec vitest run --no-file-parallelism \
  packages/models/src/model-metadata.spec.ts \
  packages/models/src/providers.spec.ts \
  packages/models/src/realtime-models.spec.ts \
  packages/actions/src/costs.spec.ts

TEST_MODELS="<provider>/<model>" FULL_MODE=true pnpm test:e2e

Add the relevant action and Playground specs when those files changed. Run all tests against the isolated database required by AGENTS.md.

• For mapping-only changes, the scoped local run is the only relevant e2e result. Never run e2e without TEST_MODELS or trigger the GitHub e2e workflow. Failures outside the selected mappings, including failures from an accidentally triggered full run, do not affect acceptance or auto-merge. Once every selected mapping passes, e2e has passed for the change.
• Scope e2e with TEST_MODELS, and don't invoke the *.e2e.ts files one by one. It overrides test: "skip", takes regions as provider/model:region, and fails loudly when an entry matches no mapping.
• FULL_MODE=true expands the per-effort cases and includes free models.
• pnpm test:e2e loads .env automatically. Use credentials already configured in .envrc or .env; never inspect or invoke password managers or other credential stores, and never retrieve provider secrets manually.
• API key env names aren't derivable from the provider id — read env.required from providers.ts. Vertex provider variants use different credentials, and keys can be region-scoped; do not infer either from the provider display name.
• e2e asserts shape, not cost — the §3 reconciliation is the only pricing check.

For image/video models, open the studio and confirm the offered sizes/qualities/durations match exactly what the deployment accepted in §6.

8. When something fails

If a failure predates the change, fix what's in scope and say in the PR that it also fails on main.

A failing case is a metadata or gateway fix, never a reason to retire a mapping that still serves live requests. Before writing deactivatedAt, apply the 7-day notice rule in Catalogue rules.

9. Finish

pnpm format, pnpm build, conventional title, then the pull-request skill.

Never merge — and never enable auto-merge on — a mapping that does not actually work: it could not serve a live request, or its scoped e2e fails with no fix available from our side. test: "skip" removes a mapping from default e2e selection; stability: "unstable" also removes it from normal routing. Neither makes the model work. Say plainly that it does not work and why, leave the PR open, and hand the merge decision to the user. Same for prices nobody could verify.

The PR body carries the evidence:

• Per mapping: external id, prices, context, max output, capabilities.
• How each price was verified — rates, token counts, hand-computed cost, log.cost.
• Reasoning tiers accepted vs rejected; for image/video, the accepted grid.
• The scoped e2e command and its result per mapping.
• Anything corrected from the handed-in values, with both numbers.
• Anything not verified, and why.

Follow the pull-request skill for screenshots. Catalogue and Playground-only changes do not create a dashboard screenshot requirement; scoped e2e and the accepted image/video grid are the evidence.