Gateway request path

The gateway is latency-critical and high-throughput.

Hot path

• Read no log rows on the request path, however narrow the query or short the cache. Credit gates, spend and limit checks, and routing signals come from Redis counters maintained on the write path (incremented at insertLog, settled by the billing worker) or small cached rows.
• Route on the capability field curated for it (supportedToolChoices, serviceTiers, supportsAssistantPrefill, …). supportedParameters is a partial list.
• Provider-bound reasoning (mappings where usesEncryptedReasoning holds: reasoning: true on an encryptedReasoning provider) can only verify on the account that produced it. chat/tools/reasoning-credential-pin.ts pins a replaying turn to that credential, including a pending_deletion key; new credential selection must honor the pin. A retry that leaves the credential goes through resolveProviderContextForRetry, which drops OpenAI-format encrypted reasoning; Gemini signatures cannot be dropped and still block fallback. New bound-reasoning shapes belong in providerBoundReasoningIds.
• Bare model ids route only to deployments in the model's full context-window tier (getContextTierMinSize in packages/models). Do not add prompt-size estimates to provider selection for a fixed model; chars/4 misjudges per-model tokenizers.

User-supplied URLs

Fetch every content URL from a request body through processImageUrl (packages/actions/src/process-image-url.ts), or for non-image content assertSafeUserContentUrl from @llmgateway/shared/url-safety-node followed by a redirect: "error" fetch. Keep validateSsrf on; only trusted provider-response URLs pass validateSsrf: false. The guard enforces https-only in every environment, blocks internal hosts and private, reserved, link-local, metadata, and IPv4-mapped IPv6 addresses, and refuses redirects. When the guard rejects a URL, fail the request; forwarding it upstream lets the provider fetch what we refused. Download URLs in a provider response (video, audio) are tenant-controlled under BYOK: fetch them with fetchProviderIssuedUrl from apps/gateway/src/lib/fetch-provider.ts. Worker requests to a BYOK base URL or webhook target pass tenantControlled to fetchWithSignals in apps/worker/src/services/video-jobs.ts. Route every non-data: URL through the guard wherever remote content is inlined (e.g. requiresBase64Images mappings).

Errors

Keep apps/gateway/src/chat/tools/get-finish-reason-from-error.ts conservative. Generic 4xx wording such as "X is not supported for this model" or unsupported_content_type stays a client error: users sending wrong requests produce the same text, and reclassifying it triggers pointless fallback. When a deployment rejects a capability the catalogue claims, fix that mapping's flag in packages/models (e.g. vision: false). If asked for a broad reclassification, raise the misclassification risk and confirm first.

Logging, streaming, shutdown

• Pass { retentionLevel } to every insertLog call; the one-argument form drops payloads for retaining orgs.
• Log a pre-provider rejection in chat.ts with logGatewayRejection before throwing; app.onError writes no log.
• Import streamSSE from apps/gateway/src/lib/pending-work.ts, not hono/streaming, so graceful shutdown waits for the billing tail.
• From response-lifecycle callbacks (close handlers, middleware finally), send Redis commands through redisClient.pipeline().<cmd>().exec(). An auto-pipelined bare command there can wedge the shared client.