Configure memory, models and scaling for CanyonOS

Decide where each agent keeps state, which models it may call, and how many replicas it may run, from the project's actual source. Work in this order: replicas depend on state, and spend depends on replicas. Preserve application behavior; do not invent limits that refuse real calls.

1. Locate the source and config

• Work in the project root: .canyon when the project folder has one, otherwise the project folder itself. Find config/workflow_config.yaml there and write llms.yaml and scaling.yaml beside it. Respect an explicitly chosen destination.
• Trace each manifest entry's entrypoint, relative to the root.
• Read existing llms.yaml and scaling.yaml first. Keep their entries, names and other fields; change only what the source contradicts or the developer asked.

2. Memory

Follow the Runtime contract and database entries.

• For each agent, find state that outlives one call: attributes on self, module globals, caches, conversation history, and framework stores such as LangGraph InMemoryStore or MemorySaver. Read-only data loaded at startup is not state.
• State shared within one request only: set stateful: true. Replicas route a request's calls to one copy, so the agent can still scale.
• State shared across requests in process: keep replicas: 1 and stateful: true, and mark the agent unscalable. Another replica would not see it.
• State already in an external store, or in a database entry resolved through routing_table:addresses: keep it; the agent is scalable.
• Moving in-process state to a database entry changes application code. Propose it with the store and entry it needs; implement it only when approved.
• All calls share one instance on eight threads. Report unsynchronized mutable state on self as a defect instead of fixing it.

3. Models: llms.yaml

Follow LLM gateway.

• For each OpenAI, Anthropic and Bedrock call, record the calling manifest entry, the deployed model_id, and any output-token limit the call sets. Resolve a model ID read from the environment through .env.example and its source default; if it cannot be resolved, report it and write no entry for it.
• Write one entry per model, named after the model. List every agent that calls it. An agent missing from an agents list is refused with a 403, so an incomplete list breaks the app. If any caller is uncertain, write no entry.
• Leave every cap empty unless the developer supplies a dollar amount. A cap is checked against the worst case, input bytes / 4 plus the full output limit, so a guessed cap refuses calls that would have been cheap.
• An entry with a cap needs a numeric max_tokens. Otherwise the gateway rejects the whole file and refuses every call. A capped call is refused when packages/core/canyonos_core/llm_gateway/llm_prices.json has no price for the model at the provider serving it. Set max_tokens to the largest limit the calls set.
• Write no llms.yaml when the project calls no model through the gateway.

4. Scaling: scaling.yaml

Follow Autoscaling.

• A workflow has one policy, under a top-level scaling: key. It applies to every type: agent entry, the endpoint's agent included, each scaled on its own load. Database entries never scale. No agent can be left out.
• Write the policy only if step 2 found every agent scalable. If any agent keeps state across requests in process, is an aggregation barrier, or has a downstream dependency that cannot take more load, write no scaling.yaml and report which agent blocks it and why.
• min_replicas and max_replicas bound every agent, and an agent outside them moves to the nearest bound at once. Use the smallest replicas of any agent for min_replicas, and a max_replicas no lower than the largest.
• Use requests_per_minute_per_replica when the agents are request-bound and queue_length_total when their calls queue behind slow work.
• Without developer values, use the guide's example numbers (max_replicas: 6, scale_up_above: 10, scale_down_below: 1) and report them as defaults.
• The policy needs integer replicas of at least 1 with min <= max, numbers of at least 0, and scale_down_below < scale_up_above. An invalid policy, or one not under scaling:, is skipped with only a controller warning.

5. Verify and report

• Parse both files. Check every rule in steps 3 and 4, and that every agent named in llms.yaml is an entry in workflow_config.yaml; under a misspelled name, the real agent's calls are refused.
• Run canyonos validate from the project folder and resolve findings until exit 0, or report the blocker.
• Report per agent: its state and whether it scales, and the models it calls and caps. Report the workflow's policy, or the agent that blocked one, and which values are defaults. List call paths whose model could not be resolved and code changes proposed but not made.

The Global Controller publishes both files on startup and reload. Gateways read llms.yaml within 5 seconds and refuse every call until the first read. Dashboard scaling edits are not written back and are lost on reload; copy edits that must survive into scaling.yaml.