Autoresearch Iteration

You are an automated researcher designing token optimization methods to minimize token-forcing loss on language models.

• Run code: $ARGUMENTS[0] — determines the method chain, branch, and log location
• Goal (everything after the run code): the research objective

This skill runs ONE iteration of the research loop. It is designed to be called repeatedly via /loop.

Derived from run code $ARGUMENTS[0]:

• Method directory: claudini/methods/claude_$ARGUMENTS[0]/
• Method name prefix: claude_$ARGUMENTS[0]_v
• Git branch: loop/$ARGUMENTS[0]
• Agent log: claudini/methods/claude_$ARGUMENTS[0]/AGENT_LOG.md

Initialization (first iteration only)

Read claudini/methods/claude_$ARGUMENTS[0]/AGENT_LOG.md. If it exists, skip this section — the run is already set up.

Config. If the user's goal mentions a specific config name (e.g. random_train, safeguard_valid), use that existing config from configs/. Otherwise, check configs/ for a preset that matches. Only create a new config if nothing fits:

# Autoresearch: <brief description>
model: <model_id>
optim_length: 15
max_flops: <budget>
dtype: bfloat16
system_prompt: ""
samples: [0, 1, 2]
seeds: [0]
final_input: tokens
use_prefix_cache: true

input_spec:
  source:
    type: random
    query_len: 0
    target_len: 10
  layout:
    type: suffix
  init:
    type: random

Parse the goal to extract model (default: Qwen/Qwen2.5-7B-Instruct) and FLOP budget (default: 1.0e+15).

Git branch. Create and switch to loop/$ARGUMENTS[0] if not already on it.

Agent log. Create claudini/methods/claude_$ARGUMENTS[0]/AGENT_LOG.md with the config name, goal, and setup details.

Step 1 — Design and implement a new method

Design and implement a new optimizer that achieves lower loss than existing methods. Read the agent log, then use whatever you need:

• Agent log: claudini/methods/claude_$ARGUMENTS[0]/AGENT_LOG.md
• Your method chain: claudini/methods/claude_$ARGUMENTS[0]/
• Other methods: claudini/methods/ (baselines and other Claude-designed chains)
• Benchmark results: results/ (shared across all runs and methods)
• Developer guide: CLAUDE.md

Create the next version as a proper Python package under claudini/methods/claude_$ARGUMENTS[0]/v<N>/ with method_name = "claude_$ARGUMENTS[0]_v<N>".

Step 2 — Run the benchmark

The method must not override config settings — suffix length, FLOP budget, model, samples, etc. are controlled by the config, not the optimizer.

Run the full benchmark. Launch in background and don't wait:

uv run -m claudini.run_bench <config> --method claude_$ARGUMENTS[0]_v<N>

Step 3 — Commit and update log

Commit the new method and any config changes to the loop/$ARGUMENTS[0] branch. Then update claudini/methods/claude_$ARGUMENTS[0]/AGENT_LOG.md with:

• What method you created and the key idea
• What to try next iteration