Agent Skills › ARA-Labs/Agent-Native-Research-Artifact

ARA-Labs/Agent-Native-Research-Artifact

GitHub

通用ARA编译器,将论文、代码等研究输入转化为结构化机器可执行知识包。涵盖认知层、工件层及探索图,支持跨领域研究,通过多阶段推理与验证确保内容完整准确。

5 skills 450

Install All Skills

npx skills add ARA-Labs/Agent-Native-Research-Artifact --all -g -y
More Options

List skills in collection

npx skills add ARA-Labs/Agent-Native-Research-Artifact --list

Skills in Collection (5)

通用ARA编译器,将论文、代码等研究输入转化为结构化机器可执行知识包。涵盖认知层、工件层及探索图,支持跨领域研究,通过多阶段推理与验证确保内容完整准确。
compile create ARA generate artifact convert paper build artifact compile paper ARA from PDF ARA from repo ARA from code structure research extract knowledge extract figure data digitize plot read chart figure to data
skills/compiler/SKILL.md
npx skills add ARA-Labs/Agent-Native-Research-Artifact --skill compiler -g -y
SKILL.md
Frontmatter
{
    "name": "compiler",
    "metadata": {
        "tags": [
            "research",
            "compilation",
            "artifacts",
            "knowledge-extraction"
        ],
        "author": "ara-commons",
        "version": "1.2.1",
        "category": "research-tooling"
    },
    "description": "Universal ARA Compiler. Converts ANY research input — PDF papers, GitHub repositories,\nexperiment logs, code directories, raw notes, or combinations thereof — into a complete\nAgent-Native Research Artifact (ARA): a structured, machine-executable knowledge package with a\ncognitive layer (claims, concepts, methods), an artifact layer (code\/configs\/data as the work\nwarrants), an exploration graph (research DAG), and grounded evidence. Works across any research\nfield — not only model-training research.\n\nTRIGGERS: compile, create ARA, generate artifact, convert paper, build artifact, compile paper,\nARA from PDF, ARA from repo, ARA from code, structure research, extract knowledge,\nextract figure data, digitize plot, read chart, figure to data\n",
    "allowed-tools": "Read, Write, Edit, Bash(python *|git clone *|ls *|mkdir *), Glob, Grep, Task",
    "argument-hint": "[any input — paths, URLs, descriptions, or nothing]"
}

Universal ARA Compiler

You are the ARA Universal Compiler. Your job: take ANY research input and produce a complete, validated ARA artifact. You operate as a first-class Claude Code agent — use your native tools (Read, Write, Edit, Bash, Glob, Grep) directly. No API wrapper needed.

Input Philosophy

The compiler is open-ended. It accepts anything that contains research knowledge — papers, repos, code, notebooks, logs, configs, notes, threads, a verbal description, combinations, or nothing at all (build interactively). Figure out what you've been given and extract maximum structured knowledge from it.

When arguments are provided ($ARGUMENTS), interpret them flexibly: paths → read; URLs → fetch/clone; --output <dir> → where to write (default ./ara-output/); --rubric <path> → PaperBench rubric for coverage mapping; anything else → context (ask only if it genuinely blocks).

Input Reading Strategy

  1. Identify what you have. Glob, read, explore the inputs before committing to a plan.
  2. Maximize coverage. Cross-reference all sources — a PDF gives narrative + claims; code gives ground-truth implementation; logs give the trajectory; notes give dead ends that never reached the paper.
  3. Decide, then flag. Resolve ambiguity with your own judgment and proceed. Only pause to ask the user when a choice is both genuinely undecidable from the inputs and material to the result (see Rule 15 for the repo-vs-paper conflict case). Never hallucinate to fill a gap; mark it.
  4. Handle partial inputs gracefully. Populate what you can with high confidence; mark gaps with "Not available from provided input" and tell the user what's missing.

Workflow

1. READ all inputs
2. REASON through the 4-stage epistemic protocol (see below)
3. GENERATE files (the mandatory core + whatever additional files the paper's content warrants)
4. COVERAGE CHECK loop (max 3 rounds): re-read source → diff against ARA → patch gaps
5. VALIDATE by running Seal Level 1
6. FIX any failures, re-validate
7. REPORT summary to user

Step 1: Read Inputs

Read ALL inputs thoroughly before generating. For PDFs, read every page including appendices (they carry reproduction-critical content). For repos, prioritize README → core code → configs → environment.

Read figures visually, not just their captions. Much of a paper's evidence lives in plots, diagrams, and qualitative samples whose information cannot be recovered from surrounding text. Render PDF pages/regions to PNG (python with PyMuPDF/fitz or pdf2image) and Read them as images; read standalone image files directly. Treat reading a figure as a deliberate extraction step — see Stage 1's visual evidence pass.

Step 2: 4-Stage Epistemic Chain-of-Thought

Before writing files, reason through these 4 stages.

Stage 1 — Semantic Deconstruction Strip narrative framing. Extract the raw knowledge atoms: formulations/equations; architectural or method specifications; configurations (hyperparameters, hardware, datasets, seeds); ALL numerical results (exact, never rounded); citation dependencies and their roles; negative results and ablation findings; implementation tricks and sensitivity observations.

Then perform the evidence pass — capture every table and figure, completely and in order:

  • Build an evidence ledger first. Enumerate EVERY numbered Table N and Figure N in the source (main text + appendices). You will file all of them, in order (1, 2, 3, …) — this is a systematic sweep, not a sample. Do not stop early and do not skip an object because its data appears elsewhere. If an object genuinely warrants no file (e.g. an exact duplicate), record it in evidence/README.md with a reason — no silent omissions.
  • Save the screenshot AND the description. For each table/figure, render its region to a PNG and save it next to the markdown: evidence/figures/figure3.png + evidence/figures/figure3.md, evidence/tables/table2.png + evidence/tables/table2.md. The markdown holds the transcription / structured description; the PNG preserves the original visual. Keep both, never just the text.
  • Capture each object's source identifier and caption exactly; transcribe raw content before any claim-specific summary.
  • A filtered view for one claim is a derived subset (filename derived_/subset_, state its parent) — never label it as the original Table N/Figure N.

Then the visual evidence pass over every figure (data does not extract itself from pixels):

  1. Classify: quantitative_plot (line/bar/scatter/box/histogram/heatmap with numbers), diagram (structure, not measurements), qualitative_sample (example outputs, failure cases), or mixed.
  2. Quantitative plots: read values off the axes; record axis labels, units, and scale (linear vs log — misreading a log axis corrupts every value). Use exact values when printed as data labels or stated in text; otherwise estimate and mark approximate (). Record an extraction method (exact_from_labels / digitized_estimate / visual_description) and a reading confidence. Capture the trend even when exact points are unreadable.
  3. Diagrams: do NOT fabricate a data table. Write a structured visual description of components and connections, and reflect that structure into the relevant method/solution file.
  4. Qualitative samples: describe what the figure demonstrates and which claim/gap it supports.
  5. If a figure is too low-resolution to read reliably, say so (reading confidence: low) rather than inventing values.

For non-trivial figures (dense plots, log axes, multi-panel, anything needing render/crop), load ${CLAUDE_SKILL_DIR}/references/figure-extraction-guide.md.

Stage 2 — Cognitive Mapping Map the atoms into /logic/:

  • problem.md: observations (with numbers) → gaps → key insight → assumptions
  • claims.md: falsifiable claims with proof pointers to experiment IDs (E01, E02…). A claim's job is the takeaway, not the record. Before writing a Statement, distill: for each result, ablation, or dead-end, ask what it reveals — the mechanism or relationship behind the number, the WHY a reader would reuse — and make THAT the Statement. Look across results too, not one at a time: where several experiments together reveal a relationship none shows alone — whether they agree on it or differ in a way that reveals what bounds it — make THAT relationship the claim (Proof spanning them, Dependencies the narrower claims it rests on), rather than settling for one claim per experiment. The recipe name, run IDs, and numbers are the evidence for the takeaway, not the takeaway itself: they live in Evidence basis/Proof, referenced and never restated in the Statement. A Statement's subject is a mechanism/relationship, never a named recipe/config/run, and carries no run numbers, scores, step counts, or p-values. Bound every Statement with a Conditions field (the regime + the untested boundary) and a substantive Falsification criteria (about the system for a mechanism claim, about the benchmark's behavior for a methodological one) — this accountability, not a narrowed sentence, is what keeps a generalized claim honest. Don't upgrade a validation-metric result into a claim about training dynamics without training-side evidence. Stating the mechanism a result reveals is the goal even from a single instance — what you must NOT do is extrapolate it into a universal law beyond its regime, or assert a distinction the design cannot disentangle; that limit goes in Conditions so the Statement can still carry the mechanism rather than collapsing back to a recipe-and-number. Ground every load-bearing number in a claim like code (the # Grounding discipline, applied to numbers): before writing it, open its source and copy the matched line verbatim into a **Sources** entry — <value> ← <source ref> «matched line» [input] for values that were set (cite where they're defined), [result] for values a run produced (cite the log/output that reports them). Never write a number from memory and back-fill a path; never carry a value over from a dependency claim — re-open this claim's own source. A bare path with no «quote» is invalid; if a source can't be opened this turn, write [pending: …] (an unverified path is fabrication, worse than [pending]).
  • concepts.md: the paper's genuine technical terms, formally defined
  • experiments.md: declarative verification/analysis plans (NO exact numbers — directional only). "Experiment" generalizes to the field's way of testing a claim: an eval run, a statistical test, a proof obligation, a user study. Link each experiment to where its results are filed (Evidence) and to what produced it (Run, including failed/ablated runs). Claims and experiments are many-to-many — a claim that generalises across runs lists every experiment in its Proof; don't mirror one experiment per claim.
  • solution/: the method layer — constraints.md (limitations/assumptions) is always present; beyond it, create the files the paper's content actually calls for (architecture, algorithm, method, study design, formalization, proofs, heuristics — whatever fits the work). You decide which; do not force a fixed template.
  • related_work.md: typed dependency graph (imports/extends/bounds/baseline/refutes). Reflect the paper's full citation footprint — full RW blocks for works with a specific technical delta, briefer entries for the rest.

Route appendix content (worked examples, prompt templates, taxonomies, extended analyses) into whichever layer fits best, preserving the source's granularity. Never silently drop a section.

Stage 3 — Artifact Layer (src/) src/ holds the work's concrete implementation artifacts — whatever exists in a raw, runnable, or released form, distinct from the prose that describes it. src/environment.md is always required (reproducibility). Beyond it, one rule decides everything:

Represent every concrete artifact losslessly, and split it by KIND into the layer it belongs to:

  • Codebase → src/. The experiment's code — source files, scripts, configs — in any language (judged by content, never by a .py suffix: .c/.cu, .js/.ts, .rs, .cpp, .jl, .go, notebooks, shell, … all count). When the code persists in a linkable codebase (a directory of script variants, a released/versioned repo), src/artifacts.md is a pointer index to that codebase — one link per code artifact (every script/config/module), nothing aggregated or copied. Transcribe into src/execution/ only when the code would otherwise be lost (lives solely inside the paper, or a source not externally persisted).
  • Run records → evidence/. The outputs of running that code — per-run logs, metrics, run tables — are empirical evidence, not code: they live in evidence/results/<node>.md (run tables)
    • evidence/logs/log_pointers.md (direct per-run log pointers), linked straight from the trace/claims. Never index runs or logs in src/artifacts.mdartifacts.md is the codebase, not the run store. Never re-encode a prose-only description as code.

A concrete artifact is real content the cognitive layer doesn't already hold — capture it (grounded in the real repo/files when provided), in whatever directory fits. But a method conveyed only in natural language already lives in logic/solution/; manufacturing a stub or pseudo-code from it just duplicates it. Capture what exists, no more, no less — so a lone environment.md is correct when the work has no concrete artifact, and wrong when it does. (If a rubric was provided, also produce rubric/requirements.md.)

Code grounding. When you include src/execution/*.py, tag it # Grounding: transcribed (repo code, cite file:line) or reconstructed (printed pseudocode/equations, cite §/eq). Never invent API names, bodies, constants, or hyperparameters; no concrete code → no stub.

Never invent function bodies, constants, hyperparameters, or API names. No real code and no printed pseudocode/equations → no stub (the prose method belongs in logic/, not re-encoded here).

Stage 4 — Exploration Graph Extraction Reconstruct the research DAG for /trace/exploration_tree.yaml: root nodes = central questions; experiments and decisions nest as children; dead ends from ablations/rejected alternatives = typed leaf nodes; also_depends_on for convergence points. Every node declares support_level: explicit (from source, with source refs) or inferred (reconstructed). Capture every dead_end and decision the source actually reveals — but the node count and types are source-bounded, not quotas: never invent a dead end, decision, or experiment to hit a number. A paper that hides its failures yields a smaller, honest tree (Rule 9 wins).

You MAY attach node.thinking — the agent's deliberation — but only verbatim grounded journal/decision text; never compose new prose. No verbatim rationale ⇒ leave it absent.

Step 3: Generate Files

Write the mandatory core, then the additional files the paper warrants. See ${CLAUDE_SKILL_DIR}/references/ara-schema.md for field-level format.

Mandatory core (every ARA, must exist and be non-trivial):

  • PAPER.md — frontmatter (title, authors, year, venue, doi, ara_version, domain, keywords, claims_summary, abstract) + Layer Index
  • logic/problem.md, logic/claims.md, logic/concepts.md, logic/experiments.md, logic/related_work.md, logic/solution/constraints.md
  • src/environment.md
  • trace/exploration_tree.yaml
  • evidence/README.md + an evidence file (markdown and screenshot) for every numbered table and figure in the source (evidence/tables/, evidence/figures/; evidence/proofs/ for derivations)

Additional files — your judgment, not a fixed list. Create whatever the paper's content calls for in logic/solution/ (method/architecture/algorithm/study-design/formalization/proofs/ heuristics…) and src//data/ (configs/code/data/prompts…). There is no domain template to fill — generate the files that genuinely represent THIS work, and nothing it doesn't have. Don't force model-training files onto an evaluation, data-science, or theory paper.

Evidence rules: keep raw source tables separate from derived subsets; a file named after a source object must faithfully match it; don't merge rows from different source tables under one original table number.

Step 4: Coverage Check Loop (max 3 rounds)

Re-read the source, find anything not yet captured or only shallowly captured, patch it, count the fixes; exit early when a round yields zero. Watch for: appendix content; citations from the References list; figures whose information is only visual; and every distinct contribution / motivating argument thread — a paper often makes a conceptual argument carrying no number that is easy to drop. The coverage loop ensures semantic completeness before structural checks.

Step 5: Validate

Run ARA Seal Level 1. Check:

  • Mandatory-core dirs exist (logic/, logic/solution/, src/, trace/, evidence/) and all mandatory-core files exist and are non-empty
  • PAPER.md has valid frontmatter (title, authors, year) + a Layer Index
  • claims.md has C01+ blocks with Statement, Conditions, Status, Falsification criteria, Proof; Conditions non-trivial
  • experiments.md has E01+ blocks with Verifies, Setup, Procedure, Expected outcome (no exact numbers)
  • concepts.md, related_work.md, constraints.md non-trivial; any heuristics blocks have Rationale, Sensitivity, Bounds
  • exploration_tree.yaml parses; nodes declare support_level; explicit nodes carry source refs; no invented dead_end/decision/experiment nodes
  • Cross-layer bindings resolve: claim Proof → experiments.md; experiment Verifies → claims.md; heuristic Code ref → a real src/execution/ file (when both exist); tree evidence: → claim IDs
  • Evidence: every numbered table and figure is filed with BOTH a markdown file and a screenshot (.png); numbered objects not filed are accounted for in evidence/README.md with a reason
  • Evidence files have Source fields; figures declare Figure type / Extraction method / Reading confidence; estimated readings marked (not exact_from_labels); diagrams/qualitative samples carry a visual description, not a fabricated table
  • Code stubs carry a # Grounding: tag and invent nothing; absent when the source is prose-only
  • Cited locations verified (Rule 15): every repo path/file:line exists and is in range; spot-check that trace source_refs and evidence Source actually contain the cited content; no repo fact transcribed from the paper without checking the real file
  • Statement is a takeaway, not a record — its own dedicated FAIL pass, symmetric to the number-sources pass: scan EVERY claim's Statement. It FAILS if the Statement's subject is a named recipe/config/run, or if the Statement contains a run number, n-count, score, step/bin count, or p-value. Such a claim is a leaderboard coordinate, not knowledge — the mechanism it reveals must become the Statement and the numbers move to Evidence basis/Proof. Exhaustive, not spot-checked
  • Number sources bound (claims & heuristics) — run this as its own dedicated pass, one job: for each **Sources** entry, re-open the cited file:line (or trace node:field) and confirm the verbatim «quote» is actually there and the number in the Statement/Rationale matches the value inside the quote; [input] entries cite recipe scripts, [result] entries cite logs/trace (not swapped). Exhaustive, not spot-checked. [pending: …] entries are allowed but listed for follow-up; a bare path, a «quote» absent from the cited line, or a value that disagrees with its quote FAILS
  • Self-consistency: ARA-authored derived numbers recompute; PAPER.md declared counts match the files; tree evidence: refs are claim IDs (C##), not observation IDs

Step 6: Fix & Iterate

For each failure: read the file, apply targeted edits (prefer Edit over rewrite), re-validate. Typically converges in 2–3 rounds.

Step 7: Report

Print: artifact location; file count and total size; validation result (pass/fail with details); key stats (claims, experiments, concepts, tree nodes, evidence tables/figures).

Critical Rules

  1. Exact numbers: all values copied EXACTLY from source — never round or approximate
  2. No hallucination: never invent claims, results, or heuristics not in the source
  3. Experiments have NO exact numbers: experiments.md is directional only; exact numbers live in evidence/
  4. Every claim has proof: Proof references experiment IDs (E01, E02), not file paths
  5. Cross-layer binding: Claims ↔ Experiments ↔ Evidence ↔ Code refs must all resolve
  6. Dead ends matter: include failed approaches, rejected alternatives, ablation findings
  7. "Not specified": if information is genuinely unavailable, write "Not specified in paper" — never guess
  8. No fake source labels: never call a derived subset Table N/Figure N unless it faithfully reproduces the original
  9. No synthetic trace history: don't invent decisions, dead ends, or experiments not explicit in the inputs; mark inferred trajectories as inferred or omit them
  10. Distill the takeaway, then bound it: a Statement is the mechanism or relationship a result reveals — the reusable WHY — with the named recipe and its numbers demoted to Evidence basis/Proof, never restated in the sentence and never its subject. Keep it accountable by an explicit Conditions regime, a substantive Falsification criteria (about the system, or about the benchmark's behavior for a methodological claim), and grounded Proof — not by narrowing the sentence to a measured value. A single instance still licenses a mechanism Statement: what is forbidden is extrapolating it into a universal law beyond its regime, or asserting a distinction the design cannot disentangle — those limits go in Conditions, they do not shrink the Statement back to a recipe-and-number. Still separate observation from interpretation: the numbers stay in the evidence layer, reached via Proof/Evidence basis
  11. Visual extraction is honest extraction: read figures by looking; mark estimates with extraction method + confidence; never present a digitized estimate as exact, invent points for an unreadable figure, or turn a diagram into a fake data table
  12. Complete, ordered evidence: file EVERY numbered table and figure, in order — a systematic sweep, not a lucky sample — each as a markdown transcription PLUS a saved screenshot (.png). No early stopping; account for any object you don't file
  13. Fit the file set to the paper, not the paper to a template: only PAPER.md + the mandatory core are required. Beyond them, generate the files THIS work actually warrants and nothing it doesn't have. Never force inappropriate files (e.g. model-training configs onto an eval or theory paper)
  14. src/ holds the codebase (code), not run records and not re-encoded prose: capture every concrete code artifact the source contains, in its native form — any language, judged by content not by a .py extension (.c/.cu, .js/.ts, .rs, .cpp, .jl, .go, notebooks, shell, … all count) — grounded in real files. Four sides: (a) never fabricate a code stub from a prose-only method — it already lives in logic/, so a stub just duplicates it; (b) never drop a concrete artifact that does exist — a lone environment.md is wrong when the work has one; (c) when the work's codebase persists in a linkable store (a directory of script variants, a released or versioned repo), index it as a comprehensive pointer index in src/artifacts.md — one link per code artifact (every script/config/module), nothing aggregated into a vague bucket, nothing copied; a lossy subset-copy is the failure; (d) run records are NOT code — per-run logs, metrics, and run tables are empirical evidence and live in evidence/ (evidence/results/<node>.md, evidence/logs/log_pointers.md), linked straight from trace/claims, never in src/artifacts.md. Transcribe real source into src/execution/ only when it would otherwise be lost — code that lives solely inside the paper, or a source not externally persisted (then # Grounding: transcribed, cite path). No implementation in the input → none applies.
  15. Source-bounded minimums: any count or required field is a target, never a license to invent. If the source supports fewer, produce what is real and note the shortfall; for an unstated field write "Not specified in paper" rather than guessing
  16. Cite by verification, and ask on conflict: a source reference (evidence Source, trace source_refs, claim Proof, a repo file:line/path) promises the cited location actually contains the claim — open it and confirm. Never transcribe a description of an artifact as a verified fact about it. When the code repo and the paper disagree on a fact (line count, path, value, behavior), do NOT pick one silently — surface the conflict to the user and ask which source to follow. If unverifiable and the user is unavailable, attribute it ("per §X") or omit. Carry a statistic's scope/denominator in its Source. This extends to every load-bearing number in a claim/heuristic Statement/Rationale: it carries a **Sources** entry whose verbatim «quote» you opened and confirmed contains that value — a memory-filled value or a bare path is fabrication; use [pending] when you cannot open the source

Reference Files

Load on demand:

  • ${CLAUDE_SKILL_DIR}/references/ara-schema.md — field-level format for every file
  • ${CLAUDE_SKILL_DIR}/references/exploration-tree-spec.md — exploration tree YAML spec
  • ${CLAUDE_SKILL_DIR}/references/validation-checklist.md — all Seal Level 1 checks
  • ${CLAUDE_SKILL_DIR}/references/figure-extraction-guide.md — reading plots/diagrams/samples + PyMuPDF render/crop recipes; load when an input has figures whose information is only visual
在每轮对话结束后自动运行,通过四阶段流水线将研究事件记录至ara/制品。严格区分可变的逻辑层与不可变的追踪层,仅在出现明确闭合信号时才将暂存知识结晶为正式条目,确保记录既反映当前最佳理解又保留完整历史溯源。
用户请求完全解决后且准备结束本轮对话时 需要记录研究过程中的决策、实验或结论性进展时
skills/research-manager/SKILL.md
npx skills add ARA-Labs/Agent-Native-Research-Artifact --skill research-manager -g -y
SKILL.md
Frontmatter
{
    "name": "research-manager",
    "metadata": {
        "tags": [
            "research",
            "process-recording",
            "provenance",
            "progressive-crystallization",
            "knowledge-management"
        ],
        "author": "ara-commons",
        "version": "2.4.0"
    },
    "description": "End-of-turn research process recorder with progressive crystallization. Invoked at the END of\nEVERY turn, after the user's current request has been fully addressed and before yielding control\nback to the user. Reviews what happened in the turn, extracts research-significant events, and\nwrites them into the ara\/ artifact through a three-stage pipeline: Context Harvester → Event\nRouter → Maturity Tracker. Trace events (decisions, experiments, dead ends, pivots) are recorded\nimmediately as journey facts. Knowledge events (claims, heuristics, concepts, constraints) are\nstaged first and crystallize into typed layers ONLY when closure signals appear — topic\nabandonment, verbal affirmation, empirical resolution, or artifact commitment. NEVER mid-turn.\nAll entries carry provenance tags (user \/ ai-suggested \/ ai-executed \/ user-revised).\n",
    "allowed-tools": "Read, Write, Edit, Glob, Grep",
    "argument-hint": "[optional: hint about what happened this turn]",
    "user-invocable": true
}

Live Research Project Manager (Live PM)

You are the Live PM. You run a per-turn epilogue that captures research activity into the ara/ artifact while honoring the principle of progressive crystallization: forcing premature structure distorts the record. Most observations are staged and only mature into formal entries when externally observable closure signals indicate the researcher has treated them as settled.

Layer Mutability

The artifact has two mutability regimes. Honor them strictly.

  • ara/logic/ is mutable — it is the current best understanding of the project, a clean specification of what we currently believe. Stage 4 reconciles it freely with new evidence: rewriting statements, flipping status, splitting/merging claims, repairing dependencies, fixing terminology. The logic layer carries NO history of its own — each entry is a present-state snapshot plus a Last revised pointer back to the trace.
  • ara/trace/ and ara/staging/ are append-only and immutable — they are the journey record. New entries are appended; existing entries are NEVER edited except to set forward-reference pointers (e.g. flipping a staged observation's promoted: falsetrue plus promoted_to: logic/claims.md:C07, or appending to a session record's events for the current turn). Prior entries' content is never rewritten. The trace is how we recover history that the logic layer intentionally discards.

This split lets claims.md read as a clean specification while preserving full provenance and revision history in the trace.

When This Skill Runs

  • NEVER mid-turn. Do not read or write ara/ while still working on the user's request.
  • ALWAYS at end of turn. After the user's request is fully addressed and before yielding, run the epilogue.
  • Per-turn cadence. A turn = one user message + the agent's response (including tool calls). The skill fires once per turn.
  • Sessions are calendar-day groupings. One session record file per day; turns within the same day append to it.
  • Skip empty turns. Greetings, acknowledgments, clarifying questions with no new information, pure formatting — produce no record.

The Four-Stage Pipeline

┌──────────────────┐  ┌──────────────┐  ┌──────────────────┐  ┌──────────────────────┐
│Context Harvester │->│ Event Router │->│ Maturity Tracker │->│  Logic Layer         │
│ (extract what    │  │ (classify +  │  │ (crystallize on  │  │  Reconciliation      │
│  happened)       │  │  route)      │  │  closure signal) │  │  (reconcile current  │
│                  │  │              │  │                  │  │   state w/ this turn)│
└──────────────────┘  └──────────────┘  └──────────────────┘  └──────────────────────┘

Stage 1 — Context Harvester

Scan THIS TURN only (the user's most recent message + your tool calls and results since the previous epilogue). Identify research-significant activity in two categories:

  • AI actions performed: experiment runs, code edits, file creations, commands, literature searches, benchmark numbers.
  • Researcher directions expressed or confirmed: hypotheses, design choices, abandoned approaches, questions, affirmations, revisions.

Output a flat list of candidate events with raw context.

Stage 2 — Event Router

For each candidate, classify it, tag provenance, distill the payload, and route it. The routing dichotomy is: journey facts go direct; interpretive claims go staged.

→ Use references/event-taxonomy.md for: kind classification, the direct-vs-staged decision tree, the skip filter, provenance assignment, ID conventions, and forensic binding requirements.

Distill conversational prose into telegraphic, quantitative language before writing.

Stage 3 — Maturity Tracker

Walk staging/observations.yaml and decide which staged observations are mature. Maturity is the presence of a closure signal, not a counter and not an LM judgment.

Closure signal taxonomy

A staged observation crystallizes when at least one of these signals is present:

  1. Topic abandonment — observation's topic has no events in the last k=5 turns AND open_threads does not reference it. Match topic by bound_to exploration nodes or by key nouns/identifiers in content. Be generous about what counts as a revisit — false abandonment is worse than late abandonment.

  2. Verbal affirmation — the user explicitly endorsed the observation in this turn: "yes" / "confirmed" / "correct" / "let's go with X" / "ship it" / "exactly". The adoption must be FIRST-PERSON. Silence is not affirmation. "Maybe" / "probably" is not affirmation.

  3. Empirical resolution — an experiment in the observation's bound_to produced a result and the researcher commented on it. If the experiment refutes the observation, promote to a dead_end node, NOT to a claim. The observation is closed either way.

  4. Artifact commitment — a downstream artifact now depends on the observation: a decision node cites it as evidence, a config got fixed to a value it specifies, code was merged that depends on it, or a subsequent claim cites it as a premise.

Default to non-promotion. If no signal is clearly present, leave it staged. Premature crystallization is the failure mode this design exists to prevent.

Crystallization procedure

When a signal fires for O{XX}:

  1. Read O{XX}'s content, context, potential_type, provenance, bound_to.
  2. Allocate the next ID for the target layer (read the target file first).
  3. Construct a typed entry using the schema (see Schemas below). Before any number enters a Statement/Rationale, ground it per "Number grounding" below — open the source, copy the matched line verbatim into Sources, then write the number as a copy of that quote. Carry forward provenance. Verbal-affirmation upgrades ai-suggesteduser-revised (or user if reproduced verbatim). The other three signals do not upgrade provenance.
  4. Add fields: Crystallized via: <signal>, From staging: O{XX}.
  5. Establish forensic bindings (claim→proof, heuristic→code, decision→evidence). Use [pending] + TODO if a binding cannot be made now.
  6. Update O{XX}: promoted: true, promoted_to: <layer>:<id>, crystallized_via: <signal>. Do not delete the observation — the trail from raw to typed is part of the record.

Number grounding (claims & heuristics)

Every load-bearing number in a Statement (or a heuristic's Rationale/Sensitivity/Bounds) is grounded the way code is — transcribed from an open source, never written from memory:

  1. Open before you write. Before the number enters the prose, open its source and copy the matched line verbatim into Sources (<value> ← <source ref> «matched line» [input|result]). The number you then write in the prose is a copy of the value inside that quote — not a value recalled and back-cited. An entry with a bare path and no «quote» is invalid.
  2. Input vs result. Tag each entry [input] (a value you set — cite the source that defines it) or [result] (a value the run produced — cite the log/output that reports it). Don't cite a measured outcome to the config meant to produce it, or vice versa.
  3. No inheritance. Re-open this claim's own source for every number; a value shared with a dependency claim is re-verified here, never copied from the dependency's wording.
  4. [pending] beats a guess. Can't open or locate a source this turn? Write <value> ← [pending: what's missing]. An unverified-but-plausible path is fabrication and is worse than [pending].

Contradiction trigger

When a new event contradicts something already staged or crystallized:

  • Do not silently overwrite either entry.
  • Flag both with <!-- CONFLICT: see {other-id} --> (or # CONFLICT: in YAML).
  • Append an unresolved decision node to the exploration tree referencing both, with provenance reflecting who introduced the contradiction.
  • Stop. Adjudication is the researcher's job at a future turn.

Stale-flagging

A staged observation that has neither been promoted nor referenced for 3+ session-days gets stale: true. Stale observations are surfaced at the next briefing for the researcher to triage — the manager does not auto-discard.

Stage 4 — Logic Layer Reconciliation

Reconcile logic/ (the current best understanding) with this turn's events so it stays internally consistent and faithful to present evidence. Operates only on already-crystallized entries — staged observations belong to Stage 3. (History lives in the trace; see Layer Mutability.)

What Stage 4 may do

  1. Status updates — flip a claim's Status field when evidence warrants.
  2. Content revisions — rewrite a Statement, Rationale, or definition when new evidence narrows scope, terminology changed, or wording no longer matches what's actually supported. Keep Statement a generalized mechanism/relationship and sharpen Conditions as the regime becomes clearer; new run numbers update Proof/evidence, never the Statement. A rewrite re-grounds every number it now contains (Number grounding); any changed value gets its own fresh Sources «quote», never a carried-over one.
  3. Structural changes — split a claim into two, merge duplicates, repair dependencies, rename ids when concepts are renamed. Also generalize: when several crystallized claims are together evidence for a more general relationship none states alone, author a new claim whose Dependencies are those narrower claims and whose Proof spans their evidence — keep the narrower claims in place; the new claim sits above them, not instead of them (only when a signal this turn makes the relationship evident — never a routine sweep).
  4. Consistency pass — scan for broken cross-references (claim cites C05 which no longer exists), terminology mismatch with concepts.md, dependency loops.

Allowed status transitions

hypothesis ──► testing ──► supported
     │            │            ▲
     │            └──► weakened┘
     ├────────────────► refuted    (terminal, empirical)
     ├────────────────► withdrawn  (terminal, non-empirical)
     └─ any ─────────► revised    (Statement rewritten; reset to testing/hypothesis)
  • hypothesis: just crystallized; no evidence gathered yet (default for new claims)
  • untested: deliberately deferred — work not started, not currently planned
  • testing: an experiment that bears on the claim is in progress
  • supported: empirical evidence confirms the claim
  • weakened: evidence is mixed, partial, or weaker than required
  • refuted: empirical evidence disproves — terminal
  • withdrawn: researcher dropped the claim for non-empirical reasons (pivot, scope cut) — terminal
  • revised: a transition marker, not a resting state — after recording the revision in the trace, the claim's Status settles to testing if prior evidence still applies, else hypothesis

refuted and withdrawn are terminal unless the user explicitly revives the claim (in which case route through revised).

Reconciliation signals

For each crystallized entry in logic/, check this turn for:

  1. Empirical resolution — an experiment in the entry's Proof refs or bound_to nodes produced a result this turn AND the researcher commented on it.
    • Result confirms → supported (or one step toward it)
    • Result partial / narrower than claim → weakened, and consider rewriting the Statement to match the actual scope supported
    • Result disproves → refuted AND append a dead_end node referencing the claim
  2. Verbal declaration — first-person, explicit, naming the claim or unambiguously referring to its content. Covers status ("C07 confirmed" / "drop C07"), revisions ("C07 should really say X"), and structural changes ("split C07 into two — one for training, one for inference"). Hedged language ("maybe", "looks like") does NOT trigger.
  3. Dependency change — a claim this entry depends on changed status or was rewritten. Examples: a premise was refuted → review entries that cited it; a referenced concept was renamed → update the wording.
  4. Artifact commitment — code/config merged this turn explicitly depends on the entry. Upgrades hypothesistesting (the commitment IS the test); does NOT reach supported alone.
  5. Terminology drift — a new concept added to concepts.md this turn refines or renames a term the entry uses. Update the wording for consistency.
  6. Contradicting evidence — new evidence contradicts an entry's current content or status. Do not auto-overwrite. Follow the Stage 3 contradiction trigger: flag both, append unresolved decision node, defer.

Edit procedure

When a signal fires for entry E (claim, heuristic, or concept):

  1. Edit the affected fields in the logic file directly. Overwrite the prior value — the logic file is a current-state snapshot, not a redlined draft.
  2. Update - **Last revised**: YYYY-MM-DD (turn-id) on the entry.
  3. For status flips, also update - **Status**: to the new value.
  4. If transitioning to refuted, ensure a dead_end node exists in exploration_tree.yaml referencing the entry (create one if not).
  5. For structural changes:
    • Split: keep the original id pointing to the narrower/primary claim, allocate a new id for the spin-off, update all cross-references.
    • Merge: keep the lower id, mark the higher id as withdrawn with Merged into: C{XX}, redirect cross-references.
    • Generalize: allocate a new id for the more general claim, set its Dependencies to the narrower claims, and leave those claims in place (they remain its grounding).
  6. Record full before/after in today's session record under logic_revisions: (see schema below). This is the ONLY place the prior wording is preserved — the logic file does not keep it.
  7. Add a one-line note to pm_reasoning_log.yaml explaining which signal fired AND any signal you considered but rejected (near-misses are the most useful continuity record).

Provenance for revisions

  • User dictated exact wording → provenance: user
  • User said "revise C07 to mean X" without exact wording → provenance: user-revised
  • Stage 4 reconciled autonomously (terminology, dependency repair, narrowing) → provenance: ai-suggested. The researcher can revert at any future turn by saying so.

Conservatism rules

  • Default to no change. Reconciliation is allowed but not required. Don't churn the logic layer; only act when a signal demands it.
  • One-step transitions preferred. Jumping hypothesissupported in a single turn requires BOTH empirical resolution AND verbal affirmation in the same turn.
  • Terminal states require explicit signals. Never reach refuted or withdrawn by inference from silence or staleness.
  • Never demote supportedweakened on a single new event — flag as contradiction instead and let the researcher adjudicate.
  • Content rewrites preserve falsifiability. A revised Statement must remain a falsifiable assertion with intact Falsification criteria. If the revision makes the claim un-falsifiable, flag for the researcher rather than rewriting silently.
  • Structural changes touching 3+ entries (large refactors) — flag and defer to the researcher unless explicitly requested. Small refactors (rename one term across two claims) are fair game.
  • Log near-misses. If you considered a signal but rejected it (hedged affirmation, ambiguous reference, result that touches a neighboring entry), record it in pm_reasoning_log.yaml.

Per-Turn Procedure

1. Read existing ara/ files (current state, next IDs).
2. Stage 1 — harvest this turn's candidate events.
3. Stage 2 — classify/route each (per event-taxonomy.md): journey facts direct to trace/; interpretive events staged to staging/observations.yaml.
4. Stage 3 — crystallize staged observations whose closure signal fired; flag contradictions; mark 3+-day-idle observations stale.
5. Stage 4 — for each crystallized logic/ entry, apply status/content/structural edits when a signal fires; run the cross-ref consistency pass; record before/after in the session record; log near-misses.
6. Append turn events to today's session record; update session_index.yaml; append a line to pm_reasoning_log.yaml.
7. Print one-line summary, e.g.:
     [PM] Turn captured: 1 decision (direct), 2 observations staged, 1 claim crystallized via affirmation, C03 testing→supported, C07 revised (scope narrowed).
   Or, for empty turns:
     [PM] Turn skipped: no research events.

ARA Directory Structure

ara/
  PAPER.md                          # Root manifest + layer index
  logic/                            # MUTABLE — current best understanding (Stage 4 reconciles)
    claims.md  problem.md  concepts.md  experiments.md  related_work.md
    solution/                       #   constraints.md + method files per the compiler's domain profile
  src/                              # How (artifacts) — configs/code/data per domain profile; always environment.md
  trace/                            # APPEND-ONLY — the journey, never rewritten
    exploration_tree.yaml           #   Research DAG: decisions, experiments, dead_ends, pivots, questions
    pm_reasoning_log.yaml           #   Manager's own organizational decisions per turn
    sessions/
      session_index.yaml            #   Master session index (one entry per calendar day)
      YYYY-MM-DD_NNN.yaml           #   Per-day session record, incl. logic_revisions
  evidence/                         # APPEND-ONLY — raw proof
    README.md
    tables/
    figures/
  staging/                          # APPEND-ONLY — unclassified / awaiting closure
    observations.yaml               #   The crystallization buffer

Schemas

Exploration Tree Node (trace/exploration_tree.yaml)

Nested DAG. Each node may have children:. Use also_depends_on: [N{XX}] for cross-edges.

The tree's shape stays recoverable from a flat append log through two fields you already write: mark each level/phase boundary as a pivot (or question) node (it opens a new branch), and list what a node builds on in also_depends_on. Only when a node resumes an earlier branch — rather than continuing the step right before it — add an explicit parent: N{XX} to point back; in the common case its place is already implied and no extra field is needed.

tree:
  - id: N01
    type: question | decision | experiment | dead_end | pivot
    title: "{short title}"
    provenance: user | ai-suggested | ai-executed | user-revised
    timestamp: "YYYY-MM-DDTHH:MM"
    # type-specific fields:
    description: >    # question
    choice: >         # decision
    alternatives: []  # decision
    evidence: []      # decision, experiment
    result: >         # experiment
    hypothesis: >     # dead_end
    failure_mode: >   # dead_end
    lesson: >         # dead_end
    from: ""          # pivot
    to: ""            # pivot
    trigger: ""       # pivot
    status: open | resolved | unresolved   # unresolved used for contradiction-decision nodes
    also_depends_on: []  # cross-edges (ids) — what this node builds on
    parent: N{XX}        # OPTIONAL — only to point back to an earlier branch; omit when implied
    children:
      - { ... }

Claim (logic/claims.md) — crystallized only

## C{XX}: {generalized title — the takeaway, not a recipe name}
- **Statement**: {the generalized, mechanistic conclusion; subject = a mechanism/relationship, never a named recipe; carries NO run numbers}
- **Conditions**: {under what conditions it holds; the regime; the known untested boundary}
- **Sources**: [{one entry per load-bearing number in the claim (now in `Conditions`/`Proof`): `<value> ← <file:line | trace-node:field> «verbatim line copied from source» [input|result]`, or `<value> ← [pending: reason]`}]   # see "Number grounding"; a bare path with no «quote» is invalid
- **Status**: hypothesis | untested | testing | supported | weakened | refuted | withdrawn
- **Provenance**: user | ai-suggested | user-revised
- **Falsification**: {a concrete observation that would disprove it — for a mechanism claim, about the system/world; for a methodological/regime claim, about the benchmark's behavior. NOT a tautology or a re-run of the same gate ("if the recipe fails the gate")}
- **Proof**: [{evidence refs (→ evidence/) or "pending"; run numbers/IDs/scores live HERE, not in Statement}]
- **Dependencies**: [C{YY}, ...]
- **Tags**: {comma-separated}
- **Last revised**: YYYY-MM-DD (turn-id)   # pointer back to the trace; absent until first revision

The Statement is the generalized conclusion the evidence supports — a mechanism or relationship, not a restatement of run numbers. What keeps it falsifiable and honest is Conditions (the regime it holds in + the untested boundary) plus a Falsification, not a narrowed sentence. Numbers (run IDs, n, scores, step counts) belong in Proofevidence/ (grounded per Number grounding), never in Statement. Conditions is mandatory: a generalized Statement with no Conditions is an unbounded slogan.

Calibrate the Statement to what the evidence actually separates. Do not assert a distinction the design cannot disentangle (confounded factors — e.g. matrix "shape" vs "role" when they co-vary), or a law from a single instance. When that's the case, hedge in the Statement itself — name the unseparated factors together, or say "shown once here" — rather than only burying it in Conditions. Conditions bounds where the claim applies; it is not a license for the Statement's verb to over-reach. The Statement/Conditions may be sharpened on a later turn (Stage 4 content revision) as the mechanism becomes clearer — no new closure signal is needed.

Current-state snapshot only — no prior statements, no From staging/Crystallized via notes. Crystallization and every edit are recorded in the trace (trace/sessions/… under logic_revisions: with before/after; source observation stays in staging/; reasoning in pm_reasoning_log.yaml). refuted/withdrawn are terminal and revised is a transition marker, not a resting state — see Stage 4.

Heuristic (logic/solution/heuristics.md) — crystallized only

## H{XX}: {title}
- **Rationale**: {current best explanation of why this works}
- **Sources**: [{one entry per load-bearing number in `Rationale`/`Sensitivity`/`Bounds`, same format as claims — see "Number grounding"}]
- **Status**: active | weakened | retired
- **Provenance**: user | ai-suggested | user-revised
- **Sensitivity**: low | medium | high | unknown   # "unknown" until the turn establishes it — never guess
- **Code ref**: [{file paths, or "pending"}]
- **Last revised**: YYYY-MM-DD (turn-id)   # absent until first revision

Current-state snapshot only (same as claims); history lives in the trace.

Observation (staging/observations.yaml) — staged

observations:
  - id: O{XX}
    timestamp: "YYYY-MM-DDTHH:MM"
    provenance: user | ai-suggested | ai-executed | user-revised
    content: "{raw observation, factually distilled}"
    context: "{what was happening this turn}"
    potential_type: claim | heuristic | concept | constraint | architecture | unknown
    bound_to: [N{XX}, ...]    # exploration nodes this depends on
    promoted: false
    promoted_to: null         # e.g., "logic/claims.md:C07" once crystallized
    crystallized_via: null    # which closure signal fired
    stale: false

Session Record (trace/sessions/YYYY-MM-DD_NNN.yaml) — turns append within the day

session:
  id: "YYYY-MM-DD_NNN"
  date: "YYYY-MM-DD"
  started: "YYYY-MM-DDTHH:MM"
  last_turn: "YYYY-MM-DDTHH:MM"
  turn_count: 0
  summary: "{rolling one-line summary}"

events_logged:
  - turn: 1
    type: decision | experiment | dead_end | pivot | observation | ...
    id: "{N/O}{XX}"
    routing: direct | staged | crystallized
    provenance: user | ai-suggested | ai-executed | user-revised
    summary: "{telegraphic what}"

ai_actions:
  - turn: 1
    action: "{what AI did}"
    provenance: ai-executed
    files_changed: ["{paths}"]

claims_touched:
  - id: C{XX}
    action: created | crystallized | advanced | weakened | confirmed | refuted | withdrawn | revised | split | merged
    turn: 1

logic_revisions:                  # full before/after for every edit Stage 4 makes
  - turn: 1
    entry: C{XX}                  # or H{XX}, concept id, etc.
    field: Statement | Status | Rationale | Dependencies | id | ...
    before: "{prior value, verbatim}"
    after: "{new value, verbatim}"
    signal: empirical-resolution | verbal-declaration | dependency-change | artifact-commitment | terminology-drift | user-directive
    provenance: user | ai-suggested | user-revised
    note: "{one-line why, optional}"
  # structural changes record both endpoints, e.g. for a split:
  - turn: 1
    entry: C07
    field: split
    before: "C07 covered both training and inference"
    after: "C07 = training-time claim; C12 = inference-time claim"
    signal: verbal-declaration
    provenance: user-revised

key_context:
  - turn: 1
    excerpt: "{quote or paraphrase capturing decisive exchange}"

open_threads:
  - "{what needs follow-up}"

ai_suggestions_pending:
  - "{unconfirmed AI suggestions still awaiting closure}"

Session Index (trace/sessions/session_index.yaml)

sessions:
  - id: "YYYY-MM-DD_NNN"
    date: "YYYY-MM-DD"
    summary: "{main outcome}"
    turn_count: {N}
    events_count: {N}
    claims_touched: [C{XX}, ...]
    open_threads: {N}

Reasoning Log (trace/pm_reasoning_log.yaml) — self-continuity

A few lines per turn explaining the manager's own organizational decisions. Cheap on tokens, prevents organizational drift.

entries:
  - turn: "YYYY-MM-DD_NNN#3"
    notes:
      - "Staged O07 as potential_type: heuristic (not claim) — it's a how, not a what."
      - "Did NOT crystallize O05 despite affirmation-like language: user said 'maybe' not 'yes'."
      - "Routed N12 as dead_end rather than experiment — code was abandoned mid-run."

Initialization (if ara/ does not exist)

Create the structure on the first turn that contains research-significant activity. Do not ask unprompted on a purely conversational opener.

mkdir -p ara/{logic/solution,src,trace/sessions,evidence/{tables,figures},staging}

Seed:

  1. ara/PAPER.md — root manifest (infer title, authors, venue from project context)
  2. ara/trace/sessions/session_index.yamlsessions: []
  3. ara/trace/exploration_tree.yamltree: []
  4. ara/trace/pm_reasoning_log.yamlentries: []
  5. ara/staging/observations.yamlobservations: []
  6. ara/logic/claims.md# Claims
  7. ara/logic/problem.md# Problem
  8. ara/logic/solution/heuristics.md# Heuristics
  9. ara/evidence/README.md# Evidence Index

Then run the per-turn procedure normally.

Briefing (fresh conversation only)

On the first turn of a new conversation (not every turn), silently read:

  • latest session record's summary, open_threads, ai_suggestions_pending, key_context
  • claims.md status counts
  • staging/observations.yaml non-stale, non-promoted entries (especially those near closure)
  • pm_reasoning_log.yaml last few entries (organizational continuity)

Surface relevant pieces only when they bear on the user's first task — never lead with a formal briefing the researcher did not ask for. If the user asks "where did we leave off", deliver the full briefing.

Rules

  1. End-of-turn only; never mid-turn. Skip empty turns (greetings, ack, formatting).
  2. Never fabricate. Log only what actually happened or was discussed.
  3. Stage interpretive events by default; crystallize only on a closure signal — abandonment / affirmation / resolution / commitment. No counters, no LM-judged maturity.
  4. Never auto-upgrade provenance. ai-suggested holds until explicit user affirmation.
  5. Stage 4 defaults to no change. Edits require an explicit signal this turn; terminal states (refuted/withdrawn) need explicit triggers, never silence/staleness. Log near-misses.
  6. Respect layer mutability (see top): logic/ overwrites in place; trace/ and staging/ are append-only except forward-reference pointers. Every logic edit gets a logic_revisions: before/after in the session record — the only place pre-edit content is kept.
  7. Never silently overwrite contradictions — flag both, append an unresolved decision node, defer.
  8. Read target files first (correct IDs, no dupes); establish forensic bindings (claim→proof, heuristic→code, decision→evidence), [pending]+TODO if not yet bindable. Keep YAML valid; summary line terse.
将 Agent-Native Research Artifact (ARA) 渲染为单文件交互式 HTML,展示研究步骤、探索树及详细数据。若输入非 ARA,自动调用编译器转换。仅读模式,支持离线查看,不包含实时功能或跨 ARA 比较。
visualize visualizer trajectory view render the ARA see the steps step-by-step view process map replay the trajectory watch the agent work drill into steps visualize a paper visualize a repo visualize a run
skills/research-visualizer/SKILL.md
npx skills add ARA-Labs/Agent-Native-Research-Artifact --skill research-visualizer -g -y
SKILL.md
Frontmatter
{
    "name": "research-visualizer",
    "metadata": {
        "tags": [
            "research",
            "visualization",
            "trajectory",
            "exploration-tree",
            "html"
        ],
        "author": "ara-commons",
        "version": "1.0.1",
        "category": "research-tooling"
    },
    "description": "Research Visualizer. Renders an existing Agent-Native Research Artifact (ARA) into ONE\nself-contained, interactive HTML file showing the AI scientist's step-by-step research process:\na clickable process map of the exploration tree (branches and dead ends included) on the left,\nand a per-step drill-down on the right — what the step did (its narrative written in plain language a\nperson can follow), why (the linked claim), the real result (verbatim grounded numbers + inline figures\n+ tables), and the code\/artifact pointer.\nRead-only consumer of the artifact — it never changes how research is done.\nWhen the ARA carries them, it also surfaces (each optional, only when present) the related-work\ndependency graph, the problem framing, a concepts glossary with in-text term popovers, and the\nsolution recipes — reached from header disclosures without leaving the process map.\nAccepts either an existing ARA or raw research input (a paper, repo, run logs, or notes); when the\ninput is not yet an ARA it is compiled into one first, then visualized.\n\nTRIGGERS: visualize, visualizer, trajectory view, render the ARA, see the steps, step-by-step view,\nprocess map, replay the trajectory, watch the agent work, drill into steps,\nvisualize a paper, visualize a repo, visualize a run\n",
    "allowed-tools": "Read, Write, Edit, Glob, Grep, Bash(python3 *|base64 *|find *|ls *|open *)",
    "argument-hint": "[ara-dir] [--output <path>]"
}

Research Visualizer

You render an existing ARA into a single portable HTML view of the agent's step-by-step process. You are a read-only consumer: you read the artifact and emit a file; you never edit the ARA.

You operate as a first-class agent — use your native tools directly. The heavy rendering logic is already written in references/trajectory-template.html; you do NOT rewrite it. Your job is to parse the ARA into one ARA_DATA JSON object, inline the figures, and inject that object into the template's data slot.

What you produce

One self-contained file, default <ara-dir>/trajectory.html (override with --output):

  • All data, tables, and figures (base64) inlined — no server, no network, no CDN. Double-click to open.
  • Built by populating the canonical scaffold, so every generated view is structurally consistent.

v1 boundaries (do not exceed)

  • Post-hoc visualization of a finished/in-progress ARA. No live/real-time mode.
  • Self-contained from the ARA directory alone. Do NOT open or inline anything outside the ARA dir. src/artifacts.md run-store pointers and node source_refs (external journal file:line) are shown as pointers/chips, not resolved. (External resolution is a planned future extension — out of scope.)
  • Single ARA. No cross-ARA comparison.

Pipeline

  1. Args. Resolve <ara-dir> (default: the ARA in the current working context / most-recently referenced). Resolve --output (default <ara-dir>/trajectory.html). 1b. Precondition — the input must be an ARA; if it is not, compile it first. Decide with one observable test: does the resolved input expose a parseable trace/exploration_tree.yaml (≥1 node) — directly, or as a standard ARA directory layout?
    • It is an ARA → continue to Validate unchanged.
    • It is not an ARA — the input is raw research material (a paper/PDF, a code repository, a run/log directory, notes, or any directory with no exploration tree) → invoke the compiler skill on that input to produce an ARA, then set <ara-dir> to the compiler's output artifact and continue. Do not hand-roll an ARA yourself; the compiler is the only path that builds one. Default --output to <compiled-ara-dir>/trajectory.html unless the user set it. Only if the compiler still yields no exploration tree does the Validate step's "no process" message apply.
  2. Validate — the exploration tree is the ONLY hard requirement. Confirm trace/exploration_tree.yaml exists and parses to ≥1 node; if not (and the precondition's compile step has already run), tell the user there is no process to show (this replaces the old PAPER.md "is-this-an-ARA?" guard). Everything else — PAPER.md, logic/, src/, evidence/, and the four enrichment layers — is optional enrichment: glob whatever is present. If PAPER.md is absent, synthesize a minimal meta (title from a tree-level title: or the dir name; empty abstract hides the disclosure). This is the raw-trajectory path: the skill produces a useful step-by-step view from just the tree (a raw agent run), not only a fully-compiled ARA — see references/parsing.md §7.
  3. Parse the trace into normalized nodes. The field conventions vary across ARAs — follow references/parsing.md exactly (handles tree: vs root:, generic vs type-named fields, evidence: routing, isolated, also_depends_on). Every node must yield a title + body.
  4. Parse the hub layers — each only when present (all optional now): logic/claims.md (the binding hub when it exists), logic/experiments.md, evidence/README.md (figure/table ↔ claim reverse index), src/artifacts.md, logic/solution/*. A missing layer simply contributes nothing; the node still renders from its own title/body/thinking. Also parse the four OPTIONAL enrichment layers when present, per references/parsing.md §8: logic/problem.mdcontext, logic/concepts.mdglossary (+ build the lexicon), logic/related_work.mddependencies, logic/solution/*.mdrecipes (role-classify by content, not filename). A missing file/dir omits its key entirely. Reproduce statements/deltas/definitions/relations/ headings/quotes/cells verbatim.
  5. Build each node's drill-down. When logic/claims.md exists, follow the claim-hub chain in references/binding.md (node → evidence:[C##] → claims → {Sources quotes, figures/tables, experiments, artifact pointers}). When it does not, the drill-down is just the node's own narrative (thinking/body) — every claim/result/verified block is empty and omitted. 5b. Bind the enrichment layers per the "four enrichment layers" section of references/binding.md: build claimIds/nodeByClaim/conceptNames/rwIds; resolve every refs[].target (drop danglers, never link off-ARA); derive each node's built_on/rejected_here (dependency→claim→node, bucketed by relation_norm), concepts (whole-word name-match), and recipe_refs (recipe→claim→node); mark cross-agent entries. All per-node enrichment fields default []. 5c. Write each step's narrative as plain language (same layout, human words). The trace's notes are written for an agent; rendered as-is they read like a log and a person can't follow what happened or why it mattered. For each node, write its narrative — thinking, and body if used — in plain language a reader who has NOT seen the ARA can follow: your own words, translating the trace's agent-facing deliberation, not a verbatim paste; expand jargon on first use and state the point, not the log line. This changes ONLY the prose that fills the existing reasoning block — keep every block and the layout exactly as they are. Stay grounded: introduce no number, name, or claim that is not already in that node, and keep claim Statements, Sources quotes and table numbers verbatim in the why/result blocks — those are the receipts.
  6. Inline figures. For each referenced figure that has a real raster (evidence/figures/*.png), base64-encode it and put the data: URI in figures[].img. Use Bash, e.g. python3 -c "import base64,sys;print('data:image/png;base64,'+base64.b64encode(open(sys.argv[1],'rb').read()).decode())" <path>. For data-only figure markdown (no raster), render its data table instead (as a tables[] entry). Carry each node's thinking (the plain-language narrative from 5c) through, and sanitize it per the Injection contract. The ARA carries no code diff — a step's code change is conveyed by its natural-language narrative (body/thinking); the code itself is pointed at via src/artifacts.md.
  7. Assemble ARA_DATA (exact schema in references/binding.md) and inject it: replace ONLY the JSON between /* __ARA_DATA_BEGIN__ */ and /* __ARA_DATA_END__ */ in the <script id="ara-data"> block of a copy of the template. Write the result to the output path. Include context/glossary/dependencies/recipes and the per-node built_on/rejected_here/ recipe_refs/concepts only when their sources exist — omit absent keys entirely (no empty stubs). A payload omitting all of them stays byte-compatible with the v1.0 schema.
  8. Report the output path. Optionally open it (open <path> on macOS). Print a one-line summary (node count, dead ends, figures inlined, which of the four enrichment overlays were emitted with their term/dependency/recipe counts, danglers dropped, any pointers left unresolved).

Injection contract (critical)

  • The injected payload MUST be valid JSON (it is read with JSON.parse). The template strips only the two named marker comments before parsing, so the payload is otherwise pure JSON.
  • It must not contain the literal substring </script>, nor the literal marker strings /* __ARA_DATA_BEGIN__ */ / /* __ARA_DATA_END__ */. Escape any < in inlined markdown/text as &lt; (or <) — this also neutralizes </script>. (A bare */ inside a string value is harmless to JSON.parse; only the exact marker strings would be stripped.)
  • Do not touch anything else in the template — only the bytes between the two markers.
  • After writing, re-validate: the file still parses (the embedded JSON loads). If a figure pushed the file very large, apply the size guards in references/binding.md (truncate logs/tables, keep figures).

Faithfulness (hard rules)

  • Speak human in the narrative, quote the evidence. A node's narrative (thinking/body) is plain language — your own words, a grounded translation (5c), not a verbatim paste. Everything that is evidence — claim Statements, Sources quotes, table cells/numbers, relations, definitions — is reproduced verbatim in the why/result/overlay blocks. The narrative explains; the receipts prove. A narrative that states a number absent from the node fails; so does an evidence block that paraphrases.
  • Reproduce claim Statements, Sources quotes, and table numbers verbatim — never paraphrase, never invent. Missing data → set the field empty/omit (the viewer shows "No …"); never fabricate.
  • Provenance, support_level, and status are shown only if present in the source; do not guess.
  • Dead-end nodes and isolated subtrees must be carried through faithfully — they are the most valuable things to display, not noise to drop.
  • For the enrichment layers: relation strings, definitions, constraint headings, and footprint citations are reproduced verbatim; relation enums are open (compound bounds / refutes / transition extends → quarantined kept as written; relation_norm is for color only). Never normalize a heading or invent a typed sub-field. A refs[].target is set only on real in-ARA resolution; dangling refs are flagged, never silently corrected or dropped. built_on/concepts/"used by" name-matches are best-effort hints (marked "inferred"), never asserted as facts.

Verify

Run on any ARA and confirm these properties — no named fixtures required:

  • Opens by double-click: no server, no network, no console errors.
  • Full process map: nesting, branches, dead ends marked, any isolated subtree boxed, depends_on chips.
  • Drill-down renders whichever blocks are present (what / why / result-with-inline-figure / how-verified / code-or-pointer), correctly under both field dialects in references/parsing.md.
  • Verbatim quotes/numbers; nothing fabricated; self-contained from the ARA dir (no needed external refs).
  • Re-running reproduces the same structure (data differs only as the ARA differs).
  • Enrichment layers: a layer's header button appears only when its source exists; an ARA with none of the four layers renders identically to v1.0 (no layer bar, no node chips). Open each emitted overlay and confirm verbatim relations/definitions/recipe cells, ungrounded/dangling/cross-agent markers, and that the built_on/rejected_here chips + the ⊕/⊘ map marker deep-link into Dependencies. Glossary popovers fire on body terms; inline $LaTeX$ renders with no network.
  • Degradation: a minimal-artifact (only problem.md) shows only the Context button, others absent, popovers off, per-node chips empty, zero console errors.
  • Compile-first (non-ARA input): pointing the skill at raw research material with no exploration tree (a paper, a repo, a run/log dir) triggers the compiler skill first, then visualizes the resulting ARA — the output is identical to running the compiler then the visualizer by hand.
  • Raw trajectory (the decoupled path): a tree-only ARA — just trace/exploration_tree.yaml, no PAPER.md, no logic/, no evidence/ — still renders the full process map + each step's narrative (thinking/body), with no layer bar and no claim/result/verified blocks, zero console errors. This is a first-class supported input, not a failure mode.

Cover the variant axes with whatever ARAs you have: both root forms (tree:/root:), both field dialects (generic / type-named), figures present as real raster vs. data-markdown-only, src/ as a pointer index vs. transcribed code, and an isolated subtree if any artifact has one.

作为ARA的语义认知审查员,对已通过结构验证的研究工件进行深度评估。依据证据相关性、可证伪性等六个维度,通过语义推理生成评分报告与改进建议,输出最终接受或拒绝建议。
level2 seal level 2 verify level 2 epistemic audit review ara audit claims
skills/rigor-reviewer/SKILL.md
npx skills add ARA-Labs/Agent-Native-Research-Artifact --skill rigor-reviewer -g -y
SKILL.md
Frontmatter
{
    "name": "rigor-reviewer",
    "metadata": {
        "version": "3.0.0",
        "category": "research-tooling",
        "last_updated": "2026-04-16"
    },
    "description": "ARA Seal Level 2: Semantic Epistemic Review. Acts as an objective research\nreviewer for Agent-Native Research Artifacts. Assumes Level 1 structural\nvalidation has already passed. Evaluates six dimensions of epistemic quality\nthrough semantic reasoning over the ARA's content. Produces a scored review\nwith per-dimension strengths\/weaknesses\/suggestions, severity-ranked findings,\nand an overall recommendation (Strong Accept to Reject).\n\nTRIGGERS: level2, seal level 2, verify level 2, epistemic audit, review ara, audit claims\n",
    "allowed-tools": "Read, Write, Glob, Grep",
    "argument-hint": "<artifact_dir>",
    "user-invocable": true
}

ARA Seal Level 2: Semantic Epistemic Review

You are an objective research reviewer for Agent-Native Research Artifacts. You receive an ARA directory path and produce a comprehensive review as level2_report.json at the artifact root. You operate entirely through your native tools (Read, Write, Glob, Grep). You do NOT execute code, fetch URLs, or consult external sources.

Prerequisite: Level 1 (structural validation) has already passed. All references resolve, required fields exist, the exploration tree parses correctly, and cross-layer links are bidirectionally consistent. Level 2 does NOT re-check any of this. Instead, it evaluates whether the content of the ARA is epistemically sound: whether evidence actually supports claims, whether the argument is coherent, and whether the research process is honestly documented.

Your review is constructive: identify both strengths and weaknesses, provide actionable suggestions, and give a calibrated overall assessment. You are not a bug detector; you are a reviewer who helps authors improve their work.


Six Review Dimensions

Each dimension is scored 1-5 and includes strengths, weaknesses, and suggestions. All checks are semantic: they require reading comprehension and reasoning, not structural validation.

Dimension What it evaluates
D1. Evidence Relevance Does the cited evidence actually support each claim in substance, not just by reference?
D2. Falsifiability Quality Are falsification criteria meaningful, actionable, and well-scoped?
D3. Scope Calibration Do claims assert exactly what their evidence supports, no more, no less?
D4. Argument Coherence Does the narrative follow a logical arc from problem to solution to evidence?
D5. Exploration Integrity Does the exploration tree document genuine research process, including failures?
D6. Methodological Rigor Are experiments well-designed with adequate baselines, ablations, and reporting?

Procedure

Step 1: Read the ARA

Read files in this fixed order. Record the list as read_order in the report.

  1. PAPER.md
  2. logic/claims.md
  3. logic/experiments.md
  4. logic/problem.md
  5. logic/concepts.md
  6. logic/solution/architecture.md, algorithm.md, constraints.md, heuristics.md
  7. logic/related_work.md
  8. trace/exploration_tree.yaml
  9. evidence/README.md (if exists)
  10. Spot-check 2-3 evidence files from evidence/tables/ or evidence/figures/

Step 2: Parse Entities

Claims (from logic/claims.md): each ## C{NN}: {title} section. Extract:

  • Statement, Status, Falsification criteria, Proof (experiment IDs), Dependencies (claim IDs), Tags

Experiments (from logic/experiments.md): each ## E{NN}: {title} section. Extract:

  • Verifies (claim IDs), Setup, Procedure, Metrics, Expected outcome, Baselines, Dependencies

Heuristics (from logic/solution/heuristics.md): each ## H{NN} section. Extract:

  • Rationale, Sensitivity, Bounds, Code ref

Observations and Gaps (from logic/problem.md): each O{N} and G{N}.

Exploration tree (from trace/exploration_tree.yaml): all nodes with id, type, title, and type-specific fields (failure_mode, lesson, choice, alternatives, result).

Step 3: Build Working Maps

Construct these maps as inputs for semantic analysis. Do NOT validate structural integrity (Level 1 guarantees it).

  • claim_proof_map: for each claim, the set of experiment IDs in its Proof
  • experiment_verifies_map: for each experiment, the set of claim IDs in its Verifies
  • claim_dependency_edges: directed edges from each claim to its Dependencies
  • gap_set: all G{N} from problem.md
  • rejected_nodes: exploration tree nodes with type = dead_end or pivot
  • decision_nodes: exploration tree nodes with type = decision

Step 4: Evaluate Each Dimension

For each dimension, perform semantic reasoning over the parsed content. Record strengths, weaknesses, and suggestions as you go.


D1. Evidence Relevance

For each claim-experiment pair linked through Proof/Verifies:

  • Relevance: Does the experiment's Setup/Procedure/Metrics actually address what the claim asserts? (Not just "link exists" but "link is substantively relevant.")
  • Type-aware entailment: Infer claim type from Statement cues, check experiment design matches:
    • Causal ("causes", "leads to", "enables") → needs isolating ablation
    • Generalization ("generalizes", "robust", "across") → needs heterogeneous test conditions
    • Improvement ("outperforms", "better", "improves") → needs baseline comparison
    • Descriptive ("accounts for", "distribution", "pattern") → needs representative sampling
    • Scoping ("when", "under conditions", "limited to") → needs declared bounds
  • Evidence sufficiency: Is a single experiment enough to support this claim, or does the claim's scope demand multiple independent experiments?

Scoring anchors:

  • 5: Type-appropriate, relevant evidence for every claim; multi-experiment support where needed
  • 4: Evidence relevant for all claims, minor type mismatches (e.g., causal claim with correlation-only evidence)
  • 3: Most claim-experiment pairs are relevant, 1-2 weak matches where evidence doesn't quite address the claim
  • 2: Multiple claims where cited experiments don't substantively address what the claim asserts
  • 1: Majority of claims cite experiments that are irrelevant to their statements

D2. Falsifiability Quality

For each claim's Falsification criteria field:

  • Actionability: Could an independent researcher execute this criterion? Does it specify what to measure, what threshold constitutes failure, and under what conditions?
  • Non-triviality: Is the criterion non-tautological? ("If the method doesn't work" is trivial. "Re-evaluation on the same 77-paper set where GPT-5 is not the top model" is actionable.)
  • Scope match: Does the falsification criterion address the same scope as the Statement? (A claim about "all datasets" with falsification mentioning only one dataset is mismatched.)
  • Independence: Could the criterion be tested without access to the authors' proprietary data or systems?

Scoring anchors:

  • 5: Every claim has specific, actionable, independently testable falsification criteria matching the claim's scope
  • 4: Most criteria are strong, 1-2 are vague or hard to operationalize
  • 3: Mixed quality; some actionable, some trivial or scope-mismatched
  • 2: Most criteria are trivial, tautological, or scope-mismatched
  • 1: Falsification criteria meaningless across claims

D3. Scope Calibration

  • Over-claiming: Does any Statement use universal scope markers ("all models", "any dataset", "state-of-the-art across all") while cited experiments cover only specific, narrow conditions? The gap must be substantial.
  • Under-claiming: Are there important experimental results present in evidence/ that are not captured by any claim? (Evidence without a corresponding claim.)
  • Attribution vs mechanism: Does any Statement merely name which components of this one system rank highest/lowest (load-bearing, dominant, decorative, inert) without stating what that ranking reveals? Apply the name-deletion test — strike the system's component names; if no transferable relationship survives, the Statement is attribution, not insight. Flag as major (the claim is a league table of this system, not a reusable finding); suggest the generalization the ranking licenses.
  • Assumption explicitness: Are key assumptions stated in problem.md (Assumptions section) or constraints.md? Are there unstated assumptions implied by the experimental design?
  • Generalization boundaries: Does the artifact clearly state what the claims do NOT apply to? Check constraints.md and limitations in the exploration tree.
  • Qualifier consistency: When claims use hedging ("tends to", "in most cases"), is this consistent with the evidence strength?

Scoring anchors:

  • 5: All claims precisely match evidence scope, assumptions explicit, limits clearly stated
  • 4: Claims well-scoped with minor gaps in assumption documentation
  • 3: Some claims slightly over/under-reach, assumptions partially stated
  • 2: Multiple over-claims or significant undocumented assumptions
  • 1: Pervasive scope mismatch between claims and evidence

D4. Argument Coherence

  • Observation → Gap derivation: Do the stated gaps follow logically from the observations? Or are they asserted without connection?
  • Gap → Insight connection: Does the key insight in problem.md address the identified gaps?
  • Insight → Solution alignment: Does the solution architecture implement the key insight?
  • Solution → Claims coverage: Do the claims cover the solution's main contributions?
  • Cross-layer consistency: Do claims, exploration tree, and evidence tell the same story? Flag contradictions.
  • Narrative completeness: Are there motivating questions from problem.md that are neither answered nor explicitly deferred?
  • Gap coverage: For each gap in problem.md, is there at least one claim that substantively addresses it? Flag gaps that are motivated but never resolved.

Scoring anchors:

  • 5: Clear logical arc (observations → gaps → insight → solution → claims → evidence), all gaps addressed, no contradictions
  • 4: Strong flow with minor logical gaps or one unaddressed gap
  • 3: General flow present but some disconnects between layers
  • 2: Significant misalignment between problem statement and claims, or unresolved contradictions
  • 1: No coherent logical flow; layers tell different stories

D5. Exploration Integrity

  • Dead-end quality: Is the failure_mode specific enough to be actionable? ("Didn't work" is bad. "Divergence after 1000 steps due to gradient explosion" is good.) Is the lesson a genuine transferable insight?
  • Decision rationale quality: Do rationales explain WHY the chosen path was preferred over alternatives? Are alternatives real alternatives or strawmen?
  • Rebutted-branch consistency: Does any claim advocate an approach marked as dead_end or pivot in the tree? (This is a logical contradiction.)
  • Exploration breadth: For the paper's main design choices, were at least 2 alternatives considered and documented?
  • Honesty signal: Does the tree document genuine negative results, or does it read like a post-hoc justification? A tree with zero dead-ends or only trivial failures is suspicious.

Scoring anchors:

  • 5: Rich tree with well-documented dead-ends (specific failure modes, actionable lessons), thorough decision rationale, genuine negative results
  • 4: Good tree with minor gaps in dead-end documentation or decision rationale
  • 3: Tree present but dead-ends lack specificity or decisions lack alternatives
  • 2: Boilerplate documentation; dead-ends and decisions read as formulaic rather than authentic
  • 1: Tree contradicts claims or reads entirely as post-hoc justification

D6. Methodological Rigor

  • Baseline adequacy: Are the right things being compared? Are baselines recent and relevant? Flag experiments with "no baseline" for comparative claims.
  • Ablation coverage: For claims involving multiple components, does at least one experiment isolate individual contributions?
  • Statistical reporting: Do experiments mention variance, confidence intervals, number of runs, or statistical tests? Flag single-run results for quantitative claims.
  • Metric-claim alignment: Does the metric actually measure what the claim asserts? (A claim about "generalization" measured only by accuracy on one test set is misaligned.)
  • Reproducibility signals: Are experiment setups specific enough for independent replication? (Model name, dataset, hardware, hyperparameters.)

Scoring anchors:

  • 5: Comprehensive baselines, proper ablations, statistical rigor, metrics precisely match claims, fully reproducible setup
  • 4: Strong methodology with minor gaps (e.g., missing variance on one experiment)
  • 3: Adequate but missing some baselines or statistical details
  • 2: Significant gaps; missing baselines for comparative claims or no ablations
  • 1: No baselines, no ablations, metrics don't match claims

Step 5: Compile Findings

Collect all issues found across the six dimensions into a single findings list. Assign each finding:

  • finding_id: F01, F02, ... (sequential)
  • dimension: which of D1-D6
  • severity: one of:
    • critical — fundamental epistemic flaw; the claim or argument cannot stand as written
    • major — significant weakness that undermines a claim or dimension score
    • minor — noticeable issue that doesn't invalidate the work
    • suggestion — constructive improvement opportunity, not a flaw
  • target_file: which ARA file
  • target_entity: C{NN}, E{NN}, H{NN}, G{N}, or node ID (if applicable)
  • evidence_span: verbatim substring from the ARA that triggered the finding (MUST be exact quote; omit if the finding is about an absence)
  • observation: what you found (factual)
  • reasoning: why it matters (analytical)
  • suggestion: how to fix or improve it (constructive)

Sort findings by severity: critical first, then major, minor, suggestion.

Step 6: Compute Overall Grade

Calculate the mean of the six dimension scores. Apply the grade mapping:

Grade Condition
Strong Accept mean ≥ 4.5 AND no dimension < 3
Accept mean ≥ 3.8 AND no dimension < 2
Weak Accept mean ≥ 3.0 AND no dimension < 2
Weak Reject mean ≥ 2.0 AND (mean < 3.0 OR any dimension < 2)
Reject mean < 2.0 OR any dimension = 1

Step 7: Write Report

Write level2_report.json to the artifact root:

{
  "artifact": "<name>",
  "artifact_dir": "<path>",
  "review_version": "3.0.0",
  "prerequisite": "Level 1 passed",

  "overall": {
    "grade": "Accept",
    "mean_score": 4.1,
    "one_line_summary": "<1 sentence: what makes this ARA strong or weak>",
    "strengths_summary": ["<top 2-3 strengths across all dimensions>"],
    "weaknesses_summary": ["<top 2-3 weaknesses across all dimensions>"]
  },

  "dimensions": {
    "D1_evidence_relevance": {
      "score": 4,
      "strengths": ["Evidence is substantively relevant for all 6 claims"],
      "weaknesses": ["C02 cites a correlation study but makes a causal claim"],
      "suggestions": ["Add an ablation experiment to isolate the causal mechanism for C02"]
    },
    "D2_falsifiability": {
      "score": 4,
      "strengths": ["..."],
      "weaknesses": ["C02 falsification criteria is hard to operationalize independently"],
      "suggestions": ["Specify a concrete re-annotation protocol for C02"]
    },
    "D3_scope_calibration": { "score": 4, "..." : "..." },
    "D4_argument_coherence": { "score": 4, "..." : "..." },
    "D5_exploration_integrity": { "score": 3, "..." : "..." },
    "D6_methodological_rigor": { "score": 4, "..." : "..." }
  },

  "findings": [
    {
      "finding_id": "F01",
      "dimension": "D6_methodological_rigor",
      "severity": "major",
      "target_file": "logic/experiments.md",
      "target_entity": "E03",
      "evidence_span": "**Baselines**: No random or retrieval-only baseline reported",
      "observation": "E03 evaluates four LLMs on research ideation but includes no non-LLM baseline.",
      "reasoning": "Without a random or retrieval-only baseline, it is impossible to assess whether LLM performance is meaningfully above chance.",
      "suggestion": "Add a retrieval-only baseline (e.g., BM25 nearest-neighbor from predecessor abstracts) to contextualize Hit@10 scores."
    }
  ],

  "questions_for_authors": [
    "What is the inter-annotator agreement on thinking-pattern classification? A single LLM pass without human validation on the full corpus leaves taxonomy reliability uncertain.",
    "..."
  ],

  "read_order": ["PAPER.md", "logic/claims.md", "..."]
}

Critical Rules

  1. Verbatim evidence_span: Findings about content present in the ARA MUST quote an exact substring. Findings about absences (missing baseline, scope mismatch) may omit evidence_span.

  2. Constructive tone: Every weakness must come with a suggestion. You are helping authors improve, not punishing them.

  3. Calibrated scoring: Most competent ARAs should land in the 3-4 range. A score of 5 means genuinely excellent, not just "no problems found." A score of 1 means fundamental problems, not just "could be better."

  4. No false grounding: Support must flow through Proof → experiments.md → evidence/. Agreement in prose (problem.md, architecture.md) does not substitute for experimental evidence.

  5. Artifact-only: Do not fetch external URLs, execute code, or consult external sources. Take the ARA's reported evidence at face value.

  6. Balanced review: Actively look for strengths, not just weaknesses. A review that only lists problems is not useful.

  7. No structural re-checks: Do NOT verify reference resolution, field presence, YAML parsing, or cross-link consistency. Level 1 has already validated all of this. Focus entirely on whether the content is epistemically sound.


Reference

See references/review-dimensions.md for scoring anchor details and check inventories per dimension.

将研究目录验证、编译并生成可视化后,发布至用户公共GitHub仓库并注册到ARA Hub。
提交 ARA 发布 ARA 上传 ARA 分享 ARA 推送到 GitHub 添加到 ARA Hub
skills/submit-ara/SKILL.md
npx skills add ARA-Labs/Agent-Native-Research-Artifact --skill submit-ara -g -y
SKILL.md
Frontmatter
{
    "name": "submit-ara",
    "metadata": {
        "tags": [
            "research",
            "publishing",
            "github",
            "ara-hub",
            "visualization"
        ],
        "author": "ara-commons",
        "version": "1.0.0",
        "category": "research-tooling"
    },
    "description": "ARA Submitter \/ Publisher. Takes a research directory, makes sure it is a valid Agent-Native\nResearch Artifact (ARA) — compiling or updating it with the `compiler` skill when it is not —\nguarantees it carries an interactive visualization (running `research-visualizer` to produce\n`trajectory.html` when missing), then publishes it as a public GitHub repository on the\nauthenticated user's own account and links it into the ARA Hub website so others can browse and\nreplay it. GitHub is the data layer; the Hub fetches from it.\n\nTRIGGERS: submit, submit ara, publish ara, upload ara, share ara, push ara to github, add to\nara hub, submit-ara, publish artifact, make my ara public\n",
    "allowed-tools": "Read, Write, Edit, Glob, Grep, Bash(gh *|git *|python3 *|curl *|cat *|ls *|find *|mkdir *|cp *|rm *|test *|basename *|jq *|open *)",
    "argument-hint": "[ara-dir] [--name <repo>] [--owner <login>] [--private] [--no-viz] [--update]"
}

ARA Submitter / Publisher

You take a research directory and get it published and viewable: validate (or build) the ARA, guarantee it has a visualization, push it to the user's own public GitHub repo, and register it with the ARA Hub. You are a first-class agent — use your native tools (Read, Write, Bash, Glob, Grep) directly, and invoke the compiler / research-visualizer skills when this skill's steps call for them.

Set expectations FIRST (do this before anything else)

The first thing you output, before running any step, is a clear time notice — publishing is slow when it has to compile and/or visualize:

Publishing an ARA can take ~15 minutes. I may need to (1) compile your input into the ARA format, (2) generate the interactive visualization (figure rendering can be slow), and (3) create and push a GitHub repository. I'll report progress at each step — please be patient and keep this session open.

Then announce each of the 6 steps as you start it (▶ Step 2/6: …) so the user can follow along.

Arguments

Interpret $ARGUMENTS flexibly:

  • First positional path → the ARA directory (or raw research input). Default: the ARA most-recently referenced in context, else the single dir under ./ara-output/, else ask.
  • --name <repo> → override the GitHub repo name (default: derived slug, see Step 4).
  • --owner <login> → override the GitHub account to publish under (default: the authenticated gh user).
  • --private → create a private repo (default: public — the Hub can only fetch public repos).
  • --no-viz → skip visualization (only if the dir already has one, or the user opts out).
  • --update → the repo already exists; update it in place instead of failing on "already exists".

Workflow (6 steps)

1. RESOLVE the input directory
2. VALIDATE it is an ARA  → if not (or incomplete), COMPILE / UPDATE with the `compiler` skill
3. VISUALIZE → ensure trajectory.html exists; if not, run the `research-visualizer` skill
4. PREFLIGHT GitHub → gh auth, resolve owner + repo slug
5. PUBLISH → create the public repo and push the ARA (incl. trajectory.html)
6. REGISTER + REPORT → emit a Hub registry entry; print repo URL + Hub viewer URL

Step 1 — Resolve the input

Resolve the directory argument to an absolute path. Confirm it exists and is a directory. The directory's name and location are irrelevant — it can live anywhere (inside this repo's working tree, under ara-output/, or any other path) and be named anything. The only thing that matters is that it carries a valid ARA structure (verified in Step 2). Regardless of where it lives, you'll always publish a clean copy and never git init inside the user's working tree.

Step 2 — Validate it is an ARA (compile/update if not)

Use ONE observable test, identical to the visualizer's precondition: does the input expose a parseable trace/exploration_tree.yaml with ≥1 node, in a standard ARA layout (PAPER.md, logic/, src/, trace/, evidence/)?

  • It is a complete ARA → continue to Step 3.
  • It is not an ARA (raw paper/PDF, repo, run logs, notes, or a dir with no exploration tree) → invoke the compiler skill on it to produce an ARA under ./ara-output/<slug>/, then set the ARA dir to the compiler's output. Do not hand-roll an ARA — the compiler is the only builder.
  • It is an ARA but incomplete or stale (missing mandatory-core files, or the user passed --update after editing sources) → invoke the compiler skill to update/fill the same directory, then continue.

Run a light Seal Level 1 sanity check (mandatory-core dirs/files present and non-empty; tree parses). If it still fails after a compiler pass, surface the specific gaps and stop — do not publish a broken artifact.

Step 3 — Ensure the visualization

The Hub's value is the step-by-step replay, so every published ARA must ship its visualization.

  • Check for a self-contained visualization at <ara-dir>/trajectory.html (this is the research-visualizer default output).
  • Missing (and not --no-viz) → invoke the research-visualizer skill with the ARA dir, writing to <ara-dir>/trajectory.html. This is typically the slowest step (figure inlining / rendering) — tell the user it's underway.
  • Present → keep it. Offer to regenerate only if the ARA changed in Step 2.

trajectory.html is single-file and self-contained, so the Hub can load it straight from the published repo over a CDN (no server, no build). It MUST be included in what you push in Step 5.

Step 4 — Preflight GitHub

  • Run gh auth status. If not authenticated, STOP and tell the user to authenticate first — suggest they run ! gh auth login in this session (the ! prefix runs it inline). Do not try to log in for them.
  • Resolve owner: --owner if given, else gh api user -q .login (the user's own account — this is the default by design: every user publishes to the account they already hold auth for).
  • Resolve repo slug: --name if given, else derive from PAPER.md title: → kebab-case, strip punctuation, prefix ara-, cap length (~50 chars). Fallback: the ARA dir basename. Example: "Andes: Defining and Enhancing QoE…"ara-andes-defining-and-enhancing-qoe.
  • Check existence: gh repo view "<owner>/<slug>". If it exists and --update was NOT passed, ask the user whether to update it or pick a new name — don't clobber silently.

Step 5 — Publish to GitHub

Publish a clean copy of the ARA dir as a new public repo. Never run git init inside the user's working tree or inside ara-output/. Full command sequence (copy-paste-safe, idempotent) is in ${CLAUDE_SKILL_DIR}/references/upload-and-hub.md §Publish. In summary:

  1. Copy the ARA dir into a scratch staging dir.
  2. Add a short README.md (title + abstract + a "View on ARA Hub" link) and a .gitignore (trajectory.html stays — do NOT ignore it).
  3. git init, commit, branch main.
  4. gh repo create "<owner>/<slug>" --public --source <stage> --remote origin --push --description "<title> — Agent-Native Research Artifact".

The default branch is main; the Hub fetches trajectory.html from main via jsDelivr.

Step 6 — Register with the Hub + report

The Hub only learns a new ARA exists when this skill tells it via the submit API. Registration is a REQUIRED step, not optional — a published repo that is never registered will never appear on the Hub landing page. Details in ${CLAUDE_SKILL_DIR}/references/upload-and-hub.md §Register.

  • Always POST — zero setup required: build the entry JSON (schema in the reference) and curl it to ${ARA_HUB_API:-https://www.evolvinglab.ai}/api/submit with Content-Type: application/json. ARA_HUB_API defaults to the live Hub (https://www.evolvinglab.ai); the user does not need to set anything. Only override it to target a local/staging Hub.
  • Verify the response: a success is HTTP 201 with {"ok":true,"backend":"supabase"}. Print the HTTP status and ok/backend so it is visible the Hub actually accepted it. Do not claim the ARA is "on the Hub" unless this POST returned 2xx.
  • Fallback — only if that POST fails (Hub unreachable, non-2xx, or ok:false): say so explicitly, print the entry JSON so the user can retry, and, when the hub repo (docs/ara-hub/) is checked out locally and writable, append the entry to docs/ara-hub/data/registry.json (the artifacts array).

Final report (always print these):

  • GitHub repo: https://github.com/<owner>/<slug>
  • 🎞️ Visualization (direct): https://cdn.jsdelivr.net/gh/<owner>/<slug>@main/trajectory.html
  • 🌐 ARA Hub viewer: ${ARA_HUB_URL:-https://www.evolvinglab.ai}/ara?repo=<owner>/<slug> (ARA_HUB_URL defaults to the live Hub at https://www.evolvinglab.ai)
  • 📊 Stats: claims / experiments / tree nodes / evidence figures, and whether the ARA was compiled or visualized in this run.

Critical rules

  1. Set the ~15-minute expectation up front — before any step, every time.
  2. Never publish a broken ARA — Step 2's validation must pass (after at most one compiler pass) before Step 5. If it can't, report the gaps and stop.
  3. Publish from a clean copy — never git init in the user's working tree or in ara-output/; stage to scratch, then push.
  4. Public by default — the Hub can only fetch public repos; warn explicitly if --private.
  5. The user's own account is the default owner — they hold the auth; GitHub is the shared data layer. Only override with --owner.
  6. trajectory.html ships with the repo — it is the thing the Hub renders. Generate it (Step 3) and include it in the push (Step 5); never .gitignore it.
  7. Don't reinvent the builders — compiling is the compiler skill's job, visualizing is the research-visualizer skill's job. This skill orchestrates; it does not hand-roll either output.
  8. Idempotent + non-destructive — check repo existence before creating; require --update to touch an existing repo; never force-push without telling the user.
  9. Registration is mandatory and verified — Step 6's POST /api/submit is what makes the ARA show up on the Hub; it is not optional. Confirm it returned HTTP 2xx with ok:true before reporting success. If it failed, say the ARA is published to GitHub but not yet on the Hub, and give the retry command. Pushing to GitHub alone does NOT put it on the landing page.

Reference files

Load on demand:

  • ${CLAUDE_SKILL_DIR}/references/upload-and-hub.md — exact gh/git publish commands, slug derivation, the Hub registry entry schema + jsDelivr/Hub URL contract, and the database seam.

trang chủ - Wiki
Copyright © 2011-2026 iteam. Current version is 2.155.2. UTC+08:00, 2026-07-07 17:30
浙ICP备14020137号-1 $bản đồ khách truy cập$