Validate napkin_math parameters.json

Overview

A thin wrapper around experiments/napkin_math/validate_parameters.py. The script reads parameters.json and emits validation.json next to it. Output shape is what summarize_assessment.py consumes (named checks_performed list + per-violation rule_id/severity/path/message/suggested_fix).

This replaces the older LLM-driven validate-parameters skill, which was written against an earlier schema and rejected the output_name/output_unit fields the digest extractor is required to emit. The Python validator runs in milliseconds, costs no tokens, and is deterministic.

When to Use

• The extractor (digest or full) has just produced parameters.json and the pipeline needs validation.json before running scenarios or Monte Carlo
• The user asks to "validate", "check", or "lint" a parameters file
• After hand-editing parameters.json, to confirm the result is still structurally valid

Not for: regenerating the parameters (use extract-parameters-from-digest or extract-parameters-from-full), generating bounds, or running calculations.

Workflow

1. Get the parameters path. If the user did not provide one, ask. Do not guess.

2. Invoke the script. Requires Python 3.11+ (no extra deps):

   /opt/homebrew/bin/python3.11 experiments/napkin_math/validate_parameters.py \
     --parameters <path>/parameters.json \
     [--output    <path>/validation.json]
   

   Default output: <dir-of-parameters>/validation.json. Exit code 0 on valid: true, 1 on validation errors, 2 on JSON parse failure. The script prints the output path on stdout.

3. Report back. Tell the user the output path, the verdict (valid / INVALID), and the error count. If there are violations, quote the first few rule_id + message pairs verbatim. Don't paraphrase the verdict — the rule ids are how downstream tooling routes the next step.

The 16 checks

valid is true iff error_count == 0. WARN-level findings do not invalidate the file.

Output shape

{
  "valid": true,
  "error_count": 0,
  "warn_count": 0,
  "violations": [],
  "summary": {
    "counts": {"key_values": 8, "derived_questions": 3, "missing_values_to_estimate": 4, "recommended_first_calculations": 5},
    "rule_id_breakdown": {},
    "checks_performed": ["json_parse", "top_level_structure", ...]
  }
}

The summary.checks_performed list is what summarize_assessment.py surfaces as the "Validated" line under ## Confidence and trust boundaries. Keep it as the authoritative list of what the validator actually ran.

Common mistakes

Reference

• Script (authoritative): experiments/napkin_math/validate_parameters.py
• Companion skills: ../extract-parameters-from-digest/SKILL.md, ../extract-parameters-from-full/SKILL.md, ../generate-bounds/SKILL.md, ../monte-carlo/SKILL.md, ../summarize-assessment/SKILL.md
• Example output: any validation.json under experiments/napkin_math/output/<version>/<plan>/