Oleafly Verify Claims

This skill answers one question for every factual sentence in a section: does the source it cites actually say that? The output is a table, a report, and a list of proposed fixes. It is not a rewrite.

Never edit the user's prose in this skill. A claim audit that silently corrects what it finds destroys the evidence of what was wrong, and the user cannot check your judgement afterwards. Propose the fix, name the file and line, and let them decide. The only files this skill writes are research/claims.md and a report under review/.

Step 1. Fix the scope

Audit one section, or one file, at a time. A whole manuscript in one pass produces a table nobody reads.

Confirm with the user which file, or infer from what they just asked about. project_map with {} returns sections with the file and line of each heading, so you can name the boundaries precisely.

Set the plan with update_todos, one item per step below.

Step 2. Read the text with line anchors

read_file with {"path": "sections/related.tex", "offset": 1, "limit": 200}. The result carries offset, lines_returned and total_lines. Line N of the returned content is file line offset + N - 1. Read further slices until you have covered the section.

To pin a line number exactly, search_project with {"query": "<six distinctive words from the sentence>"} returns the file and line of every match. That is more reliable than counting, and it is the anchor to record in the table.

Step 3. Extract the claims

Go sentence by sentence. A claim is anything a reader could be wrong to believe.

Audit these:

• A statement about what prior work did, showed, achieved or failed to handle.
• A number, a rate, a size, a threshold, a benchmark result, a date.
• A statement about the state of a field. "Most approaches assume..." is a claim about a population.
• A causal statement. "Because X, Y follows."
• A statement about what is standard, common, or widely used.
• A definition attributed to a source.

Skip these:

• The paper's own contributions and results, which are evidenced by the paper itself, not by a citation.
• Statements of intent. "We now describe the method."
• Pure mathematics that is derived in place.
• Hedged framing that asserts nothing. "One way to see this is..."

Number the claims C1, C2 and onward. Quote each one, or paraphrase tightly enough that the user recognises the sentence.

Step 4. Map each claim to its cited key

For each claim, record which keys the sentence cites. A claim with no citation is not automatically a problem, but it has to be classified deliberately (step 6).

project_map with {} gives bibKeys and unresolvedCites. Any key on the claim list that appears in unresolvedCites is already broken before the content question is asked: it cites nothing. Record it as unsupported with the reason "no bibliography entry".

Step 5. Gather the evidence

For each cited key, in this order, stopping when you have enough to judge:

1. read_file with {"path": "research/sources/<key>.md"}. If the sweep ran, this holds the abstract verbatim and the provenance. This is the cheapest real evidence available.
2. project_library_search with {"query": "<the claim, in the source's own vocabulary>"} searches the project's own files, including any imported full text and the user's reading notes.
3. read_file on the PDF at research/sources/<key>.pdf when the abstract does not settle it and the paper is in the project.
4. verify_citation with {"doi": "<doi from the entry>"} to confirm the entry resolves to a real record with the title, venue and year the entry claims. This checks the citation exists and matches; it does not check that the paper supports the claim. Both checks matter and they are different.

When no evidence is available at all, that is a finding, not a blocker. Record the claim as unsupported with the note "no source note, no full text in the project", and propose that the source be added by oleafly-literature-sweep.

Never judge from the title. Titles overstate, and a title that matches the claim is the most common way a wrong citation survives review.

Step 6. Assign a support level

Exactly one of three, defined in full in references/support-levels.md.

partial is the level that does the most work. Most citation problems in real manuscripts are not fabrications, they are a true finding stretched one step past what was measured.

The evidence note must say what the source actually says, not just that it disagrees. "Measured on graphs up to 10k nodes; the sentence says arbitrary scale" is actionable. "Overstated" is not.

Step 7. Write research/claims.md

Create the directory if needed with create_file and {"path": "research", "is_dir": true}, then write_file the table from assets/claims.md.

Columns: number, claim, location as file:line, cited key, support level, evidence note. Then a summary count, then a proposed fixes table.

If research/claims.md already exists, read_file it first and merge rather than replacing. An earlier audit's findings on other sections are not yours to discard.

Step 8. Optional deeper audit with the vendored script

scientific-writing ships scripts/audit_claims.py, which checks a Markdown manuscript against a structured claim registry and a source manifest. It is worth running when the project is Markdown and the user wants a machine checkable evidence trail, and it is overhead otherwise.

Before the first script call, check the interpreter once with run_command and {"command": "python3 --version"}. Below 3.11, skip this step and say so.

load_skill with {"id": "scientific-writing"} returns its absolute directory and file tree. It takes three positional arguments, in this order:

python3 "<scientific-writing dir>/scripts/audit_claims.py" manuscript.md research/claims.csv research/sources.json

• The manuscript, UTF-8 Markdown.
• A claim registry CSV with the columns claim_id, section, claim_kind, claim_text_sha256, evidence_ids, verification_status, uncertainty, analysis_intent. Template at assets/claim_evidence_template.csv inside that skill.
• A source manifest JSON with a sources array of records carrying evidence_id, title, year, authors, identifiers, locator and a verification block. Template at assets/source_manifest_template.json inside that skill.

Read either template with read_skill_file and {"id": "scientific-writing", "path": "assets/claim_evidence_template.csv"}.

Two related scripts from the same skill: scripts/check_references.py <sources.json> validates the manifest alone, and scripts/lint_manuscript.py <manuscript.md> checks the prose against the skill's writing rules.

Write script output to a file under research/ and read it back with read_file. Command output is capped and truncates silently.

The script checks structure and completeness. It does not read the sources. It never replaces steps 5 and 6.

Step 9. Report, and hard fail on the unsupported

Write the report to review/claim-audit-<section>.md from assets/audit-report.md, and give the user the summary in your reply.

Lead with the count at each level. Then list every unsupported claim in full, with its location and what is missing. These are the ones that must be resolved before submission, and burying them under the verified ones is how they get missed.

For each problem, propose a concrete fix and let the user choose:

• Narrow the sentence to what the source measured.
• Add a second citation that carries the rest of the claim.
• Replace the citation with one that actually supports it.
• Mark the statement as this work's own contribution rather than a citation.
• Remove the claim.

Propose. Do not apply. If the user then asks you to make the edits, do them with replace_in_file in that turn, one at a time, and compile afterwards.

Step 10. Confirm the document still builds

If you added nothing and edited no source, compile is not required. If the user accepted fixes and you applied them, compile with {}, then project_map with {} to confirm unresolvedCites is empty, then get_pdf_text with {} to confirm no citation renders as [?].

Step 11. Hand off

• The methodology behind a claim needs appraising, not just the citation: load_skill with {"id": "scientific-critical-thinking"}.
• Sources are missing and have to be found: load_skill with {"id": "oleafly-literature-sweep"}.
• The bibliography itself is inconsistent, duplicated or malformed: load_skill with {"id": "citation-management"}.
• The whole manuscript needs a reviewer's read, not a claim audit: load_skill with {"id": "oleafly-review-manuscript"}.

Failure handling

Done when

• Every factual claim in the scope has a row, with a file:line location that search_project confirms.
• Every row has a support level and an evidence note that says what the source actually says.
• Every unsupported claim is listed in full in the report, not only counted.
• Every proposed fix names the file, the line, and the specific change.
• research/claims.md and review/claim-audit-<section>.md are written.
• No sentence of the user's prose was changed without them asking for it in this turn.

Reference files

• references/support-levels.md defines the three levels with worked examples and the judgement calls between them.
• assets/claims.md is the audit table template.
• assets/audit-report.md is the report template.

From a terminal

Outside the app (a terminal, or a coding agent with a shell), the oleafly command does the same work. Always pass --json and check ok.