Edmund validation & QA

The discipline of proof. The central lesson: a green headless test is necessary but not sufficient for the bug classes that cost the most time here. Know which evidence each change actually requires.

Framework: Swift Testing (import Testing, @Test, #expect, #require) — NOT XCTest. Verified 2026-07-05: 810 @Test cases across Tests/EdmundTests/.

1. The evidence hierarchy

Ordered weakest → strongest. The rule at the bottom names which class each change type requires.

The trap that defines this skill: the delete-drift round-6 regression test passes with AND without the fix — the harness runs the queued selection fixup synchronously, so headless can't see the bug. For caret/IME/drag/viewport-timing bugs, class (a) is not evidence of a fix; you need (c)+(d).

Required evidence by change type (gates enforced in edmund-change-control):

2. Test-suite anatomy

Run: swift test (full suite; ARCHITECTURE cites ~750+ tests ≈10s — 810 @Test cases as of 2026-07-05). One suite: swift test --filter <Suite>. swift test also runs automatically as a Stop hook (.claude/settings.json) at the end of any turn touching code, so failures surface before you commit.

Map of Tests/EdmundTests/ (what the files actually cover):

• Parsing: BlockParserTests, SyntaxHighlighterTests, IncrementalParseFuzzTests, PendingEditTests, EscapeRenderingTests, HTMLTagRenderingTests, EmojiRenderingTests, LineEndingTests.
• Rendering / styling: BlockStylingTests, CalloutRenderingTests, CalloutTests, CommentRenderingTests, NestedBlockStylingTests, InlineStylingTests, MathRenderingTests, ImageRenderingTests, CodeHighlighterTests, FootnoteTests, WikiLinkTests, TableAlignmentTests, RenderingRegressionTests.
• Editor behavior: EditorIndentationTests, ListContinuationTests, BlockquoteContinuationTests, BlockquoteDeletionTests, FormattingTests, ActiveBulletMarkerTests, NewListItemAlignmentTests.
• Edit-pipeline integrity (the costly class): BypassedEditSyncTests, MarkedTextDesyncTests, WrappedParagraphCaretTests, EditorDiagnosticsTests, UnmatchedDebugTests, InternationalInputTests.
• Viewport / layout / undo: LazyRenderingTests, ScrollStabilityTests, HeightStabilityTests, TypewriterCenteringTests, UndoRedoViewportTests, EditorUndoTests, RecomposeTests, RecomposeEquivalenceTests.
• Export / read mode: HTMLRendererTests, DocumentHTMLTests, ReadModeWebViewTests, HTMLThemeTests, EditorThemeTests, ViewModeTests, ContentWidthTests.
• Infra / harness: LogTests, CrashReporterTests, PerfHarnessTests, StatusBarPrefsTests, FileIntegrationTests, EditorDocumentTests, TestHelpers.swift. _RenderDump.swift / _RenderEdit.swift are local dev tools (gitignored intent) that dump Read-mode HTML / edit output for tmp/sample.md — not part of the assertion suite.

Two invariant-guarding patterns worth copying:

• Recompose equivalence (RecomposeEquivalenceTests, helper assertMatchesFullRecomposeOracle): an incremental restyle must produce the same result as a full recompose. This is the safety net for the lazy / incremental styling paths.
• Incremental-parse fuzz (IncrementalParseFuzzTests): random edits must keep the parser's window-reparse consistent with a full reparse.

3. Test helpers (TestHelpers.swift)

@MainActor helpers you build on (verified exports):

styleBlock(_:cursorPosition:) is a method on EditorTextView (Rendering/EditorTextView+Rendering.swift), called from tests — it renders one block to an attributed string.

4. How to add a test

Skeleton (Swift Testing, @MainActor because the editor is main-thread):

import Testing
import AppKit
@testable import EdmundCore

@MainActor
@Test func deletingAtCalloutBottomKeepsInvariant() {
    let editor = makeEditor()
    editor.loadRawSource("> [!note]\n> body\n")
    drainAllStyling()
    // ... drive the edit ...
    #expect(editor.rawSource == editor.string)   // storage == rawSource invariant
}

Rules:

1. Every bug fix ships with a test even if it can't discriminate a live-only mechanism — it still locks the headless contract so a future refactor can't silently re-break the model half. For the live half, keep the frozen .repro script alongside (edmund-live-repro-and-diagnostics).
2. Assert the invariant where you can (rawSource == string), not just the surface symptom — that's what BypassedEditSyncTests / MarkedTextDesyncTests do.
3. Name the test for the behavior/bug, put edit-pipeline repros next to their siblings (the *Desync* / *Bypassed* / *CaretTests families).
4. New drawing behavior additionally needs a screencapture check (§6).

5. Golden / certified inventory

• RenderingRegressionTests + RecomposeEquivalenceTests are the closest thing to golden checks — they pin styled output and incremental-vs-full equivalence.
• test-files/*.md (callout.md, decorations.md, math.md, menu.md, test.md) are the user's manual test corpus — hand-testing fodder. Do not rewrite them to fit an automated test; build your own fixtures (helpers in §3, or a scratch dir).
• misc/bug-repros/ holds field evidence (.mov screen recordings + .log traces) for open bugs — reference these when reproducing, don't delete them.

6. Visual QA

Anything that draws is verified by screencapture pixel measurement, in light AND dark mode (per misc/before-you-release.md), never by eyeballing headless layout. Method + scripts: edmund-live-repro-and-diagnostics §5. Headless layout tests (HeightStabilityTests, ScrollStabilityTests) check geometry numbers but cannot confirm the pixels are right.

7. Flakiness & CI

• A fix/flaky-math-test branch exists in history — math rendering has shown timing flakiness; if a math test flakes, check that branch's approach before inventing a new one (verify: git log --oneline --all -- '*Math*').
• CI: .github/workflows/ci.yml on macos-14, latest-stable Xcode, SPM cache keyed on Package.resolved, concurrency: cancel-in-progress (private-repo macOS minutes bill 10×). CI runs the same swift test.

8. Performance evidence

• PerfHarnessTests measures the hot paths; makeLargeMarkdown builds big fixtures. Use it (don't hand-time) for any perf claim.
• The README claim "handles ~1–2MB files" and fullLayoutMaxLength = 100_000 UTF-16 (EditorTextView.swift:80) mark the boundary between the full-layout regime (≤100k, geometry is real) and the estimate regime (>100k, viewport glitches possible). A perf/viewport claim must state which regime it was measured in.

When NOT to use this skill

• The gate/branch/commit rules → edmund-change-control.
• Running the live repro or measuring pixels → edmund-live-repro-and-diagnostics.
• The caret-integrity investigation end to end → edmund-caret-integrity-campaign.
• Why a mechanism works → textkit2-appkit-reference / edmund-architecture-contract.
• Release verification → edmund-release-and-operate.

Provenance and maintenance

Verified 2026-07-05.

grep -rh '@Test' Tests/EdmundTests/*.swift | wc -l         # ~810 cases
grep -c 'import Testing' Tests/EdmundTests/TestHelpers.swift # confirms Swift Testing, not XCTest
grep -oE 'func [a-zA-Z]+' Tests/EdmundTests/TestHelpers.swift # helper inventory (§3)
ls Tests/EdmundTests/                                        # suite map (§2)
grep -n 'fullLayoutMaxLength' Sources/EdmundCore/TextView/EditorTextView.swift

If a helper in §3 no longer greps it was renamed; update §3 and any test skeletons. Re-derive the suite map from ls if files are added/removed.