TextKit 2 / AppKit reference (as it applies to Edmund)

The background a mid-level engineer or Sonnet-class model usually lacks. Each concept: brief theory, then where it bites in Edmund with a verified file pointer. This is not a textbook — it is only the parts that matter here.

Facts checked against source 2026-07-05. Items labeled (background) are general AppKit/TextKit behavior grounded in Apple's documentation, not directly grep-able in this repo.

1. The TextKit 2 object model (background + repo)

TextKit 2 replaced the TextKit 1 NSLayoutManager stack. The players:

• NSTextContentStorage — owns the backing string + attributes (the model).
• NSTextLayoutManager — lays text out (the TK2 analogue of the old layout manager).
• NSTextLayoutFragment — one laid-out chunk (≈ a paragraph); has a real geometric frame only once laid out.
• NSTextElement / NSTextParagraph — the model-side elements fragments render.

Viewport-based layout is the headline difference: TK2 lays out only the content near the visible viewport, not the whole document. That is what makes big documents fast — and it is the root of most viewport pain (§2).

Where it bites: Edmund subclasses the fragment as DecoratedTextLayoutFragment (DecoratedTextLayoutFragment.swift) to draw callout boxes, bars, and overlays.

2. Height ESTIMATES — the master cause of viewport glitches

A fragment that has not been laid out yet has an estimated height, not a real one; the total document height is the sum of real + estimated fragment heights. As layout reaches a fragment, its estimate is replaced by the true value and everything below shifts. Consequences: the scroller thumb jumps, and "scroll to offset Y" lands wrong because Y was computed from estimates. This is a widely documented TK2 limitation — even TextEdit shows it (background).

Where it bites / Edmund mitigations (verify names by grep; all in TextView/):

• fullLayoutMaxLength = 100_000 (EditorTextView.swift:80): documents ≤ 100k UTF-16 units are kept fully laid out (no estimate regime) by a coalesced next-run-loop settle.
• scheduleFullLayoutSettle / preservingViewportAnchor: the settle runs inside an anchor block so corrections never shift what is on screen.
• repairContentAboveOrigin (+LazyStyling.swift, logs repairing content above origin): fixes the case where an edit near the top strands the first fragment at negative y (unreachable above the scroller top).
• centerViewportOnCaret: re-measures after its first scroll and corrects the residual estimate error.

Rule: never trust an off-screen fragment's y-coordinate without laying out its span first. Deep write-up: docs/investigations/viewport-glitch-investigation.md.

3. The TextKit 1 fallback trap

An NSTextView can silently and permanently revert from TK2 to the legacy TK1 stack. Two known triggers: accessing NSTextView.layoutManager (the mere getter engages TK1), and storing NSTextBlock/NSTextTable attributes. Once reverted, TK2 APIs still exist but do nothing useful, and the whole editor misbehaves subtly.

Where it bites: Edmund ships a DEBUG tripwire — an observer on NSTextView.willSwitchToNSLayoutManagerNotification that asserts if the switch happens (EditorTextView.swift:273+, message "TextKit 1 fallback triggered"). Never add code that reads layoutManager or stores table attributes; draw tables as decorations instead (EditorTextView+TableRendering.swift).

4. Attribute-only mutation semantics

Edmund renders by writing attributes onto the storage, never by inserting or deleting characters (the storage == rawSource invariant). Two consequences from the text system:

• setAttributes does not re-measure geometry. After a restyle that changes a block's height or indent, you must call invalidateLayout(for:) on its range or the fragment keeps a stale frame (empty bands / clipped lines). recomposeDirty and the idle drain already do this; new paths must too.
• NSTextAttachment is only honored on the U+FFFC object-replacement character (background). rawSource never contains U+FFFC, so attachments can't be used — Edmund draws images/icons as overlays (§5) instead.

5. The custom drawing model (fragments, decorations, overlays)

DecoratedTextLayoutFragment draws two attribute families behind/over text:

• .blockDecoration (paragraph-level): callout boxes, quote bars, table borders, thematic-break rules, code backgrounds. Fragments tile vertically, so a multi-line run renders as one continuous box. A box's bottomPad grows the last fragment's frame (TK2 omits trailing paragraph spacing from the fragment, so padding done otherwise would be dead space).
• .fragmentOverlay (character-level): an image or a stroked vector path drawn at a glyph's laid-out position — rendered math, list bullets/checkboxes, callout header icon, the custom-title callout icon (path). The anchor glyph is hidden (≈0.01 pt font + clear color) and .kern reserves the drawing's advance width.

The image-on-wrapping-fragment wedge: drawing an image overlay on a multi-line (wrapping) fragment re-triggers a layout pass that collapses the fragment to one line. Drawing a shape/path does not. So the wrapping callout title's icon is a stroked CGPath (parsed by SVGPath from vendored Lucide geometry), never an image. Full saga: docs/investigations/archives/callout-title-wrap-investigation.md. This constraint holds for any new overlay that could share a line with wrapping text.

Hiding text = hiddenFont (≈0.01 pt) + clear foregroundColor. This is how delimiters (**, `, [!note]) vanish without changing the string.

Repainting one fragment without relayout (animated GIFs): setNeedsDisplay on the text view does not repaint TextKit 2 text. Each fragment paints in a private _NSTextViewportElementView under _NSTextContentView (see the viewtree repro command); mark every view under the fragment's rect (EditorTextView+GIFAnimation redisplay).

The trailing empty line is an extra line fragment, and its box is provisional. After a final \n there is no paragraph for the empty last line: it is the last textLineFragment of the previous fragment (characterRange.length == 0). Until the caret goes there its typographicBounds can overlap the line above (after an empty paragraph: measured y = 16, height 14 inside a 25 pt line). Clamp to the previous line's bottom before trusting it (enumerateVisibleLineNumbers).

6. The queued selection fixup (the round-6 delete-drift mechanism)

When you mutate an NSTextView's storage, AppKit queues a private step, -[NSTextLayoutManager _fixSelectionAfterChangeInCharacterRange:], that repairs the selection against the new character coordinates. Normally it fires promptly. But if an edit bypasses the normal close-out (see §7), the fixup stays queued and fires at the next endEditing — even an attribute-only restyle — where it maps the now-stale selection against post-edit coordinates and leaps the caret blocks away. It will move even a freshly set, valid caret.

Where it bites: this is delete-drift round 6. The heal must set the caret (from the pendingEdit hull) before the sync and re-assert it after (EditorTextView+EditFlow.swift). Recognize a variant by: a suspicious selection change arriving mid-recompose (up=Y in traces); traceSelectionOrigin will log the call stack of whoever moved it.

Critical for testing: a headless test harness runs this deferred fixup synchronously, so this bug class cannot reproduce in a unit test — the round-6 regression test passes with and without the fix. Only the live in-process repro discriminates (edmund-live-repro-and-diagnostics).

7. The AppKit edit pipeline contract (and where AppKit breaks it)

Normal edit: shouldChangeText(in:replacementString:) → the view calls replaceCharacters → didChangeText(). Edmund's didChangeText syncs rawSource from storage and restyles the edited block(s).

AppKit does NOT always send didChangeText. A drag-move of selected text whose drop lands on no valid target (e.g. released past the end of the document) deletes the dragged range via shouldChangeText → replaceCharacters and never calls didChangeText — silently freezing rawSource/blocks, after which every edit drifts and autosave writes stale content (delete-drift round 4). Edmund heals this: shouldChangeText schedules a next-run-loop bypass check (scheduleBypassedEditSyncCheck, +EditFlow.swift); an unconsumed storage pendingEdit by then means the close-out never came, and the editor runs the sync itself (breadcrumb: healing storage edit that bypassed didChangeText).

Never build a sync path on the assumption that didChangeText follows every edit.

The authentic key route (background + repo): keyDown → interpretKeyEvents → insertText: / deleteBackward:. This is why the repro driver synthesizes real NSEvents and pushes them through window.sendEvent(_:) rather than calling insertText directly — shortcuts skip deleteBackward's selection machinery, which is exactly where round 6 lived.

Spelling marks are rendering attributes, and only some paths set them. AppKit's continuous check re-runs on didChangeText, never on an attribute-only restyle, so marks go stale around a restyled word. checkText(in:types:options:) delivers its results on a later run-loop pass, and super.handleTextCheckingResults called directly set no marks (measured, headless and in a window). What works: NSSpellChecker.shared.check

• setSpellingState(_:range:), read back through textLayoutManager.enumerateRenderingAttributes (.spellingState). See EditorTextView+SpellCheck.swift.

8. IME / marked text lifecycle

While an input method is composing (e.g. CJK, accents), the view holds provisional "marked" text in storage; hasMarkedText() is true. During this window storage == rawSource is transiently false, and didChangeText defers syncing until the composition commits.

The cascade: any styling path that runs beginEditing/setAttributes/invalidateLayout mid-composition can strand the marked text in the input context. After that, didChangeText keeps bailing on its own guard and the invariant stays broken — so every later edit drifts the caret (the original delete-drift bug). Therefore every storage-touching styling path must guard !hasMarkedText() — including async ones scheduled before composition began (the caret-move restyle in +SelectionTracking). becomeFirstResponder resyncs from storage as a catch-all. Full write-up: docs/investigations/delete-drift-investigation.md.

9. Responder chain & nil-target actions (background + repo)

The responder chain is AppKit's search order for who handles an action. Menu items and toolbar buttons with a nil target send their action up the chain until something responds. Edmund's Format menu (FormatMenu.swift) is a declarative command table whose items use nil targets and route to the focused EditorTextView's @objc format… actions — the same wiring as undo/redo. The first responder is normally the focused EditorTextView.

10. NSWindow.sendEvent — the pre-toolbar event funnel

Every event a window receives passes through sendEvent(_:) before the toolbar acts. This matters because with NSToolbar.allowsUserCustomization = true, the toolbar claims any secondary (right/control) click over the toolbar — including a custom item view — for its own "Customize Toolbar…" menu, downstream of view-level handlers (menu, rightMouseDown, gesture recognizers all lose). Edmund once intercepted there (a DocumentWindow override) for right-click menus on the view-mode and Link buttons; both menus and the override were removed on 2026-09-23 — toolbar buttons that need a menu use a visible NSMenuToolbarItem pull-down instead.

11. Drag sessions (background + repo)

• Text drag-move arming: AppKit only starts a text drag after a mouse-down hold (~400 ms); a CGEvent driver must hold before moving or the drag never arms.
• Drag-select autoscroll: dragging past the viewport edge autoscrolls.
• Reveal at nearest end: a selection taller than the viewport must be revealed at its nearest end (Edmund's scrollRangeToVisible override) — always revealing the top fought the drag-select autoscroll and oscillated the viewport mid-drag.

12. swift-markdown walker model (brief)

Edmund parses with apple/swift-markdown (CommonMark/GFM) and walks the resulting Document with two back-ends: a SpanCollector-style walk that produces editor attributes, and HTMLRenderer that produces HTML for Read mode. One parser, two outputs — so Edit and Read can't drift. Custom (non-CommonMark) syntax — callouts, ==highlight==, wikilinks, comments, footnotes, math — is handled by SyntaxHighlighter+CustomParsers.swift.

When NOT to use this skill

• The project's rules/invariants (what you must not do) → edmund-architecture-contract.
• Which symptom means which mechanism → edmund-debugging-playbook.
• Reproducing a live bug / reading traces → edmund-live-repro-and-diagnostics.
• The history of how these mechanisms were discovered → edmund-failure-archaeology.
• Running the caret-integrity campaign → edmund-caret-integrity-campaign.

Provenance and maintenance

Verified 2026-07-05. Re-verify the load-bearing identifiers:

grep -n 'fullLayoutMaxLength' Sources/EdmundCore/TextView/EditorTextView.swift
grep -n 'willSwitchToNSLayoutManager' Sources/EdmundCore/TextView/EditorTextView.swift
grep -n 'scheduleBypassedEditSyncCheck\|healing storage edit' Sources/EdmundCore/TextView/EditorTextView+EditFlow.swift
grep -rn 'repairContentAboveOrigin\|preservingViewportAnchor' Sources/EdmundCore/TextView/
grep -rn 'blockDecoration\|fragmentOverlay\|hiddenFont' Sources/EdmundCore/
grep -rn 'hasMarkedText' Sources/EdmundCore/TextView/

Items marked (background) are AppKit/TextKit behavioral facts documented by Apple and in docs/*-investigation.md, not directly observable by grep. If any Edmund mitigation name above no longer greps, it was renamed — update this file and docs/ARCHITECTURE.md §5/§8 together.