Serial Studio — data hotpath

You are touching the highest-risk code in the repo. Read the target file in full first (doc/claude/architecture/dataflow.md has the full data-flow and threading model). These rules are non-negotiable; violating them causes silent frame drops, not compile errors.

Verbalize before the first edit (J-space discipline 1+3, doc/claude/j-space.md): state in chat, in your own words, the 3-5 hard rules below that this specific change is exposed to — e.g. "this adds a hop after onFrameReady, so it must be DirectConnection and allocation-free". Hotpath edits look like familiar Qt code, which is exactly the automatic mode where these rules get violated silently; naming the binding rules at the point of action is the deliberate-mode interrupt. Do not proceed straight from pattern-match to Edit.

Data flow

Driver → FrameReader::processData (pipeline thread) → PipelineHost::routeFrames → FrameBuilder parse → BlockStager::stage (pooled DataBlock) → BlockPublisher::publish → PipelineHost block ring → Dashboard::onDisplayTick (GUI) | CSV / MDF4 / API / Sessions (ONE clone_block_trimmed copy, only when a sink is on)

Dense typed sources (audio) skip all of that: driver SampleBlock → per-source IO::StreamWorker thread → bounded display update / export block / latest values, all per block. Never per-sample across a thread boundary.

Hard rules

• The frame pipeline lives on IO::PipelineHost's thread, not the GUI thread (spec 0051). GUI→pipeline calls self-marshal via PipelineHost::runOnObjectThread (event-loop backed); pipeline→GUI reads via runOnGuiThreadBlocking. A plain BlockingQueuedConnection from the GUI into the pipeline deadlocks against a script's apiCall — never add one.
• FrameReader and CircularBuffer are pipeline-thread / SPSC. Never add a mutex. Reconfigure by recreating via resetFrameReader() / reconfigure(), never by locking.
• Hotpath signal hops must be Qt::DirectConnection. A queued connection between two pipeline-thread objects fills the slot queue at 10+ kHz and drops frames. GUI↔pipeline traffic must stay chunk/command/tick rate.
• No allocation and no block copy on the dashboard path. Draw blocks from the pooled slots (DataModel::BlockStager on the frame lane, StreamProcessor::claimBlockSlot() on the stream lane) — never make_shared<DataModel::DataBlock> directly. The one detached clone_block_trimmed copy in the async-sink fan-out is intentional (slow export path, gated on a sink being on, keeps a backlog from pinning the pool) — not a violation.
• The hotpath reads cached flags (m_operationMode, m_playerOpen, m_anyAsyncSink, m_captureLatestFrame, m_changeDriven, Dashboard m_streamAvailable). A new input to any of them must wire its change signal to the cache refresh, or frames/exports silently stop (mechanics in doc/claude/architecture/dataflow.md "Cached Hotpath Flags"; see also doc/claude/common-mistakes.md). streamAvailable() also reads the spec-0040 mirror flag — a plain module-static bool via API::MirrorSession::mirroring(), never a construction (doc/claude/architecture/mirror.md).
• Native + PlainText parses through the span fast lane (trySpanLane → parseUtf8Spans → applyDatasetValuesSpans): byte views + in-place QString writes (assign_utf8_in_place / assign_string_in_place, never implicit-share assignment — a share-assign re-links buffers and degrades back to per-frame mallocs), zero steady-state allocation. Keep anything you add to that lane allocation-free.
• Every dashboard publish site stamps structureGeneration = m_framePoolGeneration — pool slot and heap fallback alike. The dashboard skips per-frame compare_frames() revalidation when the cached per-source generation matches; a frame left at the default 0 (or stale) makes Dashboard::hotpathRxFrame reconfigure every frame or never reconfigure after a real layout change. The generation only advances via invalidateFramePool().
• m_captureLatestFrame (control script running or API server on) gates the latest-frame capture behind io.getLatestFrame: one retained CapturedDataPtr per source (the pool probe skips pinned slots) plus the channel tokens. Keep it gated and allocation-free.
• Source owns time. Stamp at the driver boundary; never re-stamp in export/report workers (monotonicFrameNs(...) is the safety net only).
• Optimization macros come from core/Core/HotpathOptimization.h (SS_FORCE_INLINE, SS_FLATTEN, SS_HOT/SS_COLD, SS_RESTRICT, SS_ASSUME, SS_NO_UNROLL, ...). Annotate the .h declaration and .cpp definition in lockstep. Never add a fast-math / no-unwind / GCC optimize("...") macro (breaks the IEEE-stable math + Lua-unwind invariants). SS_ASSUME must restate a guard that already ran, never a precondition on a parsed frame.
• Fixed loop bounds + assertion density ≥ 2 per function (NASA Power of Ten). The frame extractors cap iterations at kMaxFramesPerCall; keep any new loop bounded the same way.
• Asserts on the per-frame/per-cell kernels are SS_ASSERT_HOTPATH(cond) (debug-only, compiles out of release): even the evaluated pass-path branch of SS_ASSERT is measurable at rate (the 2026-07 sweep cost ~5% throughput). Admissible only where the condition restates a guard that provably already ran — a condition derived from device bytes keeps SS_ASSERT and its release recovery. The hotpath-assert-scope lint pins the macro to the hotpath TUs.

Block publication, time rings & kernels — the newest invariants

• One publication payload, one ingestion path (spec 0055). Nothing publishes a TimestampedFrame any more: FrameBuilder stages parsed rows into a pooled DataModel::DataBlock and flushes on the display tick or a sample cap. Two caps, not one -- kFrameBlockSampleCap (64) because frame-lane columns carry a display string per sample, kStreamBlockSampleCap (4096) for the numeric-only dense lane. Dense sources (Audio, or streamLane on) do all per-sample work on their own IO::StreamWorker thread but emit blockReady queued to the pipeline thread, the SINGLE producer for every sink. Sinks share ONE clone_block_trimmed copy. Structure travels separately as a StructureSnapshot on pool-generation bumps. Never add a rate cap or a per-view reduction. Detail: doc/claude/architecture/dataflow.md.
• Time rings are sized from a rate, never a sample count alone (DSP::RingCapacity, built only by fromRate; kMaxSamples ceiling) and never for an assumed rate (spec 0097): TimeRingSizing::initialCapacity starts stream rings at the real rate and frame-lane rings small, then EnvelopeRing::growFor grows on coverage once per block (never per sample, never shrinking, under the 512 MB rail). Display storage is never pinned. The plot clocks and their display time are ONE UI::PlotClockState, cleared, saved and restored together by construction (Dashboard::resetPlotClocks()). appendDecimated clamps sub-cell backward jitter forward; a jump back over a whole cell drops the retained span (clamping it wedges the ring shut). Detail: doc/claude/architecture/dashboard.md.
• Kernels and macros: portable SIMD lives in core/Core/DSPSimd.h (spec 0021, per-lane bit-exact vs scalar) — reuse it, never inline intrinsics at a call site. Optimizer macros live in core/Core/HotpathOptimization.h; annotate .h and .cpp in lockstep, never add a fast-math / no-unwind / GCC optimize("...") macro, and use SS_ASSERT_HOTPATH (not SS_ASSERT) on per-frame/per-cell kernels. Detail: doc/claude/architecture/kernels.md.

Measure, don't guess

The documented "256 kHz data rate" is a CI gate, not a slogan. To check throughput after a change, build the app and run the in-process end-to-end benchmark:

serial-studio-pro --headless --benchmark-hotpath --min-fps 256000

It loads a project via ProjectModel::loadFromJsonDocument and drives the real pipeline — FrameReader extraction → FrameBuilder → frame parser → per-dataset transforms. The exit code (the release gate) fails if any gated tier misses.

Nine gated runs, all tiered off --min-fps (so a --min-fps 1 PGO training run stays effectively ungated). The seven parser gates disable the parse-budget guard (the spec-0051 fair-share governor, which a 100%-duty benchmark would engage) and run no exporters/dashboard, so they measure pure parse capacity; the two Lua reference floors run with consumers on and exist to catch a consumer-path collapse, not to measure parsing:

Mechanics and readouts:

• Throughput = FrameBuilder::parsedFrameCount() / elapsed. The synthetic chunk — string columns included — is built once before the timed loop, so chunk/string construction never sits in the measurement.
• A Native stage breakdown prints as hotpath-stage[native] (extract / tokenize / datasets+publish). datasets+publish is ~70-80% of per-frame time — gate any change there with this benchmark.
• Three Lua reference rows follow: lua+exporters (CSV/MDF4/Sessions/API/gRPC, mixed workload; prints hotpath: exporters cost N.NNx throughput), lua+dashboard (loads an all-widget-types project, flips HotpathBenchmark::active() so Dashboard::streamAvailable() accepts headless frames, arms every plot/FFT/multiplot/waterfall/GPS/3D widget; prints hotpath: dashboard costs N.NNx), and lua+dashboard(off) (same project, dashboard ingest off; prints the ingest on-vs-off cost). Exporter/dashboard workers can't keep up with a flat-out producer, so the pool exhausts into heap fallback — that penalty is the readout. The first two carry the 0.5x floor gates; lua+dashboard(off) stays ungated.
• An ungated engine × {numeric, mixed} × {exporters, dashboard} coverage matrix runs last so CI and PGO training exercise every consumer/engine combination.
• --benchmark-frames N sets the minimum workload; --benchmark-seconds N the minimum wall-clock window (default 10) — each run lasts until both floors are met. --benchmark-output FILE mirrors the report to a file (default: stdout only).
• --benchmark-channels N (default 8) widens the synthetic project for the parser tiers and the exporter floor; throughput tiers are printed but not gated at a non-default width. Above FrameBuilder::kMaxSpanFields (128) the Native rows take the list lane and the report says so (native lane: list). Every row prints Alloc/frame, counted only in a -DSS_ALLOC_STATS=ON build (n/a otherwise); the Native rows must read 0 there.

Source: app/src/Benchmark/HotpathBenchmark.cpp. CI (ci.yml, the only workflow) runs it on every push/PR as a hard gate on the PGO-optimized binary — the same binary that ships (PGO GENERATE → --min-fps 1 training run → PGO USE → gated --min-fps 256000). The same engine backs the in-app About → Benchmark dialog (Benchmark::BenchmarkRunner, exposed as Cpp_Benchmark_Runner). Do not regress the parse hotpath.

After any change here, re-read the diff against these rules before handing off, and run python scripts/code-verify.py --check <files> (hotpath violations are blockers, not advisories).