activate-tenant-table
GitHub指导将数据库表从多租户v1隔离迁移至v2,涵盖HTTP读写路径及后台作业转换,强调TDD测试与事务作用域规范。
Trigger Scenarios
Install
npx skills add OpenAEV-Platform/openaev --skill activate-tenant-table -g -y
SKILL.md
Frontmatter
{
"name": "activate-tenant-table",
"description": "Activates one table on multi-tenancy v2 (statement inspector + can_access_tenant), test-first, following the import_mappers pilot (PR #6255). Use when asked to switch a table from v1 @Filter isolation to v2. Covers HTTP paths and, since the background transaction primitive (#6398), background writers (scheduler jobs, consumers) once they are converted to the primitive. Covers eligibility gates, code-path inventory, TDD isolation tests, write attribution, the background conversion, the one-commit go-live, and the full regression pass."
}
Activate a Table on Multi-Tenancy v2
Switch one table from v1 isolation (Hibernate @Filter on the entity) to v2
(SQL rewriting by TenantStatementInspector + the can_access_tenant function).
The reference implementation is the pilot, import_mappers, in PR #6255.
Every template in this skill points to a real pilot file. Open it and copy the
pattern; do not invent new mechanisms.
For the full mechanism (read path, write path, exact names), read the pilot
PR #6255 description once before starting, plus the javadoc of
TenantStatementInspector, TenantWriteScopeResolver and
TenantScopeTransactionAspect.
If the table has a background writer (a scheduler job, queue consumer or
startup task that writes it), read the background transaction primitive too:
TenantScopedTransaction (openaev-model/src/main/java/io/openaev/context/TenantScopedTransaction.java).
The HTTP path carries its scope through @Transactional + TxCtx + the aspect;
the background path must NOT use @Transactional (its self-invocation trap
silently skips both the transaction and the scope) and opens transactions through
the primitive instead. Phase 5b (this runbook's background-writer conversion
phase, defined below between Phase 5 and the go-live) converts those writers;
it is the prerequisite for activating any table a background job writes.
Inputs
- The table name (e.g.
mitigations) and its API class (e.g.MitigationApi). - The activation issue (e.g. #6400). Its Context block names the mapped table(s) and the current state.
Hard rules
These are not style preferences. Each one prevents a security or availability incident. Do not trade them away to make a test pass.
-
TDD, strictly. Write the isolation test first and run it. It must fail for the expected reason before you write any production code. Never weaken, delete or
@Disabledan existing test to get green, with one exception introduced in Phase 2 and resolved in Phase 6 (the documented go-live guard).Red for the intended reason is not enough. A test that goes red when you remove your fix may only be pinning that fix, not the behaviour it is meant to protect. Before calling it done, break the behaviour a SECOND way, on a different line or in a different layer, and confirm it fails again.
Lot C's gauge tests are the example. They were red without their fix and green with it, then stayed green when the gauge was re-wired to the unscoped repository: they called the scoped method directly instead of the supplier the gauge actually registers.
-
Background writers go through the primitive, never
@Transactional. A scheduler job, queue consumer, connector-side path or startup task that writes the table is NOT an automatic stop anymore (it was before #6398). It becomes activable ONCE every such writer is converted toTenantScopedTransaction(Phase 5b), carrying a real per-tenant scope. Until that conversion is done and tested, an unconverted background writer is still a hard blocker: activating the table under it would make the writer read and write zero rows, silently. ONE documented waiver exists: a writer that only INSERTs fresh rows (VALUES inserts are not blocked by the inspector), never READS the table, and attributestenant_idcorrectly (listener +TenantContext, or explicit) may ship unconverted IF a test pins that write shape under activation AND the conversion is a tracked follow-up. The tenant-provisioning datapack writingcweswas the original example; it has since been converted (it now writes under the primitive scope MigrationProcessor sets), so no active waiver relies on this today. Use it only for a genuinely INSERT-only, never-reads writer you cannot convert in the same PR. A background READER, or any read-then-write path, gets no waiver. Background code must never use@Transactionalfor the write (self-invocation trap) and must never open raw transactions; both are guarded by the ArchUnit rules inopenaev-api/src/test/java/io/openaev/architecture/TenantBackgroundTransactionRules.java. -
Strict tables only. If
tenant_idis nullable (dual-scope:roles,groups,parameters, ...), STOP and report. Platform-row writes are an open policy question (Q7); this skill does not cover them. -
One-commit go-live. Removing the v1
@Filterand adding the table toopenaev.tenant.active-tableshappen in the same commit, never split. -
No hardcoded Flyway numbers. Refer to migrations by name. Never pin a version number in code comments, docs or tests.
-
Fail-closed is the point. If a query returns zero rows after wiring, the fix is to pass the scope correctly, never to bypass the inspector, never to add raw JDBC, never to widen the scope.
-
Full regression before done. The activation rewrites the SQL of every query touching the table. The full API suite must pass, not just your new tests.
-
Evidence over claims. Every red and every green leaves a trace: keep the raw test output (the failing assertion for the red, the passing summary for the green) and paste it into the Phase 8 report. A TDD step without its output did not happen.
-
Do not improvise on drift. If a model file referenced by this skill does not exist anymore, stop and find its successor (
git log --oneline --follow --all -- <path>). Never substitute an invented pattern for a missing reference. -
TxCtxposition is fixed. In every new or modified method signature that carriesTxCtx, it must be the FIRST parameter (annotations allowed, e.g.@RequireTenantSelector TxCtx ctx). Keep this order through service call chains too, so wiring remains grep-auditable and consistent. -
No v1 tenant context in v2/integration tests. For API v2 activation tests and integration tests, do not add
TenantContext.getCurrentTenant(),TenantContext.setCurrentTenant(...), orenableFilter("tenantFilter"). Use explicit tenant ids +TxCtx/TenantScopedTransactionhelpers. -
Repository extends
CrudRepository, notJpaRepository. The convention for strict tenant-scoped repositories in this codebase (seeTagRuleRepository,DomainRepository,NotificationRepository) isextends CrudRepository<Entity, Id>, JpaSpecificationExecutor<Entity>.JpaRepositoryadditionally exposesfindAll(Sort)(every row, every tenant, unbounded and unpaginated) plussaveAndFlush,deleteInBatch,deleteAllInBatch,getReferenceById— none of them take a tenant argument, so they are an easy accidental cross-tenant bypass. Only widen toJpaRepositoryif the repository genuinely needs one of those extras, and then scope every call site through a tenant-awareSpecification.
Baseline: controller entrypoints already carry TxCtx
Every @Transactional method under io.openaev.api/** and
io.openaev.rest/** declares a bare TxCtx ctx parameter, added in one pass
across the codebase, with five deliberate exclusions that carry
@NoTenantScope instead: UserApi.login, UserApi.passwordReset,
UserApi.changePasswordReset and UserApi.validatePasswordResetToken
(permitAll, pre-auth), and StreamApi.streamFlux (propagation = NEVER, so
there is no transaction to scope). That changes what a single-table
activation needs to do:
- Phase 1 no longer hunts exhaustively for missing
TxCtxon controller entrypoints, though it still spot-checks the ones it needs (see below). That search (the biggest source of the regressions cited throughout this skill — #6409, #6410, #7026, #7605/#7621) is done, once, for the whole codebase. Phase 1 is now scoped to what the blanket wiring does NOT cover: background paths (Phase 5b), non-@Transactionalhandlers, native/raw SQL shapes, and OSIV/lazy-serialization sinks (Phase 3b) — aTxCtxparameter on a method signature does not by itself fix a lazy association or computed getter resolved by Jackson AFTER the transaction has already closed. - A
TxCtxparameter is not inert. It resolves a scope and sets it on the transaction whether or not the table it touches is active, andTenantScopeTransactionAspectthrows when a nested@Transactionalmethod tries to redefine a scope already set in the same transaction. Adding or removing one is a behaviour change, not a signature change: assume it can break a caller, and re-run the suite. - This is a point-in-time fact, not a self-enforcing invariant, until the
default-secure compile rule (
EndpointTxScopeRule, #7726) is enabled foropenaev-api; it currently ships disabled. A NEW controller endpoint added after this baseline, or one that was not yet@Transactionalat the time, may still be missing it — spot-check the entrypoints this activation actually needs (Phase 1) rather than assuming. - Phase 5's
TX_SCOPED_ENTRYPOINTSregistration is now mostly documentation and drift-detection, not discovery. Most entries you would have added by hand already exist; add a new one only for a genuinely new gap (a Phase 3b force-initialize fix, a handler that just became@Transactional, or a converted background path).
Procedure
Phase 0 — Eligibility gate (stop conditions)
Replace {table}, {Entity}, {EntityRepository}, {Api} below with your
target (e.g. mitigations, Mitigation, MitigationRepository,
MitigationApi).
# 0.1 The entity must be strict tenant-scoped: implements TenantBase, not DualScopeBase
grep -n "TenantBase\|DualScopeBase" openaev-model/src/main/java/io/openaev/database/model/{Entity}.java
# 0.2 The tenant column must be non-nullable. The entity mapping is the
# reliable static check; tenant_id was added to most tables by bulk
# migrations that loop over a table list, so grepping migrations for
# "{table}" + "tenant_id" on one line finds nothing.
grep -n -A2 'name = "tenant_id"' openaev-model/src/main/java/io/openaev/database/model/{Entity}.java
# expect: @JoinColumn(name = "tenant_id", ..., nullable = false)
# authoritative check when a DB is running:
# SELECT is_nullable FROM information_schema.columns
# WHERE table_name = '{table}' AND column_name = 'tenant_id';
# 0.3 Unique constraints must be tenant-aware. A global unique index on a
# business key (external id, name, key) means two tenants cannot hold the
# same value: activation would turn that into cross-tenant interference.
grep -rn -i "unique" openaev-api/src/main/java/io/openaev/migration/ | grep -i "{table}"
# the grep misses multi-line definitions; on a live DB, `\d {table}` in psql
# lists every index and constraint and is authoritative
0.4 Hot-path check. The inspector wraps every active table in a filtered
sub-query, which can change query plans. If the table sits in frequent joins
or heavy list endpoints, look at the rewritten SQL before go-live: captured
real SQL can be replayed through the inspector with
openaev-api/src/test/java/io/openaev/config/TenantSqlReplayMeasurementTest.java
(gated by -Dtenant.sql.replay.file), and the heaviest query deserves an
EXPLAIN with the rewrite applied. For a small config table this is a
one-line note in the report; for a hot table it is a real measurement.
There is no reliable one-line grep for "no background writer" (a keyword filter on scheduler/job/consumer misses renamed packages and matches string literals). The authoritative classification is the Phase 1 inventory: read every hit.
STOP conditions, report instead of continuing:
- entity implements
DualScopeBase, or the tenant column is nullable → dual-scope, out of scope (hard rule 3) - Phase 1 finds any non-HTTP path that WRITES the table → NOT an automatic stop
since #6398, but it moves the table into the background-writer track: every
such writer must be converted to the primitive in Phase 5b before go-live. If
the conversion is out of scope for this run (e.g. the writer is a large
execution-surface job you are not converting now), STOP and report it as a
blocker; the table cannot be activated while an unconverted writer touches it.
A background READ-only hit (e.g. a telemetry counter) is not a blocker but
must be listed in the report as a documented degradation: once the table is
active it reads zero rows unless that reader also carries a scope.
ProductInventoryMetricCollector(below) is the single, always-present instance of this shape — check it on every activation, not only when Phase 1 happens to surface it. - 0.3 finds a unique index on a business key that does not include
tenant_id→ the schema needs a prep migration first (model: the existing__Update_unique_constraints_for_tenantsmigration inopenaev-api/src/main/java/io/openaev/migration/; same pattern, new migration).CREATE UNIQUE INDEX mitigations_unique ON mitigations (mitigation_external_id), global, so two tenants cannot both hold MITRE mitigation M1013. Fix the constraint (addtenant_idto it) in its own reviewed change BEFORE the activation, and only if per-tenant duplication is the intended semantics; if the rows are meant to be platform-shared reference data, the table may not be a good activation candidate at all. Report and let a human decide.
When a stop condition is hit, still run Phase 1 (the inventory is what makes the stop useful), then produce a stop report instead of code and post it on the table's activation issue. Format:
## activate-tenant-table skill run: STOPPED at eligibility (gate <n>)
**Gates**
- 0.1 PASS/STOP: <entity check result>
- 0.2 PASS/STOP: <tenant column nullability>
- 0.3 PASS/STOP: <unique constraints; quote the offending index if any>
- 0.4: <hot-path note>
**Blocker to decide before activation**
<the failed gate, the options this skill lists for it, and what each option
needs (e.g. prep migration modeled on V4_82 vs shared-reference-data
discussion)>
**Inventory (phase 1)**
<repository users and their classification; child tables; other APIs>
**Side findings**
<anything found on the way that someone should look at, one line of impact each>
The evidence rule (hard rule 8) applies here too: quote the actual grep output or file lines behind each gate verdict, do not paraphrase them.
Phase 1 — Inventory what the mass TxCtx wiring does NOT already cover
Quick hygiene checks on changed files before deeper inventory:
# Any signature carrying TxCtx with non-first position must be fixed
grep -rn "(.*,[[:space:]]*[@A-Za-z0-9_ ]*TxCtx[[:space:]]+[a-zA-Z_][a-zA-Z0-9_]*" \
openaev-api/src/main/java openaev-model/src/main/java --include="*.java"
# In API v2 / integration tests, block v1 tenant-context/filter idioms
grep -rn "TenantContext.getCurrentTenant\|TenantContext.setCurrentTenant\|enableFilter(\"tenantFilter\")" \
openaev-api/src/test/java --include="*.java"
Because of the baseline above, Phase 1 no longer hunts for missing TxCtx on
controller entrypoints. It is still fully required for everything the
blanket wiring cannot fix by construction:
- Background paths (scheduler jobs, queue consumers, startup runners,
@Asynctasks) — untouched by the mass wiring, since none of it is a@RestController. Every writer found here is a Phase 5b conversion; every reader is a documented degradation (Phase 0) or gets an explicit scope. - Non-
@Transactionalhandlers. The aspect only fires on@Transactionalmethods, so a handler that wasn't@Transactionalat baseline time was correctly left untouched. If its call graph reaches{table}, make it@Transactionaland addTxCtxnow (Phase 2). - OSIV / lazy-serialization sinks (Phase 3b). A
TxCtxparameter on an entrypoint's signature does not fix a lazy association or computed getter resolved by Jackson AFTER the transaction (and its GUC) has already ended. Phase 3b is unaffected by the baseline and remains mandatory. - Native/raw SQL shapes that
JOIN {table}— orthogonal toTxCtxentirely, since the inspector inspects SQL text, not method signatures. - Drift since the baseline. A controller method added, or made
@Transactional, after the mass-wiring PR may still be missingTxCtx. Spot-check the entrypoints this activation actually needs rather than assuming full coverage. - Query shapes that stop being valid SQL once the table is wrapped. The
inspector rewrites
FROM {table} tinto a derived table. PostgreSQL's functional-dependency rule — selecting ungrouped columns is legal when theGROUP BYcovers the table's primary key — applies to BASE TABLES only, so anyGROUP BYrelying on it becomes invalid SQL. See the GROUP BY section below; this one is not aTxCtxproblem at all and no amount of wiring fixes it.
grep -rln "{EntityRepository}" openaev-api/src/main/java openaev-model/src/main/java
grep -rn "{table}" openaev-api/src/main/java --include="*.java" | grep -v "^Binary"
The table-name grep matches string literals too (e.g. "apply mitigations" inside seeded CVE descriptions). Read each hit before classifying it; a textual match is not a code path.
Classify every hit:
- the table's own API and service → verified in Phase 2 (should already carry
TxCtx; wire it only if genuinely missing) - another API or service that reads the table → verify it already carries
TxCtx(Phase 5 is now mostly a confirmation, not new wiring) - background reader → documented degradation (Phase 0), or give it a scope too
(wrap its read in
tenantTx.execute(scope, …)) if it must keep seeing rows - background writer → convert to the primitive in Phase 5b. If you are not converting it in this run, it is a blocker: stop and report (Phase 0)
Removing the v1 @Filter silently disarms every isolation test that does not activate the table
The test profile declares no openaev.tenant.active-tables, so the inspector never fires in a test
context unless that context sets it. Every isolation test therefore needs
@TestPropertySource(properties = "openaev.tenant.active-tables={table}"), which the RED phase
above already tells you to write.
The trap is the other direction, and it is about tests you did NOT write. A suite that asserted this
table's isolation before the activation was relying on the v1 @Filter. Removing that filter
takes its isolation away, and because the inspector is not active in its context either, the
assertions keep running against nothing. They do not fail loudly: a cross-tenant read simply starts
returning the other tenant's rows, and only an assertion precise enough to notice will catch it.
That is what happened on the assets activation (#6438). EndpointApiTest's TenantIsolation
nested class had four cross-tenant tests and no @TestPropertySource. Removing Asset's @Filter
turned one of them red (given_endpointInTenantX_should_notAppearInTenantYSearch returned tenant X's
endpoint to a search under tenant Y) while the other three stayed green without isolating
anything. Attribution was correct throughout; only the read was unprotected.
Find them before go-live:
# test classes that assert something about this table's isolation, and whether they activate it
grep -rln "{Entity}\|{table}" openaev-api/src/test --include="*.java" | xargs grep -ln "Tenant\|tenant" | xargs grep -Ln "TestPropertySource"
Read every hit. A class that asserts cross-tenant behaviour and does not set active-tables is
either proving nothing or about to break.
Put the annotation on the outermost test class, not on a @Nested one. A @TestPropertySource
on a nested class builds a second Spring context, and the mock user provisioned by
WithMockUserTestExecutionListener lives in the parent's TestUserHolder; the nested context gets
an empty one and every test fails on "The given id must not be null" before reaching its assertion.
Code that relied on the v1 @Filter breaks silently, and it is not only tests
The previous section is about test suites. The same removal takes isolation away from production
paths that never carried a TxCtx and were scoped by the v1 filter alone, enabled on every
@Transactional method by HibernateFilterTransactionAspect from the thread-local.
NotificationMatchingService.matches is the case to remember. It is @Transactional with no
TxCtx, reached from an @Async @TransactionalEventListener - a pool thread, after commit, no
ambient transaction - and NotificationEngineService states the dependency in its own comment:
"runs with the trigger's tenant so the Hibernate tenant filter scopes every query correctly".
Activating assets, asset_groups or findings removes that filter, so every LIVE notification
trigger with a non-empty filter on those resources counted zero rows and stopped firing. No log, no
exception: matches catches and returns false.
session.disableFilter("tenantFilter") is the same family read from the other side. It is how v1
code declares "this read is deliberately cross-tenant", and it is inert under v2: it does nothing
to app.current_tenants, and the inspector never consults the Hibernate filter.
# both directions, before go-live
rg -n 'disableFilter\("tenantFilter"\)' --type java
rg -n 'HibernateFilterTransactionAspect|tenant filter' --type java openaev-api/src/main
# then, for each hit, answer: does this path reach {table}, and what sets its scope now?
Follow the call chain to the tables it actually reaches, not the imports of the class in front of
you. QueueChainingJob was classified as safe because its own imports name only steps,
workflow_runs and step_delay_queue; four levels down, createReadySteps reaches
ScopeService.getValidAssets -> assetService.assets(ids) on the activated assets table.
The inspector rewrites UPDATE and DELETE, not only SELECT
rewriteUpdate adds can_access_tenant(...) to an UPDATE's WHERE. With no scope the statement
updates zero rows and reports success. On the findings activation this hit the test-only date
setters (endpointRepository.setCreationDate): they silently did nothing, every endpoint kept
now() as its creation date, and three date-range dashboard assertions counted all of them. The
symptom looked like over-counting across tenants, which is the wrong diagnosis entirely.
Any @Modifying query on the table needs a scope, in tests as much as in production.
A @Transactional MockMvc test keeps the scope the request resolved
The tenant aspect is @Before-only and writes set_config(..., true), which is transaction-local.
In a @Transactional test the handler joins the test transaction, so after mvc.perform returns,
the scope the request resolved is still set. Anything the test then reads through a repository
sees it.
That silently couples the expectation to the response. FindingApiTest compared its HTTP response
against findingRepository.findAll() taken in that same transaction: if the search ever failed
closed, both sides came back empty and [] == [] held. Eight assertions were affected.
Either materialise the expectation from the fixtures you seeded, read it through raw JDBC (which the
inspector never rewrites), or at minimum assert it is non-empty - that single guard is what turns
[] == [] back into a failure.
Telemetry gauge check: ProductInventoryMetricCollector
openaev-api/src/main/java/io/openaev/telemetry/metric_collectors/ProductInventoryMetricCollector.java
registers a platform-wide {table}_total gauge for most entities, evaluated by
a Supplier lambda OUTSIDE any HTTP request — no TxCtx from the mass wiring
ever reaches it, so it is always a background reader in the Phase 1 sense
above, and it is the single, recurring, always-present instance of that shape:
check it on every activation regardless of whether the earlier greps surfaced
it. Its own javadoc documents the exact failure mode: with no scope open,
TenantStatementInspector fails closed and the gauge silently reports 0, not
an error.
grep -n "{table}\|{entity}Repository" \
openaev-api/src/main/java/io/openaev/telemetry/metric_collectors/ProductInventoryMetricCollector.java
-
No hit at all → nothing to do, the table has no gauge.
-
A hit using
safeCount({entity}Repository::count)directly (the plain, un-scoped form) → this is a go-live blocker for that one gauge line, not the whole activation: it must be converted to the scoped form BEFORE go-live, following the pattern already used for three other v2-active tables in the same file,countAssetGroups()/countChannels()/countImportMappers()(model fix forchallenges, #6416):// registration: this::count{Entities} instead of {entity}Repository::count metricRegistry.registerGauge( "{table}_total", "Number of {entities}", () -> safeCount(this::count{Entities})); /** Counts {entities} across the whole platform ({table} is v2-active, #<issue>). */ long count{Entities}() { return countAcrossAllTenants({entity}Repository::count); } -
A hit already wrapped in
countAcrossAllTenants(...)→ already correct, nothing to do; note it in the Phase 9 report as verified, not skipped.
There is no test in CI that would catch a regression here on its own: the gauge only degrades silently in a real deployment (no assertion fails, no exception is thrown). Treat this grep as mandatory evidence for the Phase 9 report even when the answer is "no hit" — a claimed activation with no note on this file is unverified, not verified-empty.
GROUP BY on a wrapped table: valid SQL before activation, a 500 after
The inspector rewrites FROM {table} t into
FROM (SELECT * FROM {table} t WHERE can_access_tenant(t.tenant_id)) AS t.
PostgreSQL lets a query select ungrouped columns when the GROUP BY covers the
table's primary key, but that rule holds for base tables only. A derived
table has no primary key to infer the dependency from, so the same query stops
being valid:
-- base table: accepted
SELECT ag.asset_group_id, ag.asset_group_name FROM asset_groups ag GROUP BY 1;
-- wrapped exactly as the inspector wraps it: refused
SELECT ag.asset_group_id, ag.asset_group_name
FROM (SELECT * FROM asset_groups ag WHERE can_access_tenant(ag.tenant_id, true)) AS ag
GROUP BY 1;
ERROR: column "ag.asset_group_name" must appear in the GROUP BY clause
Hibernate's criteria layer emits exactly that shape whenever a query helper
groups on root.get("id") alone and multiselects other columns, which is the
normal way list and search endpoints are written here.
This fails in production and passes in CI. The test profile ships an empty
active-tables, so the inspector never fires and the query keeps its base-table
form. The symptom is a 500 on a search or list endpoint, after go-live.
Find every site before activating:
# every GROUP BY in code that can reach the table, then read each one:
# does it group on the id alone while multiselecting other columns?
grep -rn "groupBy(" openaev-api/src/main/java --include="*.java"
Fix by listing every non-aggregated projected column in the GROUP BY. It is
equivalent for the planner and does not depend on the FROM item being a base
table. Worked example, AssetGroupQueryHelper in the asset_groups
activation (#6435):
cq.groupBy(
List.of(
assetGroupRoot.get("id"),
assetGroupRoot.get("name"),
assetGroupRoot.get("description"),
dynamicFilterAsJsonb));
A column of a type PostgreSQL cannot group on directly (json, for instance)
needs a groupable expression on both sides: project to_jsonb(...) and group on
that same expression, not on the raw column.
Do not wait for a fix in the rewriter to skip this step (tracked in #7843). Making the
inspector keep the primary FROM item as a base table (moving its predicate into
the WHERE) would only cover columns of that primary table. Joined tables stay
wrapped, so a query grouping on a joined table's id while selecting its other
columns breaks the same way as soon as that joined table is activated in turn:
UserQueryHelper selects the organization's name while grouping only on its id.
Listing the columns is what covers both cases.
Still walk the transitive closure of callers — but now for background
paths, association/computed-getter sinks, and other non-controller code, not
for missing TxCtx on REST entrypoints. A single hop only finds direct
callers of the repository; it misses a shared utility
(InjectUtils.resolveInjector, CollectorService.getCollectorRelationsId,
...) called from several unrelated places, each a separate hop. Most of the
historical regressions this walk used to catch were exactly "a REST
controller sibling was never re-grepped once one caller looked wired" — the
injectors #7026-class gap (AtomicTestingService.createOrUpdate, a second,
never-visited caller of InjectUtils.resolveInjector two hops from
InjectService), the executors #6409 gap (ExerciseApi#changeExerciseStatus,
a third, unenumerated caller of throwIfExerciseNotLaunchable), and the
injectors #6410 gap (ThreatArsenalApi's sibling callers of
InjectorContractService.searchInjectorContracts, never re-grepped once
InjectorContractApi#injectorContracts looked done) were all controller
entrypoints missing TxCtx — the exact failure mode the baseline now
prevents by construction. The risk that remains is the OTHER shape: a shared
symbol whose new caller is NOT a @Transactional controller method, so the
baseline never touched it:
collectors(#7026):SecurityPlatform#collectors, a lazy association serialized by a custom serializer AFTER the transaction returned — no amount ofTxCtxon the controller method fixes this; it needs a force-initialize inside the transaction (Phase 3b).- a background job, queue consumer, or startup task calling the same shared utility — invisible to the baseline entirely, and a Phase 5b conversion if it writes, a documented degradation or explicit scope if it only reads.
- a DEPRECATED controller sharing the same service as an already-wired one
(the cwes activation had to wire
CveApi, deprecated since 1.19, still deployed, alongsideVulnerabilityApi) — this one IS a REST entrypoint, so it should already carryTxCtxper the baseline; treat a miss here as baseline drift to fix directly, not as a new discovery to wire by hand.
The point of the BFS is no longer REST controllers — it's finding the
NON-controller leaves the baseline cannot reach. Use a worklist/BFS over
caller edges (an IDE's "Find Usages"/"Call Hierarchy" or a code-intelligence
tool, if available, is more reliable than chained greps; fall back to
grep -rn "\.{symbol}(" per newly found symbol otherwise) and keep expanding
each newly found caller until it resolves to one of two outcomes:
- a
@RestController@Transactionalmethod → a cheap, immediate stop: it already carriesTxCtxper the baseline, nothing to do. This is the ONLY leaf type the walk can skip without further action. - anything else — a
@RestControllermethod that is NOT@Transactional, a@Scheduled/@RabbitListener/background entrypoint, or a lazy/computed serialization sink — is real work: Phase 2 (make it@Transactional), Phase 5b (convert the background writer), or Phase 3b (force-initialize the sink) respectively.
Never stop at "a service I already expected to see" — that is exactly the
trap that hid AtomicTestingService.createOrUpdate and SecurityPlatform's
collectors association above. For every shared helper or service method
found while walking, grep every call site of that exact method name
codebase-wide, walk each one up to its enclosing method, and repeat until
every hit is a real entrypoint. Finding and fixing the first caller is a
signal that the symbol is shared, never a signal to stop.
Also list child tables (FKs pointing at {table}). A child without its own
tenant_id rides along with the parent and is NOT added to active-tables.
A child with its own tenant_id is a separate activation; report it.
Association and computed-getter accessors (entity.get{Entities}(), or a
computed @JsonProperty getter deriving a scalar from {table}) bypass the
repository grep entirely and are NOT fixed by the mass TxCtx wiring even
when their controller already carries the parameter — this is exactly the
OSIV timing problem above. Do the full scan in Phase 3b now; do not defer it
or duplicate it here.
Native query shape scan (#7007). Activating {table} does not just gate
the queries that read {table} itself — it pulls into the fail-closed
TenantStatementInspector rewrite EVERY native @Query that so much as
mentions {table} in a JOIN, however unrelated to the rest of that query's
predicates. TenantStatementInspector only accepts a closed list of
FROM/JOIN shapes; anything else is refused with TENANT_FILTERING_REFUSED,
even a shape that has nothing to do with tenant isolation. The #6751
(collectors) activation shipped this exact regression to production: adding
collectors to active-tables pulled findAgentlessExpectationsNotFilledForSource
into rewriting because it had JOIN collectors c, and its unrelated
NOT EXISTS (SELECT 1 FROM jsonb_array_elements(...) r ...) predicate — a
table-function FROM item without the LATERAL prefix — was refused fail-closed,
breaking the AI defense collector endpoint on every call (see #7007 / PR #7008).
# every native @Query that JOINs {table}, anywhere in the codebase - not just
# the table's own repository
grep -rln "JOIN {table}\|join {table}" openaev-model/src/main/java openaev-api/src/main/java --include="*.java"
For every hit, read the FULL query text (not just the JOIN {table} line)
and check every FROM/JOIN item against what TenantStatementInspector
already accepts (see TenantStatementInspectorTest). The recurring offender
is a table-function FROM item (jsonb_array_elements, jsonb_each,
unnest, ...) missing LATERAL: LATERAL is a noise word for a
function-call FROM item in PostgreSQL (identical semantics and plan) but is
exactly the marker the inspector uses to accept it — add it. If the query
uses a FROM/JOIN shape the inspector does not cover at all, that is a
blocker: stop and report (Phase 0), do not attempt to teach the inspector a
new shape inside a table-activation PR.
Also avoid CTE and alias names that match real active table names (for
example WITH tags AS (...) once tags is active). PostgreSQL can resolve
that shadowing, but the inspector's relation-name matching may treat the CTE
as the active table and fail-close the read. Prefer explicit names such as
scenario_tags_agg.
Pin the fix with a regression test in TenantStatementInspectorTest using
the REAL production SQL (read the @Query value via reflection off the
repository method, as PR #7008 does), not a hand-simplified paraphrase — the
whole point is to catch the exact shape that broke in production. Also note
for the record: openaev-api/src/test/resources/application.properties
ships an EMPTY openaev.tenant.active-tables, so IntegrationTest-based API
tests never exercise the rewriter for {table} and cannot catch this class
of regression — TenantStatementInspectorTest (constructed directly with
{table} in its active-table set) is the only test layer that does.
Test compatibility scan — now scoped to genuinely NEW wiring only. The
mass-wiring baseline already fixed every test broken by adding a TxCtx
parameter to an already-@Transactional controller method (standaloneSetup/
@WebMvcTest MockMvc tests missing a TxCtxArgumentResolver, direct Java
calls to the controller method missing the argument). This scan is needed
only for an entrypoint THIS activation newly makes @Transactional (item 2
above), and for the v1-filter case, which is unaffected by the baseline: any
test that calls session.enableFilter("tenantFilter") and then asserts a
findAll()-style read on the entity silently changes meaning once the filter
is removed at go-live and the test context keeps the allowlist empty — the
read returns EVERY tenant's rows. Found on the cwes activation:
TenantServiceTest expected 7 cwes and saw 14.
# tests relying on the v1 filter over the entity you are activating
grep -rln 'enableFilter("tenantFilter")' openaev-api/src/test/java | xargs grep -l "{EntityRepository}\|{Entity}"
Fix by asserting on explicit attribution instead
(.filteredOn(e -> tenantId.equals(e.getTenant().getId()))), never by
re-adding the filter. If item 2 above did convert a handler to
@Transactional for the first time, also register the
TxCtxArgumentResolver on any standaloneSetup test hitting that URL (or
migrate it to the full-context IntegrationTest base class) and add the
TxCtx arg to any direct Java call to that method.
Phase 2 — RED: write the HTTP isolation test first
Before writing the test, map the controller on both URIs — this is a
structural, per-table change the TxCtx baseline does not touch:
@RequestMapping({{Api}.URI, {Api}.TENANT_URI}) with
TENANT_URI = TenantUriUtils.TENANT_PREFIX + "/...". Without both mappings
the tenant-path assertions below have no route to hit.
Per the baseline, every already-@Transactional handler on {Api} should
already declare TxCtx ctx; confirm it while you're in the file rather than
assuming it (baseline drift, or a handler just made @Transactional in
Phase 1 item 2, are the two ways it can be missing). The aspect only fires on
@Transactional methods — a TxCtx parameter without the annotation is
silently ignored and the endpoint stays fail-closed, so if a handler that
touches the table isn't @Transactional, make it so and add TxCtx ctx with
the pilot's one-line comment explaining the parameter (so a reviewer doesn't
delete the "unused" argument). A handler that provably never touches the
table (works on other tables, or on transient objects never persisted) needs
neither.
Model: openaev-api/src/test/java/io/openaev/rest/mapper/ImportMapperHttpIsolationTest.java.
If the table has NO API of its own and is reached through another aggregate's
association, prove isolation through THAT aggregate's real endpoints instead;
model: openaev-api/src/test/java/io/openaev/rest/vulnerability/CweHttpIsolationTest.java
(cwes proven through the vulnerability endpoints: own-path read exposes the
row, cross-tenant read sees an empty association, ground truth by raw JDBC).
Such a test cannot be @Transactional when it must touch two tenant paths:
each request needs its own transaction (see the model's javadoc). Everything it
creates is therefore COMMITTED: clean the table rows explicitly in @AfterEach,
then remove the tenants with
TenantIsolationTestHelper#deleteCommittedTenants (null-safe, handles the one
non-cascading tenant child).
Place the new test next to the API under test
(openaev-api/src/test/java/io/openaev/rest/{domain}/).
Copy the model's structure. Key elements that must all be present:
@Transactional
@TestPropertySource(properties = "openaev.tenant.active-tables={table}")
@WithMockUser(isAdmin = true)
class {Entity}HttpIsolationTest extends IntegrationTest {
@Autowired private MockMvc mvc;
@Autowired private TenantIsolationTestHelper tenantHelper;
@BeforeEach
void seedTwoTenantsWithOneRowEach() throws Exception {
tenantA = tenantHelper.createTenantWithCurrentUser("http-iso-a").getId();
tenantB = tenantHelper.createTenantWithCurrentUser("http-iso-b").getId();
// seed one row per tenant with a native INSERT carrying an explicit tenant_id
}
}
Notes that make or break the test:
@TestPropertySourceactivates the table for this test only. The test classpath keeps the allowlist empty on purpose; never add your table to the test-wide properties.- Leave
@WithMockUserat its default (autoJoinDefaultTenantstaysfalse) on this class. The class-level mock user must resolve to exactly the tenantstenantHelper.createTenantWithCurrentUser(...)granted it — nothing more. SettingautoJoinDefaultTenant = truehere silently adds a second tenant membership, which (a) defeats the "create with no selector → 400" assertion below (the fallback selector now sees an unambiguous default tenant instead of an ambiguous multi-tenant scope) and (b) can tripTenantScopeTransactionAspect's "scope already set for this transaction" guard the moment a secondmvc.performcall in the same test method resolves a wider scope than the first. SeeWithMockUser.autoJoinDefaultTenant()javadoc for the full rationale. - Seed with a native
INSERT ... VALUESincludingtenant_id, like the pilot'sseedMapper. The inspector does not block VALUES inserts. - Ground-truth assertions (prove a row was NOT touched) use raw JDBC on the
test's own connection, like the pilot's
rawName/rawCounthelpers, with anentityManager.flush()first. - Each test method stays on ONE tenant path. Changing the scope inside the
same transaction is refused by the nesting guard in
TenantScopeTransactionAspect.
Cover, one test method each (match your API's real endpoints):
- read own row under own path → 200; other tenant's row under your path → 404
- list/search under a path, and via the
X-Tenant-Idsheader → only that tenant's rows - create under a tenant path → row stored with that
tenant_id(assert with a native query); create with no selector → 400 - update and delete cross-tenant → 404 or no-op, ground-truth read proves the row is untouched
- import/duplicate/export where the API has them
- upsert where the API has one: upserting the same business key under two
different tenant paths must yield two distinct rows, each with its own
tenant_id. This test only passes if the unique constraints are tenant-aware (Phase 0.3); a unique-violation failure here means that gate was skipped. - find-or-create by business key (Phase 4's trap): with the key existing in
BOTH tenants, a request under a MULTI-tenant scope (plain path, caller in
two tenants) must be refused with 400 — never 500 on the duplicate, never a
silent link to another tenant's row. And under one tenant's path, a second
write with the same key must REUSE that tenant's row, not duplicate it.
Models:
plainPathWithCweIsRefusedUnderMultiTenantScopeandsameTenantCweIsReusedNotDuplicatedinCweHttpIsolationTest.
Run it and check the failure reasons:
mvn -ntp -pl openaev-api test -Dtest='{Entity}HttpIsolationTest'
Expected RED: reads under a tenant path fail (no TxCtx on the handler yet,
so no scope, so zero rows) and write attribution fails (no resolver yet). If a
test fails for a different reason (compile error, fixture problem), fix that
first; the red must be the mechanism, not noise.
Save the raw output now: the failing assertions are the red evidence that goes into the Phase 8 report (hard rule 8).
The header-route list test will stay red even after wiring, while the v1
@Filter is still on the entity: v1's thread-local predicate ANDs with v2's
and returns nothing. @Disabled that one test with a comment saying exactly
that, referencing the go-live phase. This is the ONE allowed @Disabled, and
Phase 6 removes it.
GREEN. Model: openaev-api/src/main/java/io/openaev/rest/mapper/MapperApi.java.
Re-run the test class after each endpoint. Read tests go green one by one —
confirming a handler's TxCtx was already there (baseline) or adding it
(genuine gap) is the only production change needed to turn a read test green.
Do not move to writes until all reads are green.
Phase 3b — Every direct or indirect link to the table, everywhere
Phase 1 finds code that reads {table} through {EntityRepository} or a
literal string match. It does NOT reliably find another aggregate's
association pointing at {Entity} (@OneToMany, @ManyToMany,
@ManyToOne) that some UNRELATED API lazy-loads and serializes — that
association never mentions {table} or {EntityRepository} by name, so
neither Phase 1 grep sees it. This is exactly what #7026 shipped to
production for the collectors activation, months after go-live: model it.
Root cause of #7026, read it before running this phase — it is the failure
mode you are hunting for: security_platform_collectors was always
serialized as an empty array because SecurityPlatform#collectors is a lazy
association, serialized by MultiIdListSerializer AFTER the controller's
@Transactional method returned (open-in-view). By then
TenantScopeTransactionAspect's scope was gone. SecurityPlatformApi's
endpoints carried no TxCtx at all (nobody had reason to add one — that API
does not read collectors directly, it reads SecurityPlatform), so
app.current_tenants was never set, and the fail-closed
TenantStatementInspector rewrote the lazy collectors query to
can_access_tenant(...) = false for every row. The endpoint kept returning
200 with an empty list, not an error — so nothing failed loudly, and CI never
saw it either: the test profile ships an EMPTY active-tables, so the
inspector never fires (the same gap #7007 already names). The frontend then
silently mis-derived isCollectorManaged() from that empty array and unlocked
Update/Delete on a platform whose collector was still running.
This phase runs at go-live AND belongs in the permanent Definition of Done for
every activation, not just once: a NEW association to {Entity} can be added
by an unrelated PR at any time after {table} is already active, and nothing
today stops it from shipping unscoped. Re-run this phase's greps whenever a
new @OneToMany/@ManyToMany/@ManyToOne targeting {Entity} appears
in review (the {table} entry in TenantActiveTableAccessArchTest from
Phase 6 is what makes a future miss fail the build instead of shipping silently).
3b.1 — find every association pointing at the entity, anywhere, not just its own aggregate:
# every field typed as {Entity} or a collection of it, in ANY entity
grep -rn "{Entity}>\|{Entity} " openaev-model/src/main/java/io/openaev/database/model --include="*.java" | grep -i "@OneToMany\|@ManyToMany\|@ManyToOne\|@OneToOne" -A1 -B1
# faster two-step: list the annotation lines, then check the next line's type
grep -rln "@OneToMany\|@ManyToMany\|@ManyToOne\|@OneToOne" openaev-model/src/main/java/io/openaev/database/model --include="*.java" \
| xargs grep -B1 -n "{Entity}"
Read every hit's owning entity ({OwningEntity}), not just {Entity} itself
— the collectors bug lived on SecurityPlatform, an entity that has nothing
else to do with the collectors activation.
3b.2 — for every {OwningEntity} found, find every accessor call across the whole codebase, including serialization:
grep -rn "\.get{Entities}()\|\.get{Entity}()" openaev-api/src/main/java openaev-model/src/main/java --include="*.java"
Classify each call site:
- inside an explicit query (
JOIN FETCH, a@Queryprojecting the association) → already eagerly resolved inside the query's own transaction; check that query's controller/service carriesTxCtx(same rule as Phase 1/5). - a lazy getter called directly in a
@Transactionalmethod body → resolves inside that transaction; the CALLER method needsTxCtx(Phase 5 if it is not the table's own API). - a lazy getter reached ONLY through JSON serialization (a custom serializer
like
MultiIdListSerializer, a DTO mapper invoked by Jackson, a@JsonSerializefield) → the dangerous case. With open-in-view or any serialization step that runs after the controller method returns, the association resolves OUTSIDE the transaction the aspect scoped. Force-initialize the association INSIDE the scoped transaction, before the method returns — and make sure that controller method itself carriesTxCtx, since a lazy association resolved eagerly under no scope still reads zero rows. The CORRECT helper depends on the association's cardinality, not on which one happens to compile. Read 3b.2a below before picking one; treatingHibernate.initialize(...)andJOIN FETCHas interchangeable is what makes this step easy to get wrong. - no controller ever serializes it, only used inside a background job → treat as Phase 5b (background reader), not this phase.
3b.2a — which fix, by cardinality. Both patterns run the SAME rewritten, tenant-scoped SQL (the inspector inspects every statement a session issues, regardless of which helper triggered it) — the difference is what happens when the target row is invisible under the caller's scope, and that difference is driven entirely by the association's cardinality:
| Cardinality | What an invisible target does | Safe pattern | Model |
|---|---|---|---|
@OneToMany / @ManyToMany (a collection) |
The collection query is naturally a WHERE fk = ?-style list; an out-of-scope child row is just excluded from the list. Degrades to an empty collection, never throws. |
Hibernate.initialize(owning.get{Entities}()) inside the scoped transaction, right where the association is needed, before the method returns. |
SecurityPlatformApi's withCollectorsInitialized (Hibernate.initialize(securityPlatform.getCollectors()), PR #7026); InjectHelper (Hibernate.initialize(inject.getTags()), .getTeams(), …) |
@ManyToOne / @OneToOne (a single required reference), no @NotFound on the field |
Hibernate assumes referential integrity: initializing a proxy whose target row fails can_access_tenant(...) returns zero rows for a lookup-by-id, and Hibernate throws ObjectNotFoundException/EntityNotFoundException — an unhandled 500 at the point of initialization, not an empty result. Hibernate.initialize() is NOT safe here by default. |
JOIN FETCH owner.association in the SAME query that loads the owning row (or an @EntityGraph), not a separate Hibernate.initialize() call. JPQL's default JOIN FETCH is an INNER join: if the referenced row fails the tenant predicate, the join condition fails and the OWNING row itself is silently dropped from the result set — no exception, no null to handle, and it matches "nothing visible" semantics for the caller for free. |
InjectExpectationRepository#findChallengeExpectationsByExerciseAndUser / #findByUserAndExerciseAndChallenge, JOIN FETCH i.challenge added when challenges went v2-active (#6416) |
@ManyToOne / @OneToOne, association ALREADY carries @NotFound(action = NotFoundAction.IGNORE) for unrelated referential-integrity reasons (model: Inject.java, InjectorInjectorContract.java) |
null is already the documented, handled outcome for a missing target — an invisible-under-tenant-scope target degrades the exact same way a genuinely-deleted one already does. |
Hibernate.initialize() is fine to reuse as-is; verify the annotation is already there before assuming it, and confirm every existing caller already null-checks the getter. |
n/a — check the field's existing annotations first |
A required @ManyToOne/@OneToOne where you want a null instead of a dropped owning row (rare — usually only right at the association's own aggregate boundary, not through an unrelated join) |
Adding @NotFound(action = NotFoundAction.IGNORE) specifically to unlock this makes the association permanently EAGER for every caller, not just this one — it cannot stay lazy once Hibernate must silently swallow a missing target. Treat this as a deliberate, wider mapping change, not a query-local fix, and audit every other caller of that getter before adding it. |
Prefer JOIN FETCH (row above) unless you have a specific reason this is insufficient; if you do add @NotFound, you must also add the null-handling code downstream yourself — nothing does that for you. |
— |
Quick check before picking a row — confirm the cardinality and any existing @NotFound on the field:
grep -n -B3 "{fieldName}" openaev-model/src/main/java/io/openaev/database/model/{OwningEntity}.java \
| grep -E "@OneToMany|@ManyToMany|@ManyToOne|@OneToOne|@NotFound"
Getting this wrong ships one of two ways: pick Hibernate.initialize() on a
*ToOne with no @NotFound and the association throws at initialization time
instead of degrading — a 500 on whatever request happens to first touch an
invisible target, appearing later and further from the original access than
the read that triggered it; pick JOIN FETCH on a *ToMany and you likely
don't crash, but you lose the chance to reuse an already-loaded owning row
across multiple associations the way Hibernate.initialize() naturally allows
— it isn't wrong, just needlessly more invasive than the one-liner.
3b.3 — pin every fixed accessor with an ArchUnit rule and a scoped test:
- Add every
{OwningEntity}#get{Entities}caller found above to the association-accessor allowlist rule inTenantActiveTableAccessArchTest(same rule described in Phase 6, step 5), even for{OwningEntity}s that are not the table's own aggregate. A new, un-allowlisted caller must fail the build. - Add every serializing entrypoint from 3b.2 to
TenantScopedEntrypointsTxCtxArchTest(TX_SCOPED_ENTRYPOINTS), same as any otherTxCtx-bearing entrypoint. - Write one test per fixed entrypoint, modeled on
SecurityPlatformCollectorsTenantScopeTest(PR #7026): run with@TestPropertySource(properties = "openaev.tenant.active-tables={table}")(production-like, inspector active — the default test profile's empty allowlist is exactly what let #7026 through) and assert the association serializes the live link for an in-scope row, and comes back empty only once the linked{Entity}row is genuinely gone (not merely out-of-scope).
Do not defer this phase to "later regression pass" — an association missed here degrades silently (200 OK, empty array) exactly like #7026, so nothing in Phase 8's regression run will catch it unless the new test from 3b.3 exists.
3b.4 — computed getters and DTO mappers that resolve the activated table
(the #7605 / #7621 shape). An association accessor is not the only silent
reader. A COMPUTED getter on an unrelated entity — a @JsonProperty method
with no column of its own that walks a relation to {Entity} to derive a
scalar — reads the activated table on every serialization, and it is invisible
to all three greps above: it names neither {EntityRepository} nor {table},
and it is not an @OneToMany/@ManyToOne field. Inject#getType() is the
reference: @JsonProperty("inject_type") resolving
injectorContract.getFirstInjector().getType() on the v2-scoped injectors
table. Its callers are three DTO mappers (InjectMapper#toInjectOutput,
InjectMapper#toInjectResultOverviewOutput,
InjectStatusMapper#toInjectTestStatusOutput) plus direct entity
serialization, so EVERY endpoint returning Inject, InjectOutput,
InjectResultOverviewOutput, InjectResultOutput or InjectTestStatusOutput
reads injectors. The activation wired the obvious inject endpoints and
missed the rest; they shipped 200 OK with inject_type: null, which the
frontend renders as the generic "unknown" icon on the whole Execution screen
(time-based AND chaining) — a silent regression found in production, not in
CI.
Walk it in two directions, and treat BOTH as part of the closure:
# 3b.4.a - computed @JsonProperty getters anywhere in the model that resolve
# {Entity} without naming {table} or {EntityRepository}
grep -rn -B3 "get{Entity}()\|get{Entities}()\|getFirst{Entity}()" \
openaev-model/src/main/java/io/openaev/database/model --include="*.java" | grep -n "@JsonProperty" -B3
# 3b.4.b - every caller of each computed getter found (mappers included)
grep -rn "\.{computedGetter}()" openaev-api/src/main/java openaev-model/src/main/java --include="*.java"
# 3b.4.c - THE SINK SWEEP: once a DTO/entity is known to carry the computed
# value, enumerate EVERY endpoint whose return type is that DTO/entity, and
# check TxCtx on each one. This is the step that was skipped in #7605.
grep -rn "public .*\b{SinkType}\b\|Page<{SinkType}>\|List<{SinkType}>\|Iterable<{SinkType}>" \
openaev-api/src/main/java --include="*Api.java"
Rules for this sub-phase:
-
The unit of enumeration is the RESPONSE TYPE, not the API package. A computed getter leaks through
TeamApi,PlayerApi,OrganizationApi,AssetGroupApi,EndpointApi, ... simply because they return the same DTO; none of them mentions the activated table anywhere. Sweep by sink type across all controllers, then diff that list againstTX_SCOPED_ENTRYPOINTS: every endpoint returning a sink type must appear in one of the two lists (wired, or explicitly justified as never serializing the computed value). -
A criteria/JPA projection that SELECTs the derived column (
injectorJoin.get("type").alias("inject_type")) is the same sink: it joins the activated table inside the query, so its endpoints needTxCtxexactly like the lazy-getter path. -
@Transactional(propagation = Propagation.SUPPORTS)handlers (bulk update/delete, massive-operation wrappers) are a trap: with no inbound transaction the aspect has nothing to scope, so addingTxCtxalone does NOT fix them. Either the service opens the scoped transaction, or the handler is switched to a real@Transactionalboundary — decide and write it down, do not leave aTxCtxparameter that silently does nothing. -
Deprecated endpoints returning the sink type count (they still ship): the
/api/exercise/{id}/injects/testvariant is as live as its/injects/test/searchsuccessor. -
Pin the sweep: for each sink type, add one production-like test (
@TestPropertySource(properties = "openaev.tenant.active-tables={table}")) asserting the computed field is NON-NULL on a representative endpoint per controller family, not just on the table's own API. A null-valued scalar is the failure mode; an empty-array assertion will not catch it. -
That pin must NOT be
@Transactional, and this is not a style detail. The defect is a lazy association resolved by Jackson AFTER the handler's transaction closed. A@Transactionaltest class keeps that transaction open: the handler joins it, serialization happens inside it, the scope is still set, and the assertion passes. The test then looks like a pin and is not one. Seed through a committedTransactionTemplateand sweep the rows in an@AfterEach, so the request's own transaction is the one that closes before serialization.Three defects of this family have shipped or nearly shipped:
SecurityPlatform#collectors(#7026, found in production months after go-live),Finding.getAssetGroups()andFinding.assets(#6420), andAssetGroup.assets(#6438). The last two were each covered by a sink test that passed while the endpoint returned an empty array, because both tests were transactional. Correcting the test is what turned them red.
Phase 4 — RED then GREEN: write attribution
The inspector cannot attribute INSERT ... VALUES. Attribution is application
code, and it is the part most often forgotten.
Model: createImportMapper and importMappers in MapperApi.java, plus
MapperService.createAndSaveImportMapper in
openaev-api/src/main/java/io/openaev/service/MapperService.java.
On every create/import endpoint:
String tenantId = writeScopeResolver.tenantForWrite(ctx, null);
and in the service, stamp the entity before save:
entity.setTenant(new Tenant(tenantId));
Rules enforced by TenantWriteScopeResolver (do not reimplement them, inject
the component): single-tenant scope → that tenant; supplied tenant outside
scope → 400; multi-tenant scope without selector → 400.
An upsert (or any find-or-create by business key) is both paths at once, and
it hides the nastiest trap of an activation. Do NOT look the row up by the
bare business key under the request scope: after activation the unique key is
per-tenant (business_key, tenant_id), so under a MULTI-tenant read scope the
lookup can match one row per in-scope tenant. Concretely it either crashes
(IncorrectResultSizeDataAccessException on the Optional) or, when a single
in-scope tenant owns the key, silently links ANOTHER tenant's row to your
write. The per-tenant preset data makes duplicated business keys the NORMAL
case, not an edge. The correct order, proven on the cwes activation:
- Resolve the write tenant FIRST:
tenantForWrite(ctx, null)(an ambiguous multi-tenant scope is refused with 400, loudly, before any lookup). - Look up by the per-tenant unique key: a repository method
findBy{BusinessKey}AndTenantId(key, writeTenant)(model:CweRepository#findByExternalIdAndTenantId, used byVulnerabilityService#updateCweAssociations). - The insert branch stamps the entity with that same write tenant.
The update branch needs nothing extra; the inspector already refuses to touch a row outside the scope.
The plumbing also offers @RequireTenantSelector (400 when the request
carries no explicit selector, see
openaev-api/src/main/java/io/openaev/config/RequireTenantSelector.java).
The pilot does not use it: the resolver's single-tenant rule already refuses
ambiguous writes. Do not add it unless the endpoint must refuse even an
implicit single-tenant scope.
Testing the plain (non-isolation) create path. Outside the two-tenant
{Entity}HttpIsolationTest suite above, an ordinary create/import test (e.g.
{Entity}ApiCapabilityTest, a permissions test) that hits the same
@RequireTenantSelector-gated endpoint needs the mock user to resolve to
exactly ONE authorized tenant, or tenantForWrite refuses it with 400 (empty
authorized set is not a single tenant either). If that test doesn't already
grant the mock user a tenant of its own, add
@WithMockUser(autoJoinDefaultTenant = true) on that ONE test method — never
at the class level, and never on a test in the isolation suite above, which
must keep the default false (see Phase 2's note on this same flag). Models:
TagApiCapabilityTest#given_manageTags_should_createTag,
ImportExportMapperApiTest#testImportCsvWithEndpointsCsvType.
Do NOT keep TenantBaseListener / TenantIdBaseListener on the entity. It is
a v1 pattern that reads from TenantContext — which is no longer the source of
truth for activated tables. Write attribution is now explicit via the resolver;
any test that relied on the listener auto-populating tenant must be fixed to set
tenantId explicitly. Keeping the listener creates a hidden fallback path that
masks missing write attribution and blocks the eventual removal of
TenantContext from the codebase.
Write-path completeness check — before removing the listener, audit every persist call on the entity:
# 4.1 Find every save/persist of the entity
grep -rn "{EntityRepository}\.\(save\|saveAll\|saveAndFlush\)\|persist({entity}\|merge({entity}" \
openaev-api/src/main/java openaev-model/src/main/java --include="*.java"
# 4.2 For each hit, confirm one of:
# (a) it calls writeScopeResolver.tenantForWrite(ctx, ...) and stamps the entity
# with setTenant(new Tenant(tenantId)) BEFORE the save — a CREATE path
# (b) it is an UPDATE-only path (the row already exists with tenant_id set;
# the inspector scopes the statement, no re-attribution needed)
# (c) it is a background writer covered by Phase 5b (stamped inside a
# forEachTenant or execute(forTenant(id), ...) scope)
Any save that creates a new entity without explicit tenant attribution is a
silent data corruption once the listener is removed — tenant_id will be
NULL. Fix it (add tenantForWrite + setTenant) before proceeding to go-live.
This check catches paths that the Phase 1 inventory (which greps for readers)
and the Phase 2 isolation test (which covers the main API only) can miss:
services shared by multiple controllers, internal helpers, bulk importers.
Re-run: create/import/upsert tests green, including "no selector → 400".
Phase 5 — Other paths from the inventory
For every other API or service found in Phase 1 that reads the table: per
the baseline, its @Transactional entrypoint should already carry TxCtx —
confirm it, and only add the parameter if it is genuinely missing (baseline
drift, or a handler newly made @Transactional). Then:
- add an isolation test proving the cross-tenant case through that path.
Models:
openaev-api/src/test/java/io/openaev/rest/scenario/ScenarioImportApiTenantIsolationTest.javaandopenaev-api/src/test/java/io/openaev/rest/exercise/ExerciseImportApiTenantIsolationTest.java.
Then add the non-admin proof. Model:
openaev-api/src/test/java/io/openaev/rest/mapper/ImportMapperNonAdminIsolationTest.java
(@WithMockUser(isAdmin = false), tenants seeded with
tenantHelper.createTenantWithCapabilities(...)). Isolation must hold without
the admin flag.
Do NOT re-prove the out-of-rights selector refusal (403): that is shared
plumbing, already covered by
openaev-api/src/test/java/io/openaev/config/TenantSelectorMembershipTest.java.
Finally, register every new TxCtx-bearing entrypoint in
openaev-api/src/test/java/io/openaev/architecture/TenantScopedEntrypointsTxCtxArchTest.java
(add "{package}.{Api}#methodName" entries to TX_SCOPED_ENTRYPOINTS), then
run it:
mvn -ntp -pl openaev-api test -Dtest='TenantScopedEntrypointsTxCtxArchTest,TenantNonOrmAccessArchTest'
Phase 5b — Background writers: convert to the primitive
Skip this phase only if Phase 1 found NO background path that touches the table.
Otherwise every background writer (and every background reader that must keep
seeing rows) is converted here, BEFORE go-live: once the table is in
active-tables, an unconverted background path reads and writes zero rows,
silently.
Model conversion: UrlAccessTokenPurgeJob
(openaev-api/src/main/java/io/openaev/scheduler/jobs/UrlAccessTokenPurgeJob.java),
converted to tenantTx.execute(TxCtx.allTenants(), …) in PR #6398. Injected
dependency: TenantScopedTransaction tenantTx.
Maturity note (refreshed for #6398): the background path is now well
proven. Nine Quartz jobs sit directly on the primitive (eight top-level jobs plus
the nested EngineSyncExecutionJob.Job, which implements org.quartz.Job and calls
TenantScopedTransaction itself), and several more reach it through
TenantScopedJobRunner. Working model implementations exist for all three idioms:
allTenants()(bulk read or predicate delete across tenants):UrlAccessTokenPurgeJobandNotificationEventRetentionService.deleteOldEvents.forEachTenant(per-tenant, one transaction each), three production callers:AutonomousTimeoutService.sweep,XtmHubService.refreshConnectivityAllTenants, andExpectationsExpirationManagerJob.run.execute(TxCtx.forTenant(id), …)(one known tenant): viaTenantScopedJobRunner.runInTenant, used byInjectsExecutionJob,InjectsFinalizationJobandWorkflowTimeoutJob.
Copy the idiom that matches the scope decision below, not a single blessed job.
Every background family is now enumerated and classified by the background guard
(BackgroundEntrypointTenantScopeArchTest + background-guard-baseline.txt): a
new background entry point in one of the six recognised families fails the build
until it is on the primitive or classified there with a reason. This is
build-time enumeration, not a runtime scope check: see the "Known limits" note
below for what it does not prove.
Enumerate every background path first. Phase 1's greps are repository- and table-name-oriented and can miss a background surface. There is no single reliable grep, so sweep several and READ each hit:
# scheduled work and startup tasks
grep -rn "@Scheduled\|implements Job\|extends QuartzJobBean\|ApplicationRunner\|CommandLineRunner\|@PostConstruct" \
openaev-api/src/main/java/io/openaev/scheduler openaev-api/src/main/java --include="*.java" | grep -i "{table_or_entity}"
# queue / broker consumers
grep -rn "@RabbitListener\|@KafkaListener\|MessageListener\|consume\|@EventListener" \
openaev-api/src/main/java --include="*.java" | grep -i "{table_or_entity}"
# then, for each candidate job/consumer class, check whether it reaches the repository
grep -rln "{EntityRepository}\|{Entity}Service" openaev-api/src/main/java/io/openaev/scheduler --include="*.java"
A background path that WRITES the table and is not converted here is a go-live blocker (hard rule 2). List every background hit in the Phase 9 report with its scope choice or its blocker reason.
Rules for the background write path (all guarded by the ArchUnit rules in
TenantBackgroundTransactionRules.java, enforced frozen by
TenantBackgroundTransactionArchTest.java):
- No
@Transactionalon the background write. Its self-invocation trap skips both the transaction and the scope with no error. Open the transaction with the primitive instead. - No raw transaction plumbing in jobs (
TransactionTemplate,PlatformTransactionManager, manualgetTransaction/commit). The primitive is the one door. - Never catch-and-continue inside a single transaction. Any runtime exception
from a joined
@Transactionalservice marks the whole transaction rollback-only; carrying on dies at commit (UnexpectedRollbackException). Recover around a boundary, not inside one.
Choose the scope by what the job does:
| The job... | Scope | Call |
|---|---|---|
| does the same unit of work for every tenant, INSERTING or updating per-tenant rows | per tenant, one transaction each | tenantTx.forEachTenant(ctx -> …) |
| already works on one known tenant | that tenant | tenantTx.execute(TxCtx.forTenant(id), work) |
already runs its tenants IN PARALLEL on its own executor (model: ManagerIntegrationsSyncJob) |
per tenant, one transaction per task | keep the executor; each task calls tenantTx.execute(TxCtx.forTenant(id), work) |
| does a single bulk read, or a bulk delete/update by predicate, spanning all tenants | all active tenants, resolved | tenantTx.execute(TxCtx.allTenants(), work) |
A row insert cannot be attributed under allTenants(): a new row belongs to
exactly one tenant, and TenantWriteScopeResolver.tenantForWrite refuses the
intention (TenantWriteScopeException). So a job that INSERTS per-tenant rows
uses forEachTenant (or execute(forTenant(id), …)), never allTenants().
allTenants() fits a bulk read, or a bulk delete/update BY PREDICATE: those
never call tenantForWrite, the inspector simply scopes the statement to the
resolved tenant list, so no per-row attribution is needed. The
UrlAccessTokenPurgeJob model is exactly such a bulk delete under
allTenants(), not a read.
Native and raw SQL — the background-job trap. Background jobs lean on hand-optimized SQL more than HTTP code (queries rewritten as native to dodge ORM inefficiency). The two forms behave very differently once the table is active:
- Native through Hibernate (
@Query(nativeQuery = true),entityManager.createNativeQuery(...)) DOES pass through the statement inspector: it is a HibernateStatementInspector, so it sees this SQL and scopes it. The risk is not a leak, it is availability: the inspector is fail-closed, so a shape it cannot parse or rewrite (a multi-targetDELETE, a target-side-joinUPDATE, an exoticFROM, an unusual statement) throwsTenantFilteringExceptionand the query BREAKS once the table is active. Hand-optimized job queries (CTEs, window functions, unusual joins) are exactly the shapes most likely to hit "not yet covered". Every native query on the table used by a background path must therefore be EXERCISED by a test (the Phase 5b isolation test or the regression suite) so a rewrite failure surfaces in CI, not in production. If the inspector refuses a query, rewrite it into a covered shape or postpone the activation; never bypass the inspector. - Raw JDBC (
JdbcTemplate,NamedParameterJdbcOperations, a directConnection/Statement) BYPASSES Hibernate entirely, so the inspector never sees it: a silent cross-tenant read and an unattributed write. This is already guarded byTenantNonOrmAccessArchTest(no_raw_jdbc_outside_the_allowlist), which fails the build on any new raw JDBC in production code outside the audited@AllowRawJdbcallowlist (which covers non-tenant tables, plus the two narrow test-enforced exceptions below). A raw-JDBC path touching the table you are activating is a HARD BLOCKER: convert it to go through Hibernate (so it gets inspected) before go-live. Never@AllowRawJdbca tenant table to make a job compile. - Narrow exception — a provably insert-only bypass. The rule above is
deliberately blunt: it cannot tell an
INSERT ... VALUES-only path (which the inspector would not scope anyway, since tenant assignment on a VALUES insert stays an application concern) from a read/update bypass (which silently reads or writes across tenants). A raw-JDBC path may keep@AllowRawJdbcon a tenant table ONLY when all three hold: it emits nothing butINSERT, every insert carriestenant_idas an explicit column, and a test enforces both on every build so the exemption cannot silently widen. The seed generator for the attack-path tables is the reference (AttackPathSeedServiceTest). Absent that enforced insert-only proof, the HARD BLOCKER stands. - Narrow exception — a provably read-only scope-bootstrap lookup. Deriving
a service-identity scope FROM a row (an orchestrator callback scoped by its
parent run's own tenant) is a chicken-and-egg: the read that decides the
scope cannot run under the scope it is deciding, and the inspector would
fail-close it. A raw-JDBC path may keep
@AllowRawJdbcfor that bootstrap ONLY when all three hold: it emits nothing but a single-rowSELECTof the row's own immutabletenant_idaddressed by primary key, it projects no other column, and a test pins the bypass class list and the exact statement on every build so the exemption cannot silently widen. The bootstrap read must also respect tenant liveness (filter ontenant_deleted_at IS NULL): a soft-deleted tenant is excluded from every caller scope for its whole grace period, so a row-derived scope that ignored the flag would quietly re-admit a tenant no caller can reach. The autonomous-run callback locator is the reference (AutonomousRunTenantLocator/AutonomousRunTenantLocatorTest). Absent that enforced read-only proof, the HARD BLOCKER stands.
Add a grep for both to the inventory and read each hit:
grep -rn "nativeQuery *= *true\|createNativeQuery" openaev-api/src/main/java openaev-model/src/main/java --include="*.java" | grep -i "{table_or_entity}"
grep -rn "JdbcTemplate\|NamedParameterJdbc\|getConnection\|@AllowRawJdbc" openaev-api/src/main/java --include="*.java" | grep -i "{table_or_entity}"
Write attribution is the same as HTTP. Inside the scope, resolve and stamp:
String tenantId = writeScopeResolver.tenantForWrite(ctx, null);
entity.setTenant(new Tenant(tenantId));
Under forEachTenant each iteration carries a single-tenant scope, so
attribution resolves cleanly per tenant.
Nesting. If the converted job joins a @Transactional service that carries
a NARROWER TxCtx, the aspect's nesting guard refuses it and poisons the
transaction. Open the narrower scope with executeNew from the start, never by
narrowing inside the same transaction.
RED first, then GREEN — the background isolation test. Model:
openaev-api/src/test/java/io/openaev/context/TenantScopedTransactionIntegrationTest.java
(single scope) and
openaev-api/src/test/java/io/openaev/context/TenantScopeAllTenantsIntegrationTest.java
(allTenants() and forEachTenant, including the per-tenant rollback proof).
Key differences from the HTTP test:
- The test class is NOT
@Transactional. The primitive'sexecuterefuses to open inside an active transaction, so seed and clean through auto-committedJdbcTemplate, not a rolled-back test transaction. - Activate the table with
@TestPropertySource(properties = "openaev.tenant.active-tables={table}"), same as the HTTP test. - Prove: the converted job under its scope sees and writes only the in-scope tenant's rows; a cross-tenant row is invisible; for a per-tenant loop, one tenant's failure is rolled back on its own and does not poison the others.
Run it red, wire the conversion, run it green, keep both outputs (hard rule 8):
mvn -ntp -pl openaev-api test -Dtest='{Entity}BackgroundIsolationTest'
ArchUnit baseline. Converting a writer that currently sits on the frozen
background baseline (a @Transactional job, a raw-template job) SOLVES recorded
violations — and that FAILS the locked build: FreezingArchRule removes solved
violations from the store, which is a store write, and under
freeze.store.default.allowStoreUpdate=false the write throws
StoreUpdateFailedException (verified in ArchUnit 1.4.2). The conversion PR must
therefore include a deliberate store refresh; the full procedure (triage, fix
patterns, tests, re-freeze commands) is its own runbook:
.github/skills/reduce-tx-baseline/SKILL.md. Never hand-edit the store files.
Known limits of the background path — name them in the report, do not paper over them:
- No runtime scope guarantee for background writers. The HTTP side is pinned by
TenantScopedEntrypointsTxCtxArchTest, which fails the build if an active table's handler loses itsTxCtx. The background side now has a build-time analogue:BackgroundEntrypointTenantScopeArchTest(+background-guard-baseline.txt) enumerates the six background families and fails the build until a new entry point is on the primitive or classified in the baseline with a reason. That closes the "new job slips in unnoticed" gap, but only structurally: the guard proves every entry point is enumerated and reasoned, NOT that the SQL each one runs actually carries a scope. A path waivedtouches-no-tenant-table, or one whose class-level waiver no longer fits a method added later, can still read or write an already-active table with the wrong tenant and no test fails. The existing rules forbid the wrong SHAPE (@Transactional, raw plumbing, raw JDBC) but do not assert that every writer of an active table carries a real scope. Until a runtime guard exists, converting a table's writers is a point-in-time fact, not an invariant; say so in the report. - The per-tenant loop is serial and single-threaded, one transaction per tenant.
For a job over thousands of tenants, watch total runtime against the job's
window (
@DisallowConcurrentExecutionmeans an overrun skips the next fire). The loop itself provides no batching or parallelism, but parallel per-tenant work is NOT a workaround: a job with its own executor opens one transaction per task (see the scope table above), and concurrent scoped transactions are proven isolated by test, for reads and writes. Size such an executor against the connection pool: each concurrent task holds one pooled connection for its whole transaction, and an oversized fan-out starves the HTTP path. A nestedexecuteNewholds two. When the converted write is not a top-level job but a hook inside an already-transactional caller (e.g. a per-inject write from the run flow),executeNewopens a second, REQUIRES_NEW transaction, so that task holds two pooled connections at once for the duration of the inner write. Concurrency is then bounded bypool_size / 2, notpool_size, and K parallel callers each demanding their second connection at once can deadlock on the pool until the Hikari timeout. Size the caller's concurrency against half the pool, or keep the inner write short so the second connection is held briefly. And await every task before the job method returns: a fire-and-forget job defeats@DisallowConcurrentExecution, so the next fire could open a second transaction on the SAME tenant. If the SEQUENTIAL loop's runtime becomes the concern, raise it rather than hand-rolling a second loop idiom. - Partial failure is visible only in logs.
forEachTenantruns every tenant and throws one aggregate at the end, so the job is marked failed even when most tenants succeeded. Operators see "failed"; the per-tenantlog.warnand the aggregate's suppressed causes carry what actually happened. A success/failure summary metric is a follow-up, not part of the primitive.
Phase 6 — Go-live: ONE commit
Model (from PR #6255): commit "feat(multi-tenancy): activate import_mappers on v2 isolation (#6212)"
(find it with git log --oneline --grep "activate import_mappers on v2 isolation"). Four changes, together, nothing else:
- Remove
@Filter(name = "tenantFilter", ...)and remove theTenantBaseListener.class(orTenantIdBaseListener.class) from@EntityListenersinopenaev-model/src/main/java/io/openaev/database/model/{Entity}.java. Replace it with a javadoc comment stating the table is fully on v2 and why the v1 filter and listener must not come back. Fix any test that relied on the listener to auto-populatetenantId— set it explicitly instead. - Append
{table}toopenaev.tenant.active-tablesinopenaev-api/src/main/resources/application.properties(comma-separated, keep existing entries). - Re-enable the one
@Disabledheader-route test from Phase 2. - Extend the production-config guard so dropping the table from the allowlist
fails the build. Model:
openaev-api/src/test/java/io/openaev/config/ImportMapperActivationConfigTest.java. Prefer extending a shared guard over cloning the file; the assertion must name{table}explicitly. - Extend the access guard
(
openaev-api/src/test/java/io/openaev/architecture/TenantActiveTableAccessArchTest.java): add{table}toGUARDED_TABLES, a repository rule allowlisting the Phase 1 inventory (each entry commented with its scope mechanism), and an association-accessor rule for every entity accessor that lazy-loads the table from another aggregate (the cwes model:Vulnerability#getCwes). The guard's completeness check fails the build if the table is activated without this step — that is deliberate.
If anything in this phase needs "just one more fix" in production code, stop and go back to the phase that owns that fix. The go-live diff stays minimal.
Rollback story, decide it now, not during an incident: if production
misbehaves after go-live, revert the WHOLE go-live commit (the @Filter
comes back in the same change that removes the allowlist entry). Never remove
just the property: with the @Filter gone that is an isolation hole, and the
config guard fails the build precisely to stop that move.
Parallel activations: other tables go live through the same
application.properties line and the same TX_SCOPED_ENTRYPOINTS set.
Rebase right before go-live and re-read the merged allowlist line; a bad
merge that drops another table's entry is what the shared config guard
catches, do not rely on it alone.
Phase 7 — v1 remnant audit
Before running the regression suite, audit the full call stack of every public
method on {Api} (and on the other APIs from Phase 5) for v1 isolation
patterns that conflict with or duplicate the v2 mechanism.
What to search for:
TenantContext.getCurrentTenant()— the v1 thread-local tenant. Any call in the controller, service, repository, or specification layer that touches{table}is a v1 remnant. The v2 inspector handles scoping; the thread-local is no longer the source of truth for activated tables.findByIdAndTenantIdor any repository method that explicitly filters bytenant_idon{table}— redundant and restrictive. The inspector already scopes reads; an explicitAND tenant_id = ?prevents multi-tenant queries that v2 intentionally supports.TenantBaseListeneron@EntityListenersof the entity — if write attribution is now explicit viaTenantWriteScopeResolver, the listener is dead code for this entity and should be removed to avoid a hidden fallback toTenantContext.// TODO v2:markers anywhere in the codebase naming{table}— a previous activation or bugfix may have left an explicit tenant-scoping workaround (an extratenantIdparameter on a native query, a manual filter, a duplicated lookup) specifically BECAUSE{table}(or an entity it joins) was still on v1, with a comment pointing at this table's activation issue as the trigger to remove it. This is the ONLY mechanism that carries such a workaround forward from one activation to the next — the skill's own Phase 7 audit is scoped to the table being activated NOW and does not revisit it later, so a workaround left on another table's query is invisible to this phase unless it is grepped for explicitly. Model: thetenantIdparam added toConnectorInstanceConfigurationRepository.findInstanceAndCatalogIdsByKeyValueAndTenantId(issue #6408 forconnector_instances) — a native query joiningconnector_instanceshad to carry the tenant predicate by hand because the join target was still v1; onceconnector_instancesactivates, the inspector scopes that join automatically and the parameter becomes removable.
How to search:
# 7.1 Direct usage in the API package and its services
grep -rn "TenantContext.getCurrentTenant\|findByIdAndTenantId\|findByTenantId" \
openaev-api/src/main/java/io/openaev/rest/{domain}/ \
openaev-api/src/main/java/io/openaev/service/ \
--include="*.java" | grep -i "{table_or_entity}"
# 7.2 Repository-level tenant filtering on the activated table
grep -rn "TenantId\|tenant_id\|tenantId" \
openaev-model/src/main/java/io/openaev/database/repository/{EntityRepository}.java \
openaev-model/src/main/java/io/openaev/database/specification/*{Entity}*.java
# 7.3 Deep stack: check services called by the API methods
grep -rn "TenantContext" \
$(grep -rln "{EntityRepository}\|{Entity}Service" openaev-api/src/main/java --include="*.java")
# 7.4 TenantBaseListener still on entity
grep -n "TenantBaseListener" \
openaev-model/src/main/java/io/openaev/database/model/{Entity}.java
# 7.5 codebase-wide: workarounds left for THIS table's activation, wherever
# they live (may be on a repository/service far from {table}'s own package)
grep -rln "// TODO v2:" openaev-api/src/main/java openaev-model/src/main/java --include="*.java" \
| xargs grep -l "{table}\|{Entity}\|{table_or_entity}"
Classification and action:
| Finding | On {table}? |
Action |
|---|---|---|
TenantContext.getCurrentTenant() used to look up {Entity} |
Yes | Remove — the inspector scopes the query |
TenantContext.getCurrentTenant() used to look up a different entity |
No | Report only — out of scope for this activation |
findByIdAndTenantId on {EntityRepository} |
Yes | Replace with findById — inspector handles scoping, explicit tenant filter blocks multi-tenant reads |
findByIdAndTenantId on a different repository |
No | Report only — that table is still on v1 |
TenantBaseListener on {Entity} |
Yes | Remove — it is a v1 pattern that reads from TenantContext; write attribution is now explicit via the resolver. Fix any test or code path that relied on the listener to auto-populate tenant; set tenantId explicitly instead |
Specification with tenant_id predicate on {table} |
Yes | Remove the predicate — inspector handles it |
// TODO v2: comment naming {table}'s activation issue |
Yes | Resolve it now: drop the explicit tenant workaround, add/update a regression test proving the inspector's automatic scoping covers the case, remove the comment |
// TODO v2: comment naming a different table's activation issue |
No | Report only — leave it, it is that other table's future trigger |
Output: audit report
Produce a table listing every hit, its file and line, whether it targets the activated table, and the action taken (removed / reported-only). Include it in the Phase 9 report. Hits on other tables are informational — they document the v1 surface that remains for future activations.
When THIS activation itself introduces a new explicit tenant-scoping
workaround on a query that joins a still-v1 table (the mirror image of 7.5 —
you are now the one leaving a marker for someone else's future activation),
leave a // TODO v2: once {other_table} get v2 activated <issue-url>, <what to remove> comment on that workaround so the eventual {other_table} activation
finds it via its own Phase 7.5 grep.
Phase 8 — Full regression pass
# 8.1 re-run the inventory greps: a reader, a background path, a native or raw
# query added while you worked ships broken (or unscoped)
grep -rln "{EntityRepository}" openaev-api/src/main/java openaev-model/src/main/java
grep -rn "nativeQuery *= *true\|createNativeQuery\|JdbcTemplate\|NamedParameterJdbc" \
openaev-api/src/main/java openaev-model/src/main/java --include="*.java" | grep -i "{table_or_entity}"
# 8.2 format and compile
mvn -B -ntp spotless:check || mvn -ntp spotless:apply
mvn -ntp clean install -DskipTests
# 8.3 your tests, then the FULL API suite (needs the Docker services from
# openaev-dev/docker-compose.yml: PostgreSQL, Silo, OpenSearch, RabbitMQ)
mvn -ntp -pl openaev-api test -Dtest='{Entity}*IsolationTest'
Any pre-existing test that now fails is signal, not noise: it is a query on
your table that lost its scope. Fix it by passing TxCtx, never by
deactivating the table or weakening the test.
Phase 9 — Report
Before marking the issue done, write down:
- the red and green evidence: the raw failing assertions from Phase 2 and the final passing summary from Phase 8 (hard rule 8)
- the endpoints wired and the arch-test entries added
- the background writers converted (Phase 5b): each one, its scope choice
(
forEachTenant/forTenant/allTenants) and the reason, and its isolation test - the v1 remnant audit table from Phase 7 (hits found, actions taken, reported-only items)
- every direct or indirect association to the entity found in Phase 3b: its owning entity, whether it was lazy-loaded outside the transaction (the #7026 shape) or already safe, the fix applied, and its scoped test
- background readers left degraded (from Phase 0/1), each with a one-line impact
- the
ProductInventoryMetricCollectorcheck (Phase 1): hit or no-hit, and if hit, whether it was already scoped or converted tocountAcrossAllTenants() - child tables and how they are covered
- client impact: writes now require a single-tenant scope. Calls using the tenant path
(
/api/tenants/{tenantId}/...) already satisfy this; callers using the header route or no selector may get 400 on create/import until they switch to a single-tenant selector.
Definition of Done
- Phase 0 gates passed (strict table; background writers either converted in Phase 5b or reported as blockers), stop conditions reported if hit
- unique constraints on business keys include tenant_id, or a prep migration was done first (own reviewed change)
- inventory complete for what the mass
TxCtxbaseline does not cover: background paths, non-@Transactionalhandlers, native/raw SQL shapes, and association/computed-getter sinks; redone before go-live - controller entrypoints this activation touches were spot-checked for
TxCtx(should already be present per baseline — a miss means baseline drift, fix it here rather than assuming); for each shared gate/helper/ service method found while walking the closure, its full caller list was grepped and every non-baseline-covered caller (background path, OSIV sink) got its Phase 3b/5b treatment — the caller-search was RE-RUN on that symbol even after the first caller looked done, not just the first time it was seen (#6409 regression:changeExerciseStatuswas a third, unwired caller ofthrowIfExerciseNotLaunchablemissed this way; #6410 regression:ThreatArsenalApi#threatArsenals/#threatArsenalsNonTabletop/#threatArsenalwere three sibling callers of the sameInjectorContractServicesearch/association code already wired forInjectorContractApi#injectorContracts, missed because the shared service method was never re-grepped once one caller looked done) - every new/changed method signature carrying
TxCtxhasTxCtxin first parameter position (annotation allowed), and callsites follow the same order consistently - association-accessor scan run for every entity holding a reference to the activated entity, regardless of whether the activated table has its own API (eager/lazy loads bypass the repository grep either way)
- computed-getter scan run (Phase 3b.4): every
@JsonPropertygetter that derives a scalar from the activated table (model:Inject#getType()→injectors) found, its DTO mappers and JPA projections listed, and the SINK SWEEP done — every endpoint returning one of those sink types (entity or DTO), in ANY controller, diffed againstTX_SCOPED_ENTRYPOINTSso none is left unwired (#7605/#7621: the inject endpoints were wired, theInjectResultOutput/InjectTestStatusOutput/InjectResultOverviewOutputendpoints on team, player, organization, asset-group and atomic-testing were not, and shippedinject_type: null) - every
@Transactional(propagation = SUPPORTS)handler returning a sink type is explicitly resolved: either the service opens the scoped transaction or the handler gets a real transaction boundary — noTxCtxparameter left on a SUPPORTS handler where the aspect cannot fire - one production-like test per sink type asserts the computed field is NON-NULL (a null scalar, not an empty array, is this regression's shape)
- isolation test written first and seen red for the mechanism, then green; raw red/green outputs captured in the report
- reads: own row visible, cross-tenant 404, path and header selectors
- writes: attribution asserted at the SQL level, no selector → 400; upsert of the same business key from two tenants yields two rows
- other APIs from the inventory wired and tested
- every association (direct or indirect, lazy or eager) pointing at the entity from ANY other aggregate found (Phase 3b); every lazy accessor reached through serialization (custom serializer, DTO mapper) force- initialized inside a scoped transaction; each fixed entrypoint pinned in both arch tests and covered by a production-like scoped test
- every force-initialization fix from Phase 3b/3b.2a chose its pattern by
the association's cardinality, not by whichever compiled:
*ToManyusedHibernate.initialize()on the collection, a*ToOnewith no existing@NotFoundusedJOIN FETCH/@EntityGraphin the loading query instead ofHibernate.initialize()(which throws on an invisible target for a required reference), and any NEW@NotFound(IGNORE)added to unlockHibernate.initialize()on a*ToOnecame with an audit of every other caller of that getter plus explicit null-handling downstream - background writers converted to the primitive (no
@Transactional, no raw plumbing), each with a per-tenant orallTenantsscope and a green background isolation test (Phase 5b) -
ProductInventoryMetricCollectorchecked for a gauge on this table (Phase 1): no hit, or a hit already/now wrapped incountAcrossAllTenants()— never left as a plainsafeCount({entity}Repository::count) - native queries on the table (
@Query(nativeQuery=true),createNativeQuery) are exercised by a test so a fail-closed rewrite refusal surfaces in CI, not production; any raw JDBC on the table converted to Hibernate (never@AllowRawJdbcon a tenant table) - codebase-wide scan for any native
@QuerythatJOINs the table (not just its own repository) done and every FROM/JOIN shape in those queries checked againstTenantStatementInspector's accepted shapes; any table-function FROM item (jsonb_array_elements,unnest, ...) carriesLATERAL; a regression test pins the real production SQL (#7007) - non-admin variant green
- API v2/integration tests added or modified in this activation do not use
TenantContext.getCurrentTenant(),TenantContext.setCurrentTenant(...), orenableFilter("tenantFilter"); tenant scope is expressed via explicit tenant ids andTxCtx/TenantScopedTransaction - arch tests updated and green
- go-live is one commit: @Filter removed + allowlist entry + re-enabled test + config guard
- v1 remnant audit complete: no
TenantContext/findByIdAndTenantId/TenantBaseListenertargeting the activated table remains in the call stack; codebase-wide// TODO v2:markers naming this table's activation issue resolved - spotless, compile
- report written (degradations, children, client impact, v1 audit table)
Version History
- 3.260921.0 Current 2026-09-23 02:13
-
3.260827.0
2026-08-28 23:28
修复多租户作用域注入列表问题(#7605);修复注入器 API v2 回归问题(#6410)。
- 3.260818.1 2026-08-20 12:00


