HAIAMM v3.5, Infrastructure Domain Handbook

AI/HAI Infrastructure Assurance, the complete practice set for this domain

Generated document. Compiled from the canonical cell records in cells/. Do not edit by hand, edit the cell and re-run python3 tools/render.py domain.

Canonical model: HAIAMM-v3.5-Framing.md · Full model: HAIAMM-v3.5-Handbook.md

Part I, The domain

Subject of this domain: the compute, serving, registry and orchestration substrate AI/HAI systems run on

Boundary rule

The substrate, not the application. IAM that enforces a tool scope is Infrastructure; the tool scope declared in the agent is Software.

Archetypes

Cells in this domain key their content to these archetypes. A generated cell must be usable against every archetype listed, or must say explicitly which archetypes it applies to and why.

1. Inference endpoint / model-serving cluster
2. Model registry
3. GPU / accelerator fleet
4. Orchestrator / control plane
5. Vector-store infrastructure
6. AI-specific CI/CD
7. Feature store / online serving cache

A single asset is often several archetypes at once. Generated content must accommodate that rather than assuming a single clean classification.

Shadow AI in this domain

Resource tagging and cloud asset APIs — untagged GPU instances, registries and endpoints outside the governed platform.

Every domain's SM L1 carries a domain-appropriate shadow-AI discovery mechanism. Downstream practices at L1 assume the inventory it produces exists.

Consumption unit and denominator

Unit: GPU and accelerator hours, inference cost per 1k requests, reserved versus on-demand capacity

Denominator: per 1k inference requests, per served model-hour

What it detects here: shadow model hosting and GPU-shadow-workloads (already the L1 discovery signal in this domain), capacity concentration, and the denial-of-wallet blast radius at the serving layer.

Meter: cloud cost-allocation tags, scheduler and fleet telemetry, model-serving metrics.

Consumption is a governed resource in v3.5, not an incidental discovery signal. Cells for the consumption-sensitive practices (SM, SR, EH, ML, IM) divide this domain's unit by this domain's denominator; absolute spend is a ledger input and never a metric.

Primary stakeholders

DevOps / Platform Engineer · Security Architect · AppSec / SecEng · FinOps

HAI-TTP emphasis

TM and EA via orchestrator and tool-call gateway scope; RA via model-registry tampering and silent version swap; residual-state exposure on shared accelerators.

Compliance overlay

ISO/IEC 42001 operational controls; SOC 2 availability and change management; data-residency obligations on inference paths.

Part II, Effort and ownership

What this domain costs to run, per cell, incremental over the level below. These are the BSAMM-style costs and personnel the v3.0 handbooks promised and did not ship.

Reaching L3 across all twelve practices in the Infrastructure domain is roughly 6,410 hours of stand-up and 4,510 hours per quarter of ongoing effort for a mid-size organization, incremental and cumulative across the three levels. L1 alone is a far smaller commitment, and is where the dependency graph says to start.

Tooling called for in this domain

• AI telemetry ingestion into the existing detection platform
• Adaptive limit management with governed ceilings
• Adversarial testing capability, internal or contracted
• Adversary emulation tooling and an isolated exercise environment
• Automated design criteria evaluation in the delivery pipeline
• Automated remediation platform with scoped action classes
• Automated triage and bounded containment orchestration
• Behavioural baselining across AI telemetry
• Cloud tagging policy enforced in infrastructure-as-code CI
• Configuration conformance scanning against the baseline definitions
• Continuous adversarial testing capability
• Continuous configuration drift detection against declared control state
• Continuous configuration-drift detection against tier hardening baselines
• Continuous conformance assertion evaluation against existing telemetry
• Continuous conformance verification against configuration and runtime state
• Continuous control assertion framework spanning pipeline and runtime
• Continuous drift detection across configuration, policy mode and runtime state
• Continuous registry-to-serving provenance verification
• Detection quality measurement with disposition labelling
• Detection-as-code pipeline with continuous test execution
• Event pipeline joining cloud asset, orchestrator, registry and billing feeds
• Evidence collection automation against existing systems of record
• Evidence map collection wired to repository, CI and configuration systems
• Evolving adversarial harness with case generation
• Executable pattern artifacts in the platform scaffolding
• Existing learning management system, extended with applied-scenario assessment
• Guidance search and question telemetry instrumentation
• Hands-on laboratory environment for Critical-tier curricula
• Per-tenant request and spend limiting at the inference gateway
• Policy-as-code engine integrated into the delivery pipeline
• Regression corpus management with freshness tracking
• Scheduler and accelerator utilisation telemetry joined to cost data
• Test harness capable of exercising AI-specific controls in CI
• Threat model register linked to the asset inventory and change events

Part III, Build order

L1 dependencies within this domain, from the cell records. Start where nothing is required.

Part IV, The twelve practices

Business Function: Governance

Strategy & Metrics (SM)

Objective: Stand up AI infrastructure assurance, inventory every place models are served, stored and trained, and baseline the metrics that prove shadow AI infrastructure is shrinking.

A program at this level knows where its models actually run. One inventory covers all seven infrastructure archetypes, from inference endpoints and model registries through GPU fleets, orchestrators, vector-store infrastructure, AI-specific CI/CD and feature stores. The domain's shadow-AI signal is unusually strong here because compute is expensive: an unexplained GPU spend spike or an untagged accelerator reservation is a reliable indicator of a workload nobody chartered. FinOps sits on the working group by necessity rather than courtesy, because the cost-allocation tags that make infrastructure attributable are the same tags that make the shadow-AI signal readable.

Level 1, Foundational

Stand up AI infrastructure assurance, inventory every place models are served, stored and trained, and baseline the metrics that prove shadow AI infrastructure is shrinking.

Activities

A) Charter AI infrastructure assurance with FinOps at the table — Publish a charter within the first 30 days, sponsored by the CISO jointly with the VP Infrastructure or Head of Platform Engineering. Name the seven in-scope archetypes, being inference endpoint, model registry, GPU or accelerator fleet, orchestrator and control plane, vector-store infrastructure, AI-specific CI/CD, and feature store. Seat FinOps on the working group as a standing member alongside Platform and SRE, Cloud Architecture, ML Platform, AI/ML engineering and Privacy counsel, because in this domain cost allocation is not a reporting convenience but the primary discovery instrument; an untagged accelerator reservation is both a FinOps problem and a security finding, and separating those conversations means neither side sees the whole signal. Define decision rights over who may provision accelerator capacity, who may register and promote a model, who may expose an inference endpoint beyond the cluster boundary, and who can revoke any of the three. Commit to a numeric year-one target per outcome metric.

B) Inventory the estate and read the cost signal for shadow infrastructure — Build one AI infrastructure inventory recording per asset the archetype, owning team, environment, region and residency, accelerator class and quantity, the models served or stored, network exposure, identity and access model, and links to the SA reference pattern and EH hardening baseline it claims to follow. Seed from cloud asset inventories, Kubernetes and orchestrator APIs, model registries and infrastructure-as-code state. Then run the domain's shadow-AI mechanism monthly across four signals: cloud cost-allocation data filtered to accelerator instance types, managed inference services and AI platform products, where unexplained spend or an untagged cost centre is a shadow-infrastructure candidate; scheduler and cluster telemetry showing accelerator workloads whose namespace maps to no chartered team; network egress revealing self-hosted inference endpoints reachable from outside their intended boundary; and registry reconciliation revealing models served from artifacts that were never registered. Cost is the highest-yield of the four and the cheapest to read, because it requires no new agent anywhere.

C) Baseline the infrastructure scoreboard and open the capacity ledger — Instrument and report quarterly to the sponsor: inventory coverage against discovery reconciliation, the shadow-infrastructure ratio, the share of serving endpoints whose model provenance traces to a registry entry, the share of assets carrying an owning team and a cost-allocation tag, and provider and region concentration. Open the capacity ledger recording accelerator-hours and inference cost attributed to owning team and workload, with the denominator declared per archetype, being cost per thousand inference requests for serving and cost per served model-hour for fleet capacity. Report utilisation alongside cost, because an idle reserved fleet and a saturated one cost similarly and mean opposite things. Where accelerator spend is not allocated to an owning team, record the ledger as blocked on allocation and raise the tagging standard with FinOps rather than estimating, since in this domain an unallocated bill is precisely the shadow-infrastructure signal being lost.

Outcome metrics

Consumption metrics — Inference cost per thousand requests, by serving endpoint per per 1k inference requests · Accelerator cost per served model-hour, with utilisation reported alongside per per served model-hour · Unallocated AI infrastructure spend as a share of the ledger per per dollar of AI infrastructure consumption

Success criteria

• A charter is published and jointly sponsored by the CISO and the infrastructure leader, with FinOps a standing working-group member and four provisioning decision rights named.
• One AI infrastructure inventory covers all seven archetypes at 90% coverage or better within 12 months.
• The four-signal shadow-infrastructure sweep runs monthly and the shadow ratio has fallen for two consecutive quarters.
• All production AI infrastructure assets carry an owning team and a cost-allocation tag, and provenance traces to the registry for at least 95% of serving endpoints.
• A capacity ledger is open with denominators declared per archetype and utilisation reported beside cost, or explicitly recorded as blocked on allocation.

Assessment questions

1. Is there a published AI infrastructure assurance charter jointly sponsored by the CISO and the infrastructure leader, with FinOps as a standing working-group member and explicit decision rights over provisioning accelerator capacity, registering models and exposing inference endpoints?
2. Does one inventory cover all seven infrastructure archetypes, reconciled monthly against cloud cost-allocation data, scheduler telemetry, network egress and registry records, reaching 90% coverage or better of discovered assets within 12 months?
3. Do all production AI infrastructure assets carry an owning team and cost-allocation tag, does model provenance trace to a registry entry for at least 95% of serving endpoints, and is a capacity ledger open reporting utilisation alongside denominated cost?

Common pitfalls

• ❌ Treating cost allocation as a finance concern, which discards the domain's cheapest and highest-yield shadow-infrastructure signal.
• ❌ Inventorying managed inference services while missing self-hosted models running on general-purpose compute, which carry no accelerator signature.
• ❌ Reconciling the registry against serving configuration annually, so a model swapped in production drifts from what was registered and tested for months.
• ❌ Reporting accelerator spend without utilisation, so an idle reserved fleet and a saturated one look identical in the scoreboard.
• ❌ Accepting untagged AI spend as a permanent condition rather than raising it as a blocking allocation gap with the FinOps practice.

Level 2, Comprehensive

Tier the AI infrastructure estate by exposure and criticality, calibrate hardening and monitoring intensity per tier, and bound accelerator and inference consumption.

Activities

A) Publish the infrastructure risk-tier rubric — Derive four tiers from five auditable dimensions. Network exposure is the strongest: an inference endpoint reachable from the public internet is Critical or High regardless of what it serves, because reachability is what converts every other weakness into an incident. Data sensitivity in flight elevates where regulated or customer-confidential content passes through the asset at inference or training. Criticality of the served workload elevates where the asset is on a revenue path, a customer-onboarding flow or an authentication decision. Concentration elevates where the asset is the single point of failure for a production AI capability, or where a provider, region or accelerator class carries a disproportionate share of production inference. Blast radius elevates where a compromise reaches beyond the asset, which applies particularly to orchestrators and control planes whose credentials command the fleet. Publish the rubric with its derivation rule and re-derive automatically when exposure or served-workload fields change, since infrastructure exposure changes faster than infrastructure inventory does.

B) Calibrate hardening intensity and consumption limits per tier — Publish a tier-treatment matrix covering, per tier: tenancy and isolation model from shared to dedicated; identity model, moving toward no standing human access on Critical-tier fleets with time-bound elevation; egress control from monitored to explicit allowlist; secrets management and rotation cadence; hardening baseline and its drift-detection cadence; monitoring detections and log completeness; testing obligations; and re-review triggers on material change. Add the consumption row that this domain needs more than any other: Critical- and High-tier inference endpoints carry per-tenant request rate limits and spend limits with a declared behaviour at the limit, being shed, queue or refuse, and never silently continue. Denial-of-wallet at the serving layer is invisible to availability monitoring by construction, so the limit is the control and its at-cap behaviour must be tested like any other boundary. Treat a capacity change, a model swap, a region change or a retention reduction on a Critical-tier asset as a material change triggering re-review, including when the motivation was cost.

C) Report per tier and manage concentration explicitly — Report the scoreboard per tier and archetype quarterly. Cover inventory and tier state, shadow-infrastructure ratio per tier, tagging and provenance conformance, hardening-baseline drift, per-tier service-level adherence, and the Assured Spend Ratio for accelerator and inference consumption per tier with full attribution targeted at Critical. Produce a standing concentration statement naming, for production inference, the share carried by each provider, each region and each accelerator class, together with the single points of failure that follow and an explicit decision to accept, mitigate or migrate each one, signed by the sponsor. Concentration in this domain is usually the result of accumulated convenience rather than a decision anyone made, and writing it down is what converts it into a governed position. Maintain a tier-movement log recording the triggering dimension for each change, and review it quarterly.

Outcome metrics

Consumption metrics — Inference cost per thousand requests at Critical tier against the endpoint's rolling median per per 1k inference requests · Spend-limit breach rate and at-cap behaviour conformance per tier per per endpoint-quarter · Accelerator cost per served model-hour by tier, with utilisation per per served model-hour

Success criteria

• A tier rubric is published and applied to 100% of production assets, re-derived automatically on exposure change.
• A tier-treatment matrix is published covering tenancy, identity, egress, secrets, hardening, monitoring and testing, and downstream Infrastructure L2 practices are calibrated to it.
• Every Critical-tier inference endpoint carries request and spend limits with a tested at-cap behaviour.
• Standing human access to Critical-tier fleets and control planes stands at zero, with time-bound elevation only.
• A sponsor-signed concentration statement names provider, region and accelerator-class exposure with an explicit accept, mitigate or migrate decision per single point of failure.

Assessment questions

1. Is every production AI infrastructure asset assigned a tier from a published rubric covering network exposure, data sensitivity in flight, served-workload criticality, concentration and blast radius, re-derived automatically when exposure changes?
2. Is there a published tier-treatment matrix covering tenancy and isolation, identity model, egress control, secrets, hardening baseline, monitoring and testing, with at least 95% of Critical-tier assets conforming to their hardening baseline and drift remediated inside the tier window?
3. Do all Critical-tier inference endpoints carry per-tenant request and spend limits with tested at-cap behaviour, and is there a sponsor-signed concentration statement recording an explicit accept, mitigate or migrate decision for each single point of failure?

Common pitfalls

• ❌ Tiering by data sensitivity alone while ignoring network exposure, which is the dimension that converts every other weakness into a reachable one.
• ❌ Configuring rate limits without spend limits, which stops request floods but not the expensive-generation variant of denial-of-wallet.
• ❌ Declaring a limit without testing its at-cap behaviour, so the first time anyone learns what happens at the cap is during the incident.
• ❌ Leaving concentration as an accumulated default rather than a signed position, so nobody ever decided to depend this heavily on one provider.
• ❌ Exempting cost-driven capacity and model changes from change governance because they arrived through a savings initiative rather than a feature request.

Level 3, Industry-Leading

Drive infrastructure inventory and tiering from live cloud and orchestrator signals, benchmark serving unit economics externally, and contribute infrastructure assurance patterns back.

Activities

A) Automate inventory, tier and provenance from live signals — Derive the estate picture from the control planes that already emit it: cloud asset and configuration events for creation, modification and deletion of AI infrastructure; orchestrator admission telemetry for workloads requesting accelerators; model-registry events for registration, promotion and deprecation; infrastructure-as-code state changes; network exposure changes from security-group and ingress events; and provider billing feeds for cost attribution and new-tag appearance. Implement tier derivation as a versioned, replayable rule set over the L2 dimensions, and wire tier changes to open downstream obligations automatically so that an endpoint newly exposed to the internet triggers its hardening, monitoring and testing obligations without waiting on a human to notice the security-group change. Publish a data-quality service-level objective covering tier correctness within 24 hours of an exposure change, which is tighter than the other domains warrant because infrastructure exposure changes in seconds and its consequences are immediate. Reserve human curation for novel archetypes, ambiguous ownership and conflicting dimensional inputs.

B) Benchmark serving unit economics and hardening posture externally — Publish a semi-annual brief comparing the program against named peers on at least five metrics: inventory coverage and freshness, shadow-infrastructure ratio, hardening-baseline conformance, inference cost per thousand requests by serving archetype, and accelerator cost per served model-hour with utilisation. Unit economics carry particular weight in this domain because serving configurations are optimised continuously and rarely re-reviewed; a cost per thousand requests well below the peer median may indicate efficient batching, or it may indicate that per-request authorization, logging and content filtering were removed to hit a savings target. Report cost and control coverage together so that the comparison cannot be read as a pure efficiency ranking. Draw from CNCF AI and ML working groups, OpenSSF AI supply-chain work, the FinOps Foundation AI scope, sector information-sharing bodies and ML-platform community forums. Name the peer cohort and record where the program trails.

C) Publish hardened patterns and compute the domain return — Contribute at least four substantive artifacts per year, weighted toward executable rather than descriptive ones: hardened infrastructure-as-code modules for inference serving, registry and vector-store deployment published through OpenSSF AI or CNCF; accelerator-fleet identity and isolation patterns; provenance-verification tooling for registry-to-serving reconciliation; observed infrastructure-layer techniques submitted to MITRE ATLAS; and FinOps Foundation AI-scope contributions on capacity governance and concentration risk. Count only what external bodies accept. Compute the domain return from the capacity ledger: contained denial-of-wallet events valued at burn rate multiplied by time to detect; reclaimed idle reservation, which is usually the largest single component and the easiest to verify; avoided commitment overage under governed capacity planning; and capacity reclaimed by retiring endpoints serving no traffic. Reconcile with Finance annually and publish alongside the Assurance Cost Ratio for the domain.

Outcome metrics

Consumption metrics — Inference cost per thousand requests benchmarked against the peer cohort, reported with control coverage per per 1k inference requests · Reclaimed idle accelerator reservation per per served model-hour · Contained denial-of-wallet value: burn rate multiplied by time to detect per per contained event

Success criteria

• Inventory, tier and provenance are event-derived with at least 85% auto-curation and a published latency objective of 24 hours on exposure change, met at the 99th percentile.
• Model provenance is verified continuously from registry to serving configuration for at least 99% of production endpoints.
• A semi-annual benchmarking brief covers at least five peer-comparable metrics reporting cost and control coverage together, with the peer cohort named and defended.
• At least four externally accepted contributions per year, weighted toward executable infrastructure-as-code artifacts.
• Domain return is computed from capacity-ledger arithmetic, reconciled with Finance, and published with the domain Assurance Cost Ratio.

Assessment questions

1. Do inventory, tier and provenance auto-update from cloud asset, orchestrator admission, registry, infrastructure-as-code and billing events against a published objective of tier correctness within 24 hours of an exposure change, with at least 85% of curation automated?
2. Does the semi-annual benchmarking brief compare at least five peer-comparable metrics including inference cost per thousand requests and accelerator cost per served model-hour, reporting control coverage alongside cost so efficiency cannot be confused with removed controls?
3. Does the program contribute at least four externally accepted infrastructure assurance artifacts per year weighted toward executable modules, and is the domain return computed from capacity-ledger arithmetic covering contained denial-of-wallet, reclaimed idle reservation and avoided overage?

Common pitfalls

• ❌ Automating inventory from cloud events while leaving network exposure changes on a slower path, which misses the dimension that matters most and moves fastest.
• ❌ Benchmarking serving cost without reporting control coverage, which rewards the peer who removed the most controls.
• ❌ Publishing infrastructure patterns as documentation rather than as executable modules, which peers cannot adopt and therefore do not.
• ❌ Computing return from reclaimed reservation without verifying that the reclaimed capacity was not simply re-reserved elsewhere under a different tag.
• ❌ Setting a 48-hour tier latency objective borrowed from another domain, when infrastructure exposure changes in seconds and its consequences are immediate.

Policy & Compliance (PC)

Objective: Publish the priority compliance map and the minimum policy stack governing the infrastructure AI/HAI systems run on, so every obligation reaches a named policy and a named owner.

A program at this level has written down which rules apply to the infrastructure AI/HAI systems run on and who answers for each. A one-page priority compliance map traces every in-scope obligation to a policy clause and an evidence artifact, so a regulator asking how a given article is addressed reaches one row, one policy and one piece of evidence rather than a research project. The AI Infrastructure Standard is published, acknowledged by the in-scope population, and specific enough to be violated, which is the test that separates a policy from a statement of values.

Level 1, Foundational

Publish the priority compliance map and the minimum policy stack governing the infrastructure AI/HAI systems run on, so every obligation reaches a named policy and a named owner.

Activities

A) Publish the priority compliance map — Produce a one-page priority compliance map covering the obligations that actually apply to the infrastructure AI/HAI systems run on: EU AI Act Art. 15 accuracy robustness and cybersecurity; ISO/IEC 27001 A.8; ISO/IEC 42001 8.1; NIST AI RMF MANAGE 2.2; GDPR Art. 32 security, plus any sector-specific rules in scope. Each row names the obligation, the specific clause of the internal policy that implements it, the accountable owner, the evidence artifact that demonstrates conformance, and the practice that produces that evidence. The map is one page by design. A compliance register of two hundred rows is a document nobody reads and nobody maintains, whereas a one-page map that traces the obligations a regulator is actually likely to ask about is a document that survives contact with an audit. Review it quarterly and whenever a new obligation enters scope, and treat an obligation with no named evidence artifact as an open finding rather than as a row to be filled in later.

B) Publish the AI Infrastructure Standard and the supporting policy stack — Publish an AI Infrastructure Standard covering permitted hosting regions and residency, accelerator provisioning authority, model registration and promotion requirements, inference endpoint exposure rules, and the hardening baseline each archetype must meet. Write each requirement so that conformance is observable: a rule stating that AI must be used responsibly cannot be assessed, whereas a rule naming the seven infrastructure archetypes, being inference endpoint, model registry, accelerator fleet, orchestrator and control plane, vector-store infrastructure, AI-specific CI/CD and feature store and stating what is permitted for each can be. Route the policy through the same approval and review cycle as the enterprise security policy stack rather than creating a parallel governance path, and set an explicit review interval of at most 12 months. Publish a short summary aimed at the population that must follow it, because a policy that requires reading twenty pages to answer a five-second question will be guessed at instead. Establish an exception process with a named approver, a required rationale, a mandatory expiry date and a register, since undocumented exceptions are how policy stacks quietly become fiction.

C) Attest the population and baseline conformance — Run an attestation campaign across the in-scope population, delivered through the existing human resources or learning system so that completion is recorded where compliance evidence already lives. Track coverage as a percentage of the in-scope population rather than as a count of completions, because the denominator is what an auditor asks for. Baseline three conformance measures in the first quarter: the share of obligations on the map carrying a named evidence artifact, the share of the population with a current acknowledgement, and the count of active exceptions with their age distribution. Reconcile the map against infrastructure-as-code definitions and cloud configuration state to confirm that what the policy requires is what the environment actually does, and record every divergence as a finding with an owner rather than as a note. Report all three to the executive sponsor quarterly.

Outcome metrics

Success criteria

• A one-page priority compliance map is published, tracing every in-scope obligation to a policy clause, an owner and an evidence artifact.
• The AI Infrastructure Standard is published with an explicit review interval of at most 12 months and written so that violations are identifiable.
• At least 95% of the in-scope population holds a current acknowledgement.
• An exception process operates with named approvers, mandatory expiry dates and a reviewed register carrying no expired entries.
• Policy requirements are reconciled against infrastructure-as-code definitions and cloud configuration state, with every divergence raised as a finding with an owner.

Assessment questions

1. Is there a published one-page priority compliance map tracing every in-scope obligation governing the infrastructure AI/HAI systems run on to a specific policy clause, an accountable owner, an evidence artifact and the practice that produces it?
2. Is the AI Infrastructure Standard published with requirements written so conformance is observable, routed through the enterprise policy review cycle, carrying a review interval of at most 12 months, and acknowledged by at least 95% of the in-scope population?
3. Does an exception process operate with named approvers, mandatory rationale and expiry dates and a register carrying no expired entries, and are policy requirements reconciled against infrastructure-as-code definitions and cloud configuration state with divergences raised as findings?

Common pitfalls

• ❌ Building a compliance register of hundreds of rows instead of a one-page map, which nobody reads, nobody maintains and no auditor is helped by.
• ❌ Writing policy requirements that cannot be violated, such as requiring AI to be used responsibly, which makes conformance unassessable.
• ❌ Creating a parallel AI policy governance path separate from the enterprise stack, which doubles the review burden and halves the authority.
• ❌ Granting exceptions without expiry dates, after which the exception register becomes the real policy.
• ❌ Publishing the policy without reconciling it against what the environment actually does, so policy and practice diverge from day one.

Level 2, Comprehensive

Calibrate policy depth and evidence requirements to the risk tier of each infrastructure asset, and assemble tier-specific evidence packs that can be produced on demand.

Activities

A) Calibrate policy requirements per tier — Extend the L1 policy stack with tier-differentiated requirements keyed to the SM L2 rubric. For each requirement, state what Critical, High, Medium and Low each owe: which controls are mandatory versus recommended, what review cadence applies, what approval authority an exception requires, and what evidence must exist. A Critical-tier the infrastructure AI/HAI systems run on might owe a documented conformance assessment, an annual independent review and a named accountable executive, where a Low-tier one owes an intake record and an annual self-attestation. Publishing the difference is what prevents the two failure modes that follow from uniform policy: either every asset carries Critical-tier burden, which makes the policy unenforceable and drives teams around it, or every asset carries Low-tier burden, which leaves the Critical ones under-governed. Route tier-calibrated requirements through the same approval cycle as the base policy. In this domain tier calibration turns mainly on exposure and residency: an internet-reachable inference endpoint in a third country carries obligations that an internal batch-scoring host does not, and the difference is a property of the deployment rather than the model.

B) Build bidirectional control mappings and tier-specific evidence packs — Map each obligation on the priority compliance map to the specific controls that satisfy it, and map each control back to the obligations that depend on it. The reverse direction is the one usually missing and the one that matters operationally: when a control fails or is deferred, the programme needs to state immediately which obligations are now unevidenced, rather than discovering it during the next audit. Assemble an evidence pack per tier specifying exactly which artifacts constitute conformance, drawn from infrastructure-as-code definitions and cloud configuration state and from the outputs of TA, SR, DR, IR, ST and ML. Automate collection where the artifact already exists in a system of record, and set a service level for producing a complete pack on request, targeting 10 business days at Critical tier. An evidence pack assembled only when asked for is a pack that will be incomplete the one time it matters. The evidence pack for infrastructure draws on infrastructure-as-code definitions, cloud configuration state and change records, all of which are machine-readable and therefore the cheapest evidence in the model to automate.

C) Operate regulatory change management and tiered exceptions — Establish a defined path from a regulatory change to the assets it affects. Assign a named owner to monitor the in-scope obligation sources, assess each change for applicability within 30 days of publication, identify the affected assets by tier, and open remediation work items with owners and dates. Without this, regulatory change arrives through whoever happens to read an update, which is neither reliable nor auditable. Calibrate the exception process by tier: a Critical-tier exception requires executive approval, a shorter maximum duration and quarterly review, where a Low-tier exception can be approved by the practice lead with an annual review. Report exception volume and age by tier to the sponsor, and treat a rising Critical-tier exception count as a signal that the policy is mis-calibrated rather than that teams are non-compliant. Regulatory change in this domain most often arrives as accuracy, robustness and cybersecurity guidance under Art. 15, which lands on the serving and hardening configuration rather than on the application.

Outcome metrics

Success criteria

• Tier-calibrated policy requirements are published, stating what each tier owes in controls, cadence, approval authority and evidence.
• Evidence packs exist per tier and are refreshed on cadence, with a Critical-tier production service level of 10 business days met.
• Bidirectional control mappings are complete, so a failing control names the obligations it jeopardises.
• A named owner assesses every regulatory change for applicability within 30 days and opens remediation work items by tier.
• Exceptions are tier-calibrated in approval authority, maximum duration and review cadence, with Critical-tier volume and age reported to the sponsor.

Assessment questions

1. Are policy requirements calibrated by the SM L2 risk tier, stating for each tier what controls are mandatory, what review cadence applies, what approval authority an exception requires and what evidence must exist? Evidence this in the Infrastructure domain against the AI Infrastructure Standard and the residency position for one production endpoint.
2. Do bidirectional control mappings exist so that a failing or deferred control immediately names the obligations it leaves unevidenced, and do tier-specific evidence packs get refreshed on cadence rather than assembled on request? For Infrastructure, the assessor should be shown the AI Infrastructure Standard and the residency position for one production endpoint.
3. Does a named owner assess every regulatory change for applicability within 30 days of publication and open tier-prioritised remediation work items, and are exceptions tier-calibrated in approval authority, duration and review cadence? In Infrastructure, ask to see the AI Infrastructure Standard and the residency position for one production endpoint.

Common pitfalls

• ❌ Applying one policy depth to every asset, which either over-burdens Low-tier work until teams route around the policy, or under-governs the Critical-tier assets that justify the programme.
• ❌ Mapping obligations to controls in one direction only, so a deferred control's regulatory consequence is discovered at audit rather than at the deferral decision.
• ❌ Assembling evidence packs only when requested, which guarantees the pack is incomplete on the one occasion it is needed urgently.
• ❌ Leaving regulatory monitoring to whoever happens to read an update, which is neither reliable nor demonstrable to an auditor.
• ❌ Treating a rising Critical-tier exception count as non-compliance by teams rather than as evidence that the policy is mis-calibrated.

Level 3, Industry-Leading

Generate compliance evidence continuously from the operating environment, prove conformance by assertion rather than assembly, and engage in the regulatory process for the infrastructure AI/HAI systems run on.

Activities

A) Generate conformance evidence continuously — Instrument the environment so that each policy requirement has a continuously evaluated conformance assertion rather than a periodically collected artifact. Draw the signals from infrastructure-as-code definitions and cloud configuration state and from the telemetry the other practices already emit, and express each requirement as a check that returns conformant, non-conformant or not-evaluable. The third state matters and is usually omitted: a requirement that cannot be evaluated is a gap in the evidence chain, not a pass, and collapsing it into conformant is how continuous compliance dashboards become reassuring and wrong. Publish a service level covering detection of a conformance drift within 5 business days. Reserve manual attestation for requirements that are genuinely not machine-evaluable, list them explicitly, and review that list quarterly with the aim of shrinking it, since the list is the honest measure of how far the automation actually reaches.

B) Prove conformance on demand and shrink audit preparation — Convert the evidence pack from an assembled document into a generated view over the continuous assertions, so that producing it is a query rather than a project. Target production within one business day at every tier, and run unannounced internal exercises at least twice a year to verify the claim, because an evidence pipeline that has never been exercised under time pressure is an assumption. Track audit preparation effort explicitly in hours and drive it toward zero: the measure of this level is not that audits pass, which they did before, but that they cost almost nothing to prepare for. Where an external audit or regulatory enquiry does raise a finding, trace it back to the assertion that should have caught it and either fix the assertion or record why the requirement is not machine-evaluable.

C) Engage in the regulatory and standards process — Participate in the process that produces the obligations rather than only consuming its output. Submit implementation experience to consultations on EU AI Act implementing acts and successor guidance, contribute to the standards bodies developing conformance methods including ISO/IEC 42001 and the relevant national standards organisations, and share anonymised implementation patterns through sector bodies and practitioner communities. Implementation experience is the input regulators most consistently lack and most consistently ask for, which makes it the contribution with the highest leverage and the lowest cost to produce, since the programme generates it as a by-product of operating. Target at least four substantive submissions per year, count only those accepted or published, and have legal review every submission before it leaves the organisation.

Outcome metrics

Success criteria

• At least 85% of policy requirements carry a continuously evaluated conformance assertion, with the non-evaluable remainder listed explicitly rather than counted as conformant.
• Conformance drift is detected within 5 business days at the 90th percentile.
• A complete evidence pack is produced within one business day at any tier, verified by unannounced exercise twice a year.
• Audit preparation effort is tracked in hours and falling year over year.
• At least four accepted regulatory or standards contributions per year, each legally reviewed before submission.

Assessment questions

1. Do at least 85% of policy requirements carry a continuously evaluated conformance assertion returning conformant, non-conformant or not-evaluable, with the non-evaluable set listed explicitly and reviewed quarterly for reduction rather than collapsed into conformant? Evidence this in the Infrastructure domain against the AI Infrastructure Standard and the residency position for one production endpoint.
2. Is a complete evidence pack produced within one business day at any tier, verified by unannounced exercises at least twice a year, and is audit preparation effort tracked in hours and falling year over year? For Infrastructure, the assessor should be shown the AI Infrastructure Standard and the residency position for one production endpoint.
3. Does the organisation make at least four accepted regulatory or standards contributions per year drawn from its own implementation experience, each reviewed by legal counsel before submission? In Infrastructure, ask to see the AI Infrastructure Standard and the residency position for one production endpoint.

Common pitfalls

• ❌ Collapsing the not-evaluable state into conformant, which produces a compliance dashboard that is reassuring and wrong.
• ❌ Claiming one-day evidence production without ever exercising it unannounced, which makes the claim an assumption rather than a capability.
• ❌ Automating the assertions that were easy and leaving the non-evaluable list unpublished, so the automation's real reach is unknown.
• ❌ Measuring audit success rather than audit preparation cost, when the latter is what this level actually changes.
• ❌ Submitting to consultations without legal review, which can commit the organisation to positions it has not agreed internally.

Education & Guidance (EG)

Objective: Give the people who provision and operate AI infrastructure the role-based AI security literacy and the practical guidance they need to make safe decisions without asking, in the Infrastructure domain.

A program at this level has stopped relying on general security awareness to cover AI-specific risk. Role-based training reaches platform and site-reliability engineers, cloud architects, ML platform engineers, and the security architects reviewing their baselines, and it teaches the failure modes those roles actually encounter rather than a survey of AI risk in general. A guidance library sits where the work happens, so the answer to a routine question is reachable in under a minute rather than requiring a security consultation. Competence is assessed rather than assumed, because completion rates measure attendance and say nothing about whether anyone can now do the thing.

Level 1, Foundational

Give the people who provision and operate AI infrastructure the role-based AI security literacy and the practical guidance they need to make safe decisions without asking, in the Infrastructure domain.

Activities

A) Define roles and build role-based curricula — Identify the roles in scope, being platform and site-reliability engineers, cloud architects, ML platform engineers, and the security architects reviewing their baselines, and build a short curriculum per role rather than one course for everyone. A single general course is the most common failure in this practice: it is long enough to be resented, general enough to be forgettable, and calibrated to no one. Each curriculum covers the AI failure modes that role actually meets, which in this domain means unsanctioned model hosting, standing credentials on accelerator fleets, inference endpoints exposed beyond their intended boundary, and denial-of-wallet at the serving layer, together with the specific controls that role is expected to apply and the escalation path when something looks wrong. Keep each curriculum under 90 minutes, because attention is the binding constraint and a curriculum nobody finishes teaches nothing. Set a 90-day competence deadline for new joiners in scope, and deliver through the existing learning system so that completion and assessment records land where compliance evidence already lives.

B) Assess competence rather than completion — Build an assessment per role that tests whether the learner can apply the material, not whether they sat through it. Include a hands-on exercise where the learner exposes an inference endpoint, observes the absence of any availability signal under a wallet-drain load, and then applies per-tenant limits, because a scenario the learner works through is the only assessment format that distinguishes recognition from capability, and recognition is what multiple-choice tests measure. Set a pass threshold, require a retake on failure, and record the pass rate per role as the practice's headline outcome metric. Publish item-level results to the curriculum owner so that a question the whole cohort fails is treated as a teaching failure rather than a learner failure, which is the usual cause. Re-assess at least annually and whenever the threat picture or the sanctioned toolset changes materially.

C) Publish the guidance library where the work happens — Publish a guidance library covering hardened infrastructure-as-code modules for serving, registry and vector-store deployment, with egress allowlisting and no-standing-access identity patterns. The binding constraint is reachability, not completeness: guidance that requires leaving the working context to find will be guessed around, so place it where the work is, in the repository for engineers, in the workflow tool for process owners, in the service catalogue for requesters. Every guidance item names an owner and a review date, and stale guidance is withdrawn rather than left standing, because out-of-date guidance is worse than none and is trusted for longer. Publish a named route for questions the library does not answer, staffed with a response service level, and mine those questions monthly as the highest-quality signal available about where the guidance is missing or wrong.

Outcome metrics

Success criteria

• Every in-scope role among platform and site-reliability engineers, cloud architects, ML platform engineers, and the security architects reviewing their baselines has a curriculum under 90 minutes covering the failure modes that role actually encounters.
• Competence is assessed by applied scenario rather than recall, with an 80% or better pass rate per role and per-role results published.
• New joiners in scope reach baseline literacy within 90 days, enrolled automatically from the joiner process.
• A guidance library sits where the work happens, with every item carrying an owner and a review date and stale items withdrawn.
• A named question route operates with a published response service level and is mined monthly for guidance gaps.

Assessment questions

1. Does every in-scope role have its own curriculum under 90 minutes covering the AI failure modes that role actually encounters in the Infrastructure domain, delivered through the existing learning system with a 90-day deadline for new joiners?
2. Is competence assessed by applied scenario rather than recall, with a published pass threshold, an 80% or better pass rate per role, and item-level results reviewed by the curriculum owner so cohort-wide failures are treated as teaching defects? For Infrastructure, the assessor should be shown the platform-engineer curriculum and the wallet-drain exercise.
3. Does a guidance library sit where the work happens with every item carrying an owner and a review date, are stale items withdrawn rather than retained, and is there a named question route with a response service level that is mined monthly for gaps? In Infrastructure, ask to see the platform-engineer curriculum and the wallet-drain exercise.

Common pitfalls

• ❌ Building one general AI security course for everyone, which is too long to finish, too general to apply, and calibrated to no actual role.
• ❌ Measuring completion rather than competence, which tells the programme who attended and nothing about what anyone can now do.
• ❌ Publishing guidance in a wiki nobody opens, when reachability from the working context is the whole of the control.
• ❌ Leaving stale guidance standing, which is worse than no guidance because it is trusted for longer.
• ❌ Treating cohort-wide assessment failures as a learner problem rather than as evidence the material taught the wrong thing.

Level 2, Comprehensive

Scale Infrastructure AI security literacy through tier-calibrated depth, a champions network embedded in the teams, and exercises drawn from the organisation's own incidents.

Activities

A) Calibrate curriculum depth to asset tier — Extend the L1 curricula with tier-calibrated depth keyed to the SM L2 rubric. Practitioners working on Critical- and High-tier assets take an extended curriculum with hands-on laboratory work and a higher pass threshold; those on Medium and Low take the baseline. Publish the mapping from asset tier to required curriculum so that assignment is derived rather than negotiated, and wire enrolment to the asset inventory so that a practitioner joining a Critical-tier team is enrolled automatically rather than when someone remembers. Set a currency requirement: Critical-tier practitioners re-assess annually, others every two years. Report pass rates and currency by tier rather than in aggregate, because an organisation-wide 85% pass rate is consistent with the Critical-tier cohort being the weakest, and aggregate reporting actively conceals exactly the case that matters. In this domain the extended curriculum has learners expose an inference endpoint, drive a wallet-drain load against it, and observe that every availability dashboard stays green throughout, because that observation changes how platform engineers think about limits far more effectively than a policy does.

B) Build and resource a champions network — Establish a security champion inside each team working on Critical- and High-tier assets. The champion is not a security professional and should not be treated as one: the role is to be the person who recognises when something needs the central function, to answer the routine questions locally, and to carry context the central team cannot have. Resource the role explicitly with allocated time, typically half a day a week, agreed with the engineering or business management chain rather than assumed as goodwill, because an unresourced champion network decays within two quarters and its decay is invisible until it is total. Give champions a deeper curriculum, a private channel to the central team, early access to threat and guidance updates, and visible recognition in performance processes. Meet the network monthly and mine it for what the central function cannot see. Infrastructure champions sit inside the platform and site-reliability teams, and their distinguishing value is knowing which workload just requested accelerators and why, which cost allocation reveals only a billing cycle later.

C) Run exercises from the organisation's own incidents — Convert incidents and near-misses from the issue-management practice into teaching material on a defined cadence, targeting one new exercise per quarter. An exercise built from something that actually happened in this organisation carries an authority that no generic example achieves, and it converts the incident from a cost into an asset. Anonymise where necessary but keep the specifics that make it recognisable, since the recognisable detail is the part that is remembered. Run the exercises with the champions network first, then let champions run them with their teams, which both scales delivery and tests whether the champion can teach the material rather than only pass an assessment. Track which failure modes recur across exercises and feed that back to the threat assessment and guidance practices as evidence of where understanding is genuinely weak. Infrastructure exercises are built from real exposure incidents and shadow-workload discoveries, where the recognisable detail is usually a security group or namespace the team recognises.

Outcome metrics

Success criteria

• Curriculum depth is calibrated to asset tier with enrolment derived from the inventory, and pass rates are reported per tier rather than in aggregate.
• Every team working on Critical-tier assets has a trained champion with management-agreed allocated time.
• At least four exercises per year are built from the organisation's own incidents and near-misses.
• Champions resolve at least 60% of routine questions without central escalation.
• Recurring failure modes across exercises are fed back to threat assessment and guidance as evidence of genuine weakness.

Assessment questions

1. Is curriculum depth calibrated to the SM L2 asset tier with enrolment derived automatically from the inventory, and are pass rates and currency reported per tier rather than in an aggregate that could conceal a weak Critical-tier cohort? Evidence this in the Infrastructure domain against the platform-engineer curriculum and the wallet-drain exercise.
2. Does every team working on Critical-tier assets have a trained champion with explicitly allocated time agreed with the management chain, and do champions resolve at least 60% of routine questions without central escalation? For Infrastructure, the assessor should be shown the platform-engineer curriculum and the wallet-drain exercise.
3. Are at least four exercises per year built from the organisation's own incidents and near-misses, run first with champions and then by champions with their teams, with recurring failure modes fed back to threat assessment and guidance? In Infrastructure, ask to see the platform-engineer curriculum and the wallet-drain exercise.

Common pitfalls

• ❌ Reporting one organisation-wide pass rate, which is consistent with the Critical-tier cohort being the weakest and conceals precisely that case.
• ❌ Appointing champions without allocated time, after which the network decays within two quarters and its decay is invisible until total.
• ❌ Treating champions as junior security staff rather than as embedded context-holders, which drives away the people best suited to the role.
• ❌ Using generic exercise material when the organisation's own incidents are available, discarding the authority that recognisable detail carries.
• ❌ Anonymising exercises so thoroughly that the recognisable specifics are gone, which removes the part learners actually remember.

Level 3, Industry-Leading

Make Infrastructure AI security learning continuous and evidence-driven, measure whether it changes behaviour rather than knowledge, and contribute curricula back to the field.

Activities

A) Drive curriculum and guidance from observed signal — Close the loop between what the organisation observes and what it teaches. Feed incident root causes, design and implementation review findings, security test failures and monitoring detections into a quarterly curriculum review, and treat any failure mode appearing three times as a mandatory curriculum change rather than a discussion item. Instrument the guidance library so that search terms returning nothing, and the questions reaching the named route, are captured as gap signals, because what practitioners look for and fail to find is the highest-quality guidance-gap data available and it is free. Publish a service level covering the interval from an observed recurring failure mode to updated guidance, targeting 30 days. Retire material that the signal shows is no longer relevant, since a curriculum that only grows becomes one nobody completes. In this domain the extended curriculum has learners expose an inference endpoint, drive a wallet-drain load against it, and observe that every availability dashboard stays green throughout, because that observation changes how platform engineers think about limits far more effectively than a policy does.

B) Measure behaviour change rather than knowledge — Move the headline measure from assessment scores to behaviour in the environment. Compare teams and cohorts before and after training on measures the other practices already produce: the rate of findings by category in implementation review, the reuse rate of sanctioned patterns, the proportion of designs arriving at review already conformant, the rate of self-reported near-misses, which should rise as psychological safety improves and is one of the few metrics where an increase is good news. Accept that attribution is imperfect and say so, rather than claiming a causal link the data cannot carry; the honest claim is that behaviour moved in the trained cohort and did not in the untrained one, over a stated period. Report behaviour measures alongside assessment results so that a high pass rate with unchanged behaviour is visible as the failure it represents. Infrastructure champions sit inside the platform and site-reliability teams, and their distinguishing value is knowing which workload just requested accelerators and why, which cost allocation reveals only a billing cycle later.

C) Contribute curricula and measure capability in progression — Contribute at least four substantive artifacts per year to the field, weighted toward teaching material: role-based AI security curricula, applied exercise and laboratory scenarios, anonymised incident case studies that other organisations can teach from, and guidance patterns. Good AI security teaching material is the field's scarcest resource, far scarcer than frameworks or tooling, which makes curriculum contribution unusually high-leverage and unusually likely to be adopted. Work through OWASP education projects, sector bodies, professional associations and academic partnerships. Separately, establish AI security capability as a recognised factor in performance and progression for in-scope roles, agreed with human resources, because a capability the organisation says matters but never recognises is one practitioners correctly deprioritise. Infrastructure exercises are built from real exposure incidents and shadow-workload discoveries, where the recognisable detail is usually a security group or namespace the team recognises.

Outcome metrics

Success criteria

• Curricula and guidance update from observed incidents, findings and telemetry, with recurring failure modes reaching updated guidance within 30 days.
• Guidance gaps are detected from search misses and unanswered questions rather than from periodic review alone.
• At least three behaviour measures are tracked pre and post training, reported alongside assessment results with attribution limits stated.
• At least four externally accepted curriculum or exercise contributions per year.
• AI security capability is a recognised factor in performance and progression for every in-scope role, agreed with human resources.

Assessment questions

1. Do curricula and guidance update from incident root causes, review findings, test failures and detections, with any failure mode appearing three times triggering a mandatory curriculum change within 30 days, and are guidance gaps detected from search misses and unanswered questions? Evidence this in the Infrastructure domain against the platform-engineer curriculum and the wallet-drain exercise.
2. Does the programme measure behaviour change in the environment using at least three downstream measures tracked pre and post training, reported alongside assessment results with attribution limits stated rather than a causal claim the data cannot carry? For Infrastructure, the assessor should be shown the platform-engineer curriculum and the wallet-drain exercise.
3. Does the organisation contribute at least four externally accepted curriculum or exercise artifacts per year, and is AI security capability a recognised factor in performance and progression for every in-scope role? In Infrastructure, ask to see the platform-engineer curriculum and the wallet-drain exercise.

Common pitfalls

• ❌ Reviewing curricula annually while the threat picture moves quarterly, so the material lags by a planning cycle permanently.
• ❌ Adding material without ever retiring any, producing a curriculum too long to complete and therefore completed by nobody.
• ❌ Claiming training caused a behaviour improvement when the data supports only correlation, which discredits the honest measures alongside it.
• ❌ Ignoring search misses and unanswered questions, which are the cheapest and highest-quality guidance-gap signal available.
• ❌ Declaring AI security capability important while never recognising it in progression, which practitioners correctly read as a statement of priority.

Business Function: Building

Threat Assessment (TA)

Objective: Publish an archetype-level threat library for the Infrastructure domain, tagged to the HAI TTPs and MITRE ATLAS, so threat modelling starts from a reusable baseline rather than a blank page.

A program at this level has written down what can go wrong with each infrastructure archetype, once, so that no team has to rediscover it. The library covers inference endpoint, model registry, accelerator fleet, orchestrator and control plane, vector-store infrastructure, AI-specific CI/CD and feature store, and every entry carries a HAI TTP tag, an ATLAS technique where one applies, an OWASP LLM reference and the practice expected to address it. This is what makes threat modelling tractable at scale: a team adopting a known archetype inherits its threat model and spends its effort on what is genuinely novel about their case, rather than re-deriving prompt injection from first principles every time.

Level 1, Foundational

Publish an archetype-level threat library for the Infrastructure domain, tagged to the HAI TTPs and MITRE ATLAS, so threat modelling starts from a reusable baseline rather than a blank page.

Activities

A) Build the archetype threat library — Author one threat snapshot per archetype covering inference endpoint, model registry, accelerator fleet, orchestrator and control plane, vector-store infrastructure, AI-specific CI/CD and feature store. For each archetype, identify the entities and trust boundaries first, being the registry, the serving endpoint, the orchestrator control plane, the accelerator scheduler and the CI/CD pipeline that promotes models, because a threat list without a boundary model is a list of worries rather than an analysis. Then enumerate the threats that follow from those boundaries, which in this domain centre on model artifacts obtained from a registry or serving host, inference endpoints reachable beyond their intended boundary, control-plane credentials commanding the whole fleet, and denial-of-wallet that degrades no availability signal. Keep each snapshot to a length a team will actually read, targeting two pages, and write each threat as a concrete failure with a named precondition rather than as a risk category. Derive the snapshots from infrastructure-as-code definitions, cloud configuration state and network topology and validate each with one practitioner who works on that archetype daily, because a threat library authored entirely by security will describe the system security imagines rather than the one that exists.

B) Tag every threat to the shared vocabulary — Tag each threat entry with the HAI TTP it expresses, being excessive agency, agent goal hijack, tool misuse or rogue agents; the MITRE ATLAS technique where a real one applies; the OWASP LLM 2025 reference; and the compliance obligation it bears on where relevant. Use only technique identifiers that genuinely exist, because a plausible but invented identifier is worse than none, it survives review and misleads everyone downstream. Where no ATLAS technique fits, record that explicitly rather than forcing the nearest match, and treat the gap as a candidate contribution at L3. The tagging is what makes the library composable with everything else: it lets monitoring ask which threats have detections, testing ask which have tests, and requirements ask which have controls, without any of them re-reading the prose.

C) Trace every threat to a treating practice and publish the gaps — For every threat in the library, name the practice expected to address it and the specific control, requirement, pattern, test or detection that does so. Then publish the gaps, which is the activity teams most often skip and the one that gives the library its value: a threat with no named treatment is an accepted risk whether or not anyone decided to accept it, and making that visible is the point of the exercise. Report the treatment-coverage rate to the sponsor quarterly and treat an untreated threat at a high-risk archetype as an open finding with an owner rather than as a line in a register. Establish a 30-day service level for producing a threat snapshot when a new archetype enters the inventory, and review the whole library at least quarterly against external threat intelligence.

Outcome metrics

Success criteria

• A threat snapshot exists for at least 90% of inventoried infrastructure archetypes, each opening with entities and trust boundaries rather than a threat list.
• Every threat entry carries a HAI TTP tag and, where applicable, a genuine ATLAS technique identifier, with no invented references.
• At least 85% of threats trace to a named treating practice and specific control, with the untreated remainder published as gaps rather than omitted.
• Untreated threats at high-risk archetypes carry an owner and a remediation date.
• A 30-day snapshot service level operates for new archetypes, triggered from the inventory.

Assessment questions

1. Does a published threat snapshot exist for at least 90% of inventoried infrastructure archetypes, each establishing entities and trust boundaries before enumerating threats, validated with a practitioner who works on that archetype?
2. Does every threat entry carry a HAI TTP tag, a genuine MITRE ATLAS technique where one applies with gaps recorded rather than forced, an OWASP LLM reference and the relevant compliance obligation? For Infrastructure, the assessor should be shown the threat snapshot for the inference endpoint, with its reachability graph.
3. Are at least 85% of threats traced to a named treating practice and specific control, with untreated threats published as gaps and those at high-risk archetypes carrying an owner and remediation date? In Infrastructure, ask to see the threat snapshot for the inference endpoint, with its reachability graph.

Common pitfalls

• ❌ Writing a threat list without first drawing entities and trust boundaries, which produces a list of worries rather than an analysis anyone can act on.
• ❌ Inventing plausible ATLAS technique identifiers where none fit, which survives review and misleads every downstream practice that trusts the tag.
• ❌ Authoring the library entirely within security, which describes the system security imagines rather than the one practitioners operate.
• ❌ Producing the library and never publishing the treatment gaps, which leaves threats accepted by default without anyone deciding to accept them.
• ❌ Writing snapshots so long that teams do not read them, after which the library is maintained for an audience of nobody.

Level 2, Comprehensive

Model threats per asset at a depth calibrated to its risk tier, refresh models on material change, and keep the Infrastructure threat picture current against real adversary behaviour.

Activities

A) Calibrate threat-model depth to asset tier — Publish what each tier receives. Critical: a per-asset threat model, an adversarial overlay walking the relevant ATLAS tactics end to end, an explicit abuse-case set, and a named architect reviewer. High: the archetype model plus documented per-asset deltas and an ATLAS tactic walk. Medium: the archetype model with deltas recorded. Low: the archetype model unchanged. Publishing this is what stops the two failure modes of uniform depth, namely modelling everything shallowly so the Critical assets are under-analysed, or modelling everything deeply so the practice becomes a bottleneck and teams route around it. Set turnaround service levels per tier and report adherence, because a threat-modelling practice that cannot say how long it takes will be worked around by teams under delivery pressure whatever its analytical quality. Per-asset modelling in this domain means walking the reachability graph: what can reach the endpoint, what the endpoint can reach, what credentials exist in its runtime, and what an attacker holding the control plane could do to every asset it commands, because blast radius in this domain is the dominant variable and is invisible in an archetype model.

B) Refresh on material change rather than on the calendar — Define material change per domain and wire the trigger to the signals that already exist. In this domain a material change includes a new capability or interface on the asset, a change to the data classes it handles, a model or provider change, a change in exposure or the population reached, and a tier increase from the SM rubric. Each triggers a re-model inside the tier's window, being 14 days at Critical and 30 at High. Annual review remains as a floor, not as the mechanism; the calendar is the least informative trigger available and relying on it means an asset that changed materially in month two carries a stale model for ten months. Record every refresh with what changed and what it changed in the model, so that a reviewer can see whether the re-model was substantive or a date stamp. In this domain the material change that most often invalidates a model is a security-group or ingress change, which happens in seconds, frequently during an incident, and is the change least likely to be accompanied by a threat-model review.

C) Feed external intelligence in and push threat output downstream — Consume external threat intelligence on a defined cadence, drawing from MITRE ATLAS updates, OWASP LLM and Agentic project output, vendor and provider advisories, sector information-sharing bodies and published incident analyses. Assess each item for applicability to the archetypes in the library within 30 days, and record the assessment even when the answer is not applicable, because the record is what demonstrates the cadence is real. Push threat-model output downstream mechanically rather than by expectation: each Critical- and High-tier threat generates a requirement in the SR pack, a test case in the ST battery and a detection candidate in the ML set, tracked to closure. A threat model whose output is not visible in what gets tested and detected has produced a document, and the practice is then measured on documents. Infrastructure threat output lands most usefully in the egress allowlist, the exposure tests and the per-tenant quota configuration, which are the controls that actually encode the reachability decisions.

Outcome metrics

Success criteria

• Threat-model depth per tier is published with turnaround service levels, and adherence is at or above 90% per tier.
• Every Critical-tier asset has a per-asset model with an adversarial ATLAS overlay and a named architect reviewer.
• Material change triggers a re-model inside the tier window, with refresh records stating what actually changed.
• External intelligence is assessed for applicability within 30 days, with negative assessments recorded.
• At least 90% of Critical- and High-tier threats generate a tracked requirement, test case and detection candidate.

Assessment questions

1. Is threat-model depth calibrated by asset tier with published turnaround service levels, such that Critical-tier assets receive a per-asset model with an adversarial ATLAS overlay and a named architect reviewer while Low-tier assets inherit the archetype snapshot? Evidence this in the Infrastructure domain against the threat snapshot for the inference endpoint, with its reachability graph.
2. Does material change, including a new capability, a data-class change, a model or provider change, an exposure change or a tier increase, trigger a re-model within 14 days at Critical and 30 at High, with refresh records stating what actually changed rather than only a date? For Infrastructure, the assessor should be shown the threat snapshot for the inference endpoint, with its reachability graph.
3. Is external threat intelligence assessed for applicability within 30 days with negative assessments recorded, and do at least 90% of Critical- and High-tier threats generate a tracked requirement, test case and detection candidate? In Infrastructure, ask to see the threat snapshot for the inference endpoint, with its reachability graph.

Common pitfalls

• ❌ Relying on annual review as the refresh mechanism, which leaves an asset that changed in month two carrying a stale model for ten months.
• ❌ Modelling every asset at the same depth, which either under-analyses the Critical assets or turns the practice into a bottleneck teams route around.
• ❌ Recording refreshes as date stamps without stating what changed, which makes it impossible to tell a substantive re-model from a compliance gesture.
• ❌ Consuming threat intelligence without recording negative assessments, so there is no evidence the cadence is actually operating.
• ❌ Ending the threat model at the document, so nothing in testing or detection changes and the practice is judged on paperwork.

Level 3, Industry-Leading

Drive Infrastructure threat modelling from live signal and adversary emulation, validate models against what actually happens, and contribute observed techniques back to ATLAS and the community.

Activities

A) Validate models through adversary emulation — Run adversary emulation against Critical-tier assets at least semi-annually, scoped from the threat model and executed by a team that did not author it. Measure the two directions of error explicitly: threats the model predicted that emulation could not demonstrate, and behaviours emulation found that the model did not anticipate. The second is the more valuable and the more uncomfortable, and a practice that only reports confirmations is not learning. Record for every threat in the library whether it has been demonstrated in this environment, demonstrated elsewhere, or remains theoretical, and report the distribution, since a library where most entries are theoretical is a library built from reading rather than from testing. Feed every unanticipated finding into the archetype library, not only into the asset's model, so that one exercise improves every asset sharing that archetype. Per-asset modelling in this domain means walking the reachability graph: what can reach the endpoint, what the endpoint can reach, what credentials exist in its runtime, and what an attacker holding the control plane could do to every asset it commands, because blast radius in this domain is the dominant variable and is invisible in an archetype model.

B) Update models from live signal continuously — Close the loop from operations back to the models. Feed detection telemetry, incident root causes, security test failures and near-misses into the library continuously rather than at review time, and treat any observed technique not represented in the library as a defect in the library with an owner and a date. Publish a service level covering the interval from an observed technique to an updated model, targeting 30 days. Track the share of incidents whose root cause was already present in the library as an untreated threat, which is the single most informative metric this practice produces: it separates a failure to foresee, which is hard, from a failure to act on what was already written down, which is not, and the second is far more common than most programmes admit. In this domain the material change that most often invalidates a model is a security-group or ingress change, which happens in seconds, frequently during an incident, and is the change least likely to be accompanied by a threat-model review.

C) Contribute observed techniques back — Submit techniques observed in this environment to MITRE ATLAS, and contribute to OWASP LLM and Agentic project revisions, sector information-sharing bodies and published incident analyses. This matters more in AI security than in most fields because the public technique catalogue is young, and the gap between what practitioners observe and what is publicly documented is wide, which means a single well-documented submission can be genuinely novel rather than incremental. Target at least four substantive submissions per year, count only those accepted, anonymise and legally review every one, and coordinate disclosure with affected model providers or vendors where a technique touches their systems. Where the L1 activity recorded threats for which no ATLAS technique fit, revisit that list first: it is the highest-yield source of contribution candidates the programme has. Infrastructure threat output lands most usefully in the egress allowlist, the exposure tests and the per-tenant quota configuration, which are the controls that actually encode the reachability decisions.

Outcome metrics

Success criteria

• Every Critical-tier asset is exercised by adversary emulation at least annually, by a team that did not author the model.
• Every threat in the library is classified as demonstrated here, demonstrated elsewhere or theoretical, with the theoretical share reported and falling.
• Emulation findings the model did not anticipate are fed back into the archetype library within 30 days, improving every asset sharing that archetype.
• Incidents whose root cause was an already-documented untreated threat are reported openly and falling year over year.
• At least four accepted external technique contributions per year, anonymised, legally reviewed and disclosure-coordinated.

Assessment questions

1. Is every Critical-tier asset exercised by adversary emulation at least annually by a team that did not author the model, with both error directions measured, including behaviours the model failed to anticipate? Evidence this in the Infrastructure domain against the threat snapshot for the inference endpoint, with its reachability graph.
2. Is every library threat classified as demonstrated here, demonstrated elsewhere or theoretical, and is the share of incidents whose root cause was an already-documented untreated threat reported openly and falling? For Infrastructure, the assessor should be shown the threat snapshot for the inference endpoint, with its reachability graph.
3. Does the organisation contribute at least four accepted techniques per year to ATLAS and the community, anonymised, legally reviewed and disclosure-coordinated, drawing first on the threats for which no existing technique fitted? In Infrastructure, ask to see the threat snapshot for the inference endpoint, with its reachability graph.

Common pitfalls

• ❌ Reporting only the threats emulation confirmed, which converts a learning exercise into a validation ritual.
• ❌ Feeding emulation findings into the single asset's model rather than the archetype library, so one exercise improves one asset instead of all of them.
• ❌ Not tracking incidents whose root cause was already in the library untreated, which conceals the difference between failing to foresee and failing to act.
• ❌ Leaving the unmapped-technique list from L1 unrevisited, discarding the best contribution candidates the programme has.
• ❌ Submitting techniques that touch a provider's systems without coordinating disclosure, which damages the relationships the programme depends on.

Security Requirements (SR)

Objective: Publish a base security requirements pack per infrastructure archetype, every requirement carrying a testable acceptance criterion, so teams inherit requirements rather than negotiating them.

A program at this level has written down what each infrastructure archetype must do, once, in a form a test can check. The pack covers inference endpoint, model registry, accelerator fleet, orchestrator and control plane, vector-store infrastructure, AI-specific CI/CD and feature store, and every requirement carries an acceptance criterion specific enough that two reviewers assessing the same asset reach the same verdict. This is the practice that converts threat-model output into something buildable: a threat without a requirement is an observation, and a requirement without a testable criterion is an aspiration. Requirements trace back to the threats they mitigate and forward to the tests that verify them.

Level 1, Foundational

Publish a base security requirements pack per infrastructure archetype, every requirement carrying a testable acceptance criterion, so teams inherit requirements rather than negotiating them.

Activities

A) Author the base requirements pack per archetype — Write one base pack per archetype covering inference endpoint, model registry, accelerator fleet, orchestrator and control plane, vector-store infrastructure, AI-specific CI/CD and feature store. The pack states what any asset of that archetype must do, drawn from the TA threat library so that every requirement answers a specific threat rather than a general anxiety. In this domain the base requirements centre on no standing human access to production fleets, egress allowlisting from inference hosts, model provenance verification before serving, per-tenant isolation and quota, secrets held in a vault with defined rotation, and signed model artifacts through the promotion pipeline. Keep the base pack deliberately small, targeting fifteen to twenty-five requirements: a pack of two hundred requirements is negotiated down in practice, and the negotiation happens invisibly at the team's discretion rather than visibly at the programme's. Write each requirement as an obligation on the system rather than on the team, because a requirement phrased as an activity is satisfied by doing the activity badly. Version the packs and publish the change history so teams can see what changed and when.

B) Attach a testable acceptance criterion to every requirement — Give every requirement an acceptance criterion that states how conformance is demonstrated and by what evidence. The criterion must be specific enough that two reviewers assessing the same asset independently reach the same verdict, which is the operational test of whether a criterion is real. Prefer criteria a test can evaluate over criteria a person must judge, and where judgment is unavoidable, state the evidence the judgment rests on. Requirements that cannot be given a testable criterion are either too vague to be useful or belong in guidance rather than requirements, and moving them is better than leaving them to be argued about at every review. Map each criterion to the practice that will verify it, being implementation review, security testing or environment hardening, so that the verification path exists before the requirement is issued.

C) Establish traceability and declare the consumption envelope — Build a requirements evidence map linking each requirement backward to the TA threat it mitigates and forward to the test case, review check or detection that verifies it. Publish the map and treat a requirement with no verifying mechanism as an open gap, because an unverified requirement is a statement of intent that the programme has no way to know is met. Separately, require every archetype to declare a consumption envelope: a spend or volume cap appropriate to its expected workload, and an explicit behaviour when that cap is reached, being degrade, queue or refuse, and never silently continue. Stating the cap as a requirement rather than an operational setting is what makes it testable at per 1k inference requests and reviewable at design time, and it is the point at which unbounded consumption stops being an operational surprise and becomes a design decision.

Outcome metrics

Consumption metrics — Archetypes whose requirements pack declares a consumption cap and an explicit behaviour at the cap per per archetype requirements pack · Declared consumption envelope per unit of work, stated per archetype per per 1k inference requests · Consumption-cap requirements carrying a testable acceptance criterion per per consumption requirement

Success criteria

• A base requirements pack exists for every inventoried infrastructure archetype, deliberately small at fifteen to twenty-five requirements.
• Every requirement carries an acceptance criterion specific enough that independent reviewers reach the same verdict, verified by double-review calibration at 85% or better.
• A requirements evidence map traces at least 90% of requirements backward to a threat and forward to a verifying mechanism, with gaps published.
• Every archetype declares a consumption envelope with an explicit behaviour at the cap, expressed per unit of work.
• Requirements are phrased as obligations on the system rather than activities for the team.

Assessment questions

1. Does a base security requirements pack exist for every inventoried infrastructure archetype, deliberately small, derived from the TA threat library, and phrased as obligations on the system rather than activities for the team?
2. Does every requirement carry an acceptance criterion specific enough that two reviewers independently reach the same verdict, verified by a periodic blind double-review calibration exercise at 85% agreement or better? For Infrastructure, the assessor should be shown the serving requirements pack and its egress-allowlist acceptance criterion.
3. Does a requirements evidence map trace at least 90% of requirements backward to a threat and forward to a verifying mechanism with gaps published, and does every archetype declare a consumption envelope with an explicit, testable behaviour at the cap? In Infrastructure, ask to see the serving requirements pack and its egress-allowlist acceptance criterion.

Common pitfalls

• ❌ Writing packs of two hundred requirements, which get negotiated down invisibly at each team's discretion rather than visibly at the programme's.
• ❌ Phrasing requirements as activities for the team rather than obligations on the system, which are satisfied by performing the activity badly.
• ❌ Accepting acceptance criteria that two reviewers would read differently, which turns every review into a negotiation about what the requirement meant.
• ❌ Issuing requirements with no verifying mechanism, so the programme has no way to know whether they are met.
• ❌ Leaving the consumption cap to operations, which makes unbounded spend an operational surprise rather than a design decision.

Level 2, Comprehensive

Calibrate infrastructure requirements packs to asset tier, maintain a per-asset requirements evidence map, and make conformance demonstrable rather than asserted.

Activities

A) Derive requirements scope from tier — Publish the mapping from asset tier to requirements scope: which requirements are mandatory at each tier, which are recommended, and which tier-specific additions apply above the base pack. Derive the scope from the inventory tier automatically rather than letting each team select, because scope selection under delivery pressure reliably produces the minimum defensible set and does so invisibly. Add tier-specific requirements where the tier warrants them, such as an independent conformance assessment and dual approval at Critical. Where a team believes a requirement does not apply, that is a deviation and goes through the deviation process rather than through scope interpretation, which is the distinction that keeps the tier mapping meaningful rather than advisory. Tier scope in this domain keys chiefly on network exposure, because reachability is what converts every other weakness into an incident and it dominates every other dimension in practice.

B) Maintain a per-asset requirements evidence map — For every Critical- and High-tier asset, maintain a requirements evidence map recording per requirement its status, the evidence that demonstrates conformance, where that evidence lives, its evidence grade from E0 to E3, and its last verification date. The map is maintained continuously as the asset is built rather than assembled at review, which is what makes design review cheap and audit response fast. Wire evidence collection to the systems that already hold it, being the code repository, the CI pipeline, the test registry and the configuration store, so that the map reflects reality rather than the last time someone updated it. Report the distribution of evidence grades per asset, since an asset whose conformance rests entirely on E0 assertions is in a materially different position from one resting on E2 telemetry, and a single conformance percentage conceals that difference entirely. Evidence for infrastructure requirements is the most machine-readable in the model, since infrastructure-as-code state is itself the evidence, which is why E2 coverage should be highest here.

C) Govern deviations and verify consumption caps per tier — Operate a deviation process: any requirement not met is recorded with a rationale, a compensating control, a named approver whose authority scales with tier, and a mandatory expiry date. Review the deviation register monthly, report Critical-tier deviations to the sponsor, and treat a deviation past expiry as an open finding. Deviations are healthy when visible and corrosive when absorbed, and the register is what determines which. Separately, make consumption caps tier-proportionate and verify them rather than declaring them: each Critical- and High-tier asset's cap is expressed relative to per 1k inference requests, and the at-cap behaviour is exercised at least annually so that what happens at the limit is known before it happens in production. Record the exercise result, because a cap whose behaviour has never been observed is a cap nobody should rely on. Cap exercises in this domain drive an endpoint to its per-tenant limit and confirm shedding, which is the only test that distinguishes a configured limit from an enforced one.

Outcome metrics

Consumption metrics — Critical- and High-tier assets whose consumption cap behaviour has been exercised and recorded per per Critical or High tier asset · Declared consumption envelope against observed consumption, per tier per per 1k inference requests · Deviations granted against a consumption requirement, with compensating control recorded per per active deviation

Success criteria

• Requirements scope is derived automatically from asset tier, and scope disagreements route through the deviation process rather than through interpretation.
• At least 95% of Critical-tier assets carry a current per-asset requirements evidence map maintained continuously rather than assembled at review.
• Evidence grades are reported per asset, with at least 60% of Critical-tier requirements resting on E2 evidence or better.
• Every deviation carries rationale, compensating control, a tier-appropriate approver and an expiry, with none past expiry.
• Every Critical-tier asset's consumption cap behaviour has been exercised within 12 months and the result recorded.

Assessment questions

1. Is requirements scope derived automatically from the SM L2 asset tier, with disagreements routed through a deviation process rather than resolved as scope interpretation under delivery pressure? Evidence this in the Infrastructure domain against the serving requirements pack and its egress-allowlist acceptance criterion.
2. Does every Critical- and High-tier asset carry a per-asset requirements evidence map maintained continuously from repository, CI and configuration signals, reporting an evidence grade per requirement rather than a single conformance percentage? For Infrastructure, the assessor should be shown the serving requirements pack and its egress-allowlist acceptance criterion.
3. Do all deviations carry rationale, compensating control, tier-appropriate approver and expiry with none past expiry, and has every Critical-tier asset's consumption cap behaviour been exercised and recorded within the last 12 months? In Infrastructure, ask to see the serving requirements pack and its egress-allowlist acceptance criterion.

Common pitfalls

• ❌ Letting teams select their own requirements scope, which reliably produces the minimum defensible set and does so invisibly.
• ❌ Assembling evidence maps at review time, which makes every review expensive and every audit a project.
• ❌ Reporting a single conformance percentage, which conceals whether conformance rests on assertion or on telemetry.
• ❌ Granting deviations without expiry, after which the deviation register becomes the real requirements set.
• ❌ Declaring consumption caps without ever exercising them, so their behaviour at the limit is first learned during an incident.

Level 3, Industry-Leading

Express infrastructure requirements as executable policy evaluated continuously in the pipeline, so conformance is enforced at merge rather than assessed at review.

Activities

A) Express requirements as executable policy in the pipeline — Convert every machine-evaluable requirement into a policy check that runs in the delivery pipeline, evaluating the change rather than the documentation. Blocking severity follows asset tier: a Critical-tier breach fails the build, a Low-tier breach warns. Agree the blocking behaviour with engineering before enabling it, because a security check that blocks merges without an agreed escalation path is a check that gets disabled within a month and takes the practice's credibility with it. Publish the list of requirements that are not machine-evaluable and review it quarterly with the aim of shrinking it, since that list is the honest measure of how far enforcement actually reaches. Provide a documented break-glass path with automatic notification and mandatory post-hoc review, because emergencies happen and an unbreakable gate gets bypassed in ways nobody sees. Infrastructure requirements are the most naturally machine-evaluable in the model, because infrastructure-as-code is already a declarative artifact and an exposure or egress requirement is a property of it.

B) Evolve requirements from operational evidence — Close the loop from operations to requirements. Feed incident root causes, security test failures, implementation review findings and monitoring detections into a quarterly requirements review, and treat any failure mode appearing three times without a covering requirement as a mandatory pack change. Publish a service level covering the interval from a recurring failure mode to a new or amended requirement, targeting 60 days including the policy implementation. Retire requirements that operational evidence shows are no longer load-bearing, because a pack that only grows becomes one teams route around; the willingness to remove requirements is what keeps the pack small enough to be respected. Record for each requirement how many times its policy check has actually fired, which distinguishes requirements that are doing work from those that are decoration. The recurring failure modes that drive new infrastructure requirements are a security group widened during an incident and a promotion that bypassed provenance verification, both of which look like operational necessity at the time.

C) Contribute executable requirement packs back — Contribute at least four substantive artifacts per year, weighted toward executable form: requirement packs with their policy implementations, acceptance criteria that others can adopt directly, and conformance check libraries. Executable contributions are worth substantially more than prose in this practice, because a peer organisation can run a policy check and merely read a requirement, and the adoption difference between the two is large. Work through OWASP projects, OpenSSF AI, sector bodies and the relevant standards communities. Count only accepted or published artifacts, review each legally before release, and prefer contributing the checks that fire most often in this environment, since those are the ones carrying the most operational evidence behind them. Infrastructure requirement packs contribute exceptionally well because they ship as modules rather than prose, and a peer can adopt a hardened serving module in an afternoon.

Outcome metrics

Consumption metrics — Consumption-cap requirements enforced as pipeline policy rather than asserted in a document per per consumption requirement · Changes blocked at merge for breaching a declared consumption envelope per per 1k inference requests · Consumption-cap policy checks that have never fired per per consumption policy check

Success criteria

• At least 80% of machine-evaluable requirements run as pipeline policy checks, with the non-evaluable remainder listed explicitly and shrinking.
• Critical-tier breaches are blocked at merge, with a documented break-glass path carrying automatic notification and mandatory post-hoc review within 5 business days.
• Recurring failure modes reach a new or amended requirement, including its policy implementation, within 60 days.
• Requirements are retired as well as added, with firing counts reviewed annually and never-firing requirements examined.
• At least four accepted external contributions per year, the majority in executable rather than prose form.

Assessment questions

1. Do at least 80% of machine-evaluable requirements run as pipeline policy checks with tier-proportionate blocking severity agreed with engineering, and is the non-machine-evaluable remainder listed explicitly and reviewed quarterly for reduction? Evidence this in the Infrastructure domain against the serving requirements pack and its egress-allowlist acceptance criterion.
2. Do recurring failure modes from incidents, test failures and review findings reach a new or amended requirement including its policy implementation within 60 days, and are requirements retired as well as added based on firing counts? For Infrastructure, the assessor should be shown the serving requirements pack and its egress-allowlist acceptance criterion.
3. Is there a documented break-glass path with automatic notification and mandatory post-hoc review within 5 business days, and does the organisation contribute at least four accepted artifacts per year, the majority executable rather than prose? In Infrastructure, ask to see the serving requirements pack and its egress-allowlist acceptance criterion.

Common pitfalls

• ❌ Enabling blocking checks without an agreed escalation path, after which the checks are disabled within a month and take the practice's credibility with them.
• ❌ Automating the easy requirements and leaving the non-evaluable list unpublished, so nobody knows how far enforcement actually reaches.
• ❌ Adding requirements without ever retiring any, producing a pack large enough that teams route around it.
• ❌ Tolerating high false-positive rates on policy checks, which trains engineers to bypass rather than to fix.
• ❌ Contributing requirement prose rather than executable checks, which peers read and do not adopt.

Secure Architecture (SA)

Objective: Publish reference architecture patterns and an antipattern catalogue for each infrastructure archetype, so the secure design is the default path rather than an achievement.

A program at this level has drawn the secure shape of each infrastructure archetype once, so teams adopt rather than invent. The library covers the serving pattern with per-tenant isolation and egress allowlisting, the registry pattern with signed artifacts and provenance verification before promotion, the fleet pattern with no standing human access and time-bound elevation, and the control-plane pattern with blast-radius bounds, and it is paired with an antipattern catalogue naming the shapes that recur and what to replace them with. An antipattern catalogue is the more useful half and the more commonly missing one: teams recognise what they are building far more readily than they recognise what they should have built. Every pattern states which threats it mitigates, so adoption is an argument rather than an instruction.

Level 1, Foundational

Publish reference architecture patterns and an antipattern catalogue for each infrastructure archetype, so the secure design is the default path rather than an achievement.

Activities

A) Publish reference patterns per archetype — Author one reference pattern per archetype, covering the serving pattern with per-tenant isolation and egress allowlisting, the registry pattern with signed artifacts and provenance verification before promotion, the fleet pattern with no standing human access and time-bound elevation, and the control-plane pattern with blast-radius bounds. Each pattern states the components and their boundaries, the controls at each boundary, the threats it mitigates by identifier from the TA library, the requirements it satisfies from the SR pack, and the conditions under which it does not apply. Keep each pattern to a length a team reads before designing rather than after, targeting three pages with a diagram. Validate each with an engineer who has built that archetype, because a pattern authored entirely within security describes a system that is secure and unbuildable, and teams respond to that by ignoring it rather than by arguing. Publish patterns in the repository where the work happens rather than in a separate architecture space.

B) Publish the antipattern catalogue — Author an antipattern catalogue naming the insecure shapes that actually recur in this domain, being the inference endpoint reachable from the default network, the registry that serves unsigned artifacts, and the accelerator fleet with a shared standing administrative credential. For each antipattern, state why it is attractive, since it is always attractive for a reason and a catalogue that does not acknowledge that reads as naive; what specifically goes wrong; and which reference pattern replaces it. The antipattern half of the library is the more useful and the more commonly omitted, because teams recognise the thing they are building far more readily than they recognise the thing they should have built instead. Draw the catalogue from real designs seen in review rather than from literature, and add to it whenever review encounters a new recurring shape.

C) Establish the untrusted-content boundary and a design-time self-check — Document explicitly where untrusted content enters each archetype and how it is kept separate from instruction, control and identity. This is the single most consequential architectural decision in AI systems and the one most often left implicit, because the failure it prevents, content becoming instruction, does not look like a boundary violation in any conventional sense. State the pattern for isolation, for provenance marking, and for what the system must never do with content it did not author. Publish a short design-time self-check, at most ten questions, that a team can answer before requesting review, which both raises the quality of what arrives at review and tells the team what the review will ask, removing the asymmetry that makes design review feel arbitrary.

Outcome metrics

Oversight metrics — Share of designs where the reviewer recorded a required change before approval (detects: rubber-stamping at design review, where approval is recorded without the review having altered anything, which is the default state of a review with no capacity to say no) · Designs approved with a human gate declared on promotion of a model into production serving but no stated refusal authority (detects: a gate that exists structurally while the person holding it cannot actually refuse, which satisfies the control on paper and not in fact)

Success criteria

• A reference pattern exists for at least 85% of inventoried infrastructure archetypes, each validated with an engineer who has built that archetype.
• Every pattern states the threats it mitigates by TA identifier and the requirements it satisfies.
• An antipattern catalogue of at least eight entries is published, each drawn from a design actually seen in review and naming its replacement pattern.
• Every archetype ingesting content the organisation does not author has a documented untrusted-content boundary.
• A design-time self-check of at most ten questions is published and completed by at least 70% of designs arriving at review.

Assessment questions

1. Does a reference pattern exist for at least 85% of inventoried infrastructure archetypes, each stating components, boundaries, controls, the threats it mitigates by TA identifier, the requirements it satisfies and where it does not apply, validated with an engineer who has built that archetype?
2. Is there an antipattern catalogue of at least eight entries drawn from designs actually seen in review, each stating why the shape is attractive, what goes wrong, and which reference pattern replaces it? For Infrastructure, the assessor should be shown the hardened serving module and its egress allowlist.
3. Does every archetype ingesting content the organisation does not author have a documented untrusted-content boundary, and is a design-time self-check published and completed by at least 70% of designs arriving at review? In Infrastructure, ask to see the hardened serving module and its egress allowlist.

Common pitfalls

• ❌ Authoring patterns entirely within security, producing designs that are secure and unbuildable, which teams ignore rather than contest.
• ❌ Publishing reference patterns without an antipattern catalogue, omitting the half teams actually recognise themselves in.
• ❌ Writing an antipattern catalogue that does not acknowledge why each shape is attractive, which reads as naive and is dismissed.
• ❌ Leaving the untrusted-content boundary implicit, which is the most consequential architectural decision in AI systems and the least conventional-looking.
• ❌ Publishing patterns in a separate architecture space rather than where the work happens, after which they are found only by people already looking.

Level 2, Comprehensive

Calibrate the infrastructure reference-pattern set by risk tier and scale the autonomy and oversight controls in each pattern with the asset's capability tier.

Activities

A) Declare tier applicability across the pattern library — Annotate every pattern in the library with the tiers it applies to and the tiers at which it is mandatory rather than recommended. Publish the mapping so that a team's pattern set is derived from the asset's tier rather than assembled by preference. At Critical tier, pattern adherence is mandatory and any deviation requires a full-lane design review with a named architect; at Low tier the pattern is recommended and deviation needs only a recorded note. Stating this removes the ambiguity that otherwise resolves in favour of whatever is quickest to build. Re-derive the applicable pattern set automatically when an asset's tier changes, because a Medium-to-Critical promotion that does not change the pattern obligations has changed nothing that matters.

B) Scale autonomy and oversight controls with capability tier — Within each pattern, define how its controls tighten as the asset's capability tier rises. Bound blast radius more tightly, shorten the autonomy budget between human checkpoints, and require a human gate on promotion of a model into production serving at higher capability tiers where lower tiers need only logging. Specify each gate numerically rather than in principle: who holds the authority to refuse, what information they receive, and how long they have. A gate specified as a principle is compressed under delivery pressure until scrutiny is impossible, and because nothing in the specification objects, the erosion is invisible. Publish a reviewer-load ceiling per tier above which review capacity must be added rather than reviewers exhorted, since a gate held by an overloaded reviewer is a gate in name only.

C) Run the incident-to-pattern learning loop — Establish a standing review that takes every incident reaching a root cause and asks two questions: was there a pattern that should have prevented this, and if so why was it not followed, or if not what pattern should now exist. Record the answer for every incident, including the answer that no pattern change is warranted, because the discipline of recording the negative case is what keeps the loop honest rather than performative. Where a pattern existed and was not followed, the finding belongs to the review and adoption practices rather than to the library. Where no pattern existed, author one and publish it within 60 days. Track the share of incidents attributable to a pattern that existed and was not followed, which separates a library problem from an adoption problem and is the single most useful number this practice produces.

Outcome metrics

Oversight metrics — Reviewer override rate on pattern deviation requests, paired with sustained-override rate (detects: deviation approval becoming automatic under delivery pressure, where pairing with sustained rate counters the gaming that follows once the rate becomes a target) · Design reviews per reviewer per week against the published ceiling (detects: reviewer overload, where architectural scrutiny cannot survive the volume regardless of reviewer skill, which is a capacity defect rather than a personnel one) · Critical-tier designs where the human gate on promotion of a model into production serving specifies authority, information and a time budget numerically (detects: oversight specified as a principle rather than a number, which is compressed under delivery pressure until scrutiny is impossible and nothing objects)

Success criteria

• Every pattern declares its tier applicability and whether it is mandatory or recommended at each tier.
• Controls within each pattern scale with capability tier, with every human gate specified numerically in authority, information and time.
• At least 95% of Critical-tier assets either follow a reference pattern or carry a full-lane deviation review, with no third category.
• Every incident reaching root cause is checked against the library, with negative findings recorded.
• A per-tier reviewer-load ceiling is published and monitored, and breaching it adds capacity rather than exhortation.

Assessment questions

1. Does every pattern declare its tier applicability and mandatory-versus-recommended status, with the applicable set re-derived automatically when an asset's tier changes? Evidence this in the Infrastructure domain against the hardened serving module and its egress allowlist.
2. Do the controls inside each pattern tighten as capability rises, with the human gate on promotion of a model into production serving specified numerically in authority, information and time budget rather than as a principle, and is a per-tier reviewer-load ceiling published?
3. Is every incident reaching a root cause checked against the library for a preventing pattern with negative findings recorded, and is the share attributable to an existing pattern not followed tracked separately from gaps in the library? In Infrastructure, ask to see the hardened serving module and its egress allowlist.

Common pitfalls

• ❌ Publishing patterns without tier applicability, which leaves adoption scope to resolve in favour of whatever is quickest to build.
• ❌ Specifying human gates as principles rather than numbers, so they compress invisibly under delivery pressure until scrutiny is impossible.
• ❌ Recording only the incidents that produced a pattern change, which makes the learning loop performative rather than honest.
• ❌ Conflating a pattern that was missing with a pattern that was ignored, which sends improvement effort to the wrong practice.
• ❌ Responding to reviewer overload with exhortation rather than capacity, which converts the gate into a formality.

Level 3, Industry-Leading

Publish infrastructure patterns as executable artifacts adopted by default, verify conformance continuously, and contribute the pattern library back to the field.

Activities

A) Ship patterns as executable artifacts — Convert each reference pattern into something a team runs rather than reads: infrastructure modules, project scaffolds, libraries or configuration bundles that instantiate the pattern correctly by default. This is the level's central move and the one that changes adoption economics, because a pattern that is faster to adopt than to circumvent is adopted without enforcement, and no amount of mandate achieves what convenience achieves. Version the artifacts, publish their change history, and provide a migration path when a pattern changes so that existing adopters are not stranded on an old version, which is the most common reason executable pattern libraries decay. Measure adoption of the artifact rather than conformance to the document, since the artifact is now the pattern. In this domain the executable artifact is an infrastructure-as-code module, which is the most directly adoptable form in the entire model: a peer runs it and gets the hardened serving, registry or vector-store deployment with no translation.

B) Verify conformance continuously against the running system — Verify that what is running still matches the pattern that was approved, continuously rather than at review. Draw the signals from configuration state, runtime telemetry and the deployment pipeline, and evaluate each pattern's invariants as checks that can fail. Architecture drift is the defining failure of this practice at scale: designs are reviewed once and systems change continuously, so a library verified only at design time describes the past. Publish a service level covering detection of a Critical-tier conformance breach within 5 business days, and route breaches to the issue-management practice with tier-appropriate severity rather than to a report nobody owns. Where an invariant cannot be checked automatically, list it explicitly so the verification's real reach is known. Conformance verification reads live cloud configuration, admission policy and network state against the pattern, which is close to continuous by construction because these signals already stream.

C) Contribute the executable pattern library back — Contribute at least four substantive artifacts per year, weighted toward executable form: reference implementations, infrastructure modules, scaffolds and the conformance checks that verify them. Executable contributions carry disproportionate value in this practice, because a peer organisation can adopt a module and can only read a diagram, and the adoption gap between the two is the difference between influencing the field and describing it. Work through OpenSSF AI, CNCF where infrastructure applies, OWASP projects and sector bodies. Count only accepted artifacts, review each legally before release, and prefer contributing the patterns with the highest internal adoption, since those carry the most operational evidence and are the most likely to survive contact with another organisation's environment. Infrastructure patterns are the highest-value contribution class in the model, since a hardened module is immediately usable and the underlying platforms are shared across the industry.

Outcome metrics

Oversight metrics — Pattern deviations later implicated in an incident, as a share of deviations approved (detects: deviation approval that is procedurally correct and substantively wrong, which only post-hoc linkage to incidents can reveal) · Automated pattern conformance checks overridden by a human, with the override sustained (detects: automation override becoming the routine path around a check that is inconvenient rather than wrong, which is how enforced patterns quietly become advisory) · Reviewer disagreement rate on a blind double-review sample of Critical-tier designs (detects: reviewer drift, where the same design would be approved by one architect and refused by another, making the gate a lottery rather than a control)

Success criteria

• At least 80% of reference patterns ship as executable artifacts with versioning and a migration path on breaking change.
• At least 75% of new Critical- and High-tier assets adopt the executable artifact at creation.
• At least 85% of pattern invariants are verified continuously against the running system, with unverifiable invariants listed explicitly.
• Critical-tier conformance breaches are detected within 5 business days and routed to issue management with tier-appropriate severity.
• At least four accepted external contributions per year, the majority executable rather than documentary.

Assessment questions

1. Do at least 80% of reference patterns ship as executable artifacts with versioning and a migration path, and do at least 75% of new Critical- and High-tier assets adopt the artifact at creation rather than adopting the document? Evidence this in the Infrastructure domain against the hardened serving module and its egress allowlist.
2. Are at least 85% of pattern invariants verified continuously against the running system with unverifiable invariants listed explicitly, and are Critical-tier breaches detected within 5 business days and routed to issue management? For Infrastructure, the assessor should be shown the hardened serving module and its egress allowlist.
3. Does the organisation contribute at least four accepted artifacts per year in majority-executable form, prioritising the patterns with the highest internal adoption and therefore the most operational evidence? In Infrastructure, ask to see the hardened serving module and its egress allowlist.

Common pitfalls

• ❌ Shipping executable patterns without a migration path, which strands existing adopters on old versions and is the usual cause of library decay.
• ❌ Verifying conformance only at design time, which describes the past while the running system changes continuously.
• ❌ Measuring conformance to the document after the artifact has become the real pattern, which measures the wrong thing.
• ❌ Leaving unverifiable invariants unlisted, so the verification's actual reach is unknown and overstated.
• ❌ Contributing diagrams rather than modules, which peers read and do not adopt.

Business Function: Verification

Design Review (DR)

Objective: Establish a two-lane design review for infrastructure AI changes with a published checklist and service level, so review is predictable enough that teams route through it.

A program at this level reviews infrastructure AI designs against a published checklist, in one of two lanes, with a stated turnaround. The fast lane clears low-risk changes against a standard checklist in days; the full lane handles novel or high-risk designs with a named architect. Publishing both the checklist and the service level is what makes the practice work: a review whose criteria and duration are unknown is a review teams plan around rather than through, and the routing-around is invisible until an incident reveals it.

Level 1, Foundational

Establish a two-lane design review for infrastructure AI changes with a published checklist and service level, so review is predictable enough that teams route through it.

Activities

A) Establish two review lanes with published service levels — Define a fast lane and a full lane, and publish the routing rule and the service level for each. The fast lane handles designs adopting a sanctioned reference pattern without material deviation, cleared against a standard checklist within 5 business days. The full lane handles novel architectures, pattern deviations and high-risk designs, with a named architect reviewer and a decision within 15 business days. Publishing both numbers is the practice's single most important act: a review of unknown duration is a review teams plan around, and they do so quietly. Measure and report adherence per lane, and treat a persistent breach as a capacity problem to be resourced rather than a communication problem to be explained.

B) Publish the checklist and make decisions written — Publish the review checklist so teams can self-assess before submitting. In this domain the checklist covers network exposure justified and minimised, egress allowlist defined, no standing human access with time-bound elevation instead, model provenance verification before serving, per-tenant isolation and quota, secrets in a vault with a rotation interval, and signed artifacts through promotion. Every review produces a written decision of approve, approve with conditions, or refuse, naming the reviewer and stating the rationale. Conditions carry an owner and a date and are tracked to closure, because an approval with untracked conditions is an approval, and the conditions are the part that mattered. Record the checklist items that most often fail, and feed that back to the education and architecture practices as the clearest available evidence of where design understanding is weak.

C) Define review triggers and detect what bypassed review — Define what triggers a review rather than relying on teams to request one. In this domain the triggers are an exposure change, a new region or residency, a capacity or tenancy model change, or a promotion-pipeline change, plus any new asset entering the inventory. Wire the triggers to the signals that already exist, being inventory changes, deployment events and change records, so that a review obligation is raised by the system rather than remembered by a person. Then detect bypass: reconcile assets reaching production against review decisions, and treat a production asset with no review record as a finding with an owner. Bypass detection is what converts design review from a service teams may use into a gate whose absence is visible, and without it the practice measures only the designs it happened to see.

Outcome metrics

Success criteria

• Two lanes operate with published routing rules and service levels of 5 and 15 business days, with adherence at or above 90%.
• The review checklist is published and teams self-assess against it before submitting.
• Every decision is written with a named reviewer and rationale, and every condition carries an owner, a date and closure tracking.
• Review triggers fire from inventory, deployment and change signals rather than relying on team initiative.
• Bypass reconciliation runs monthly and every unreviewed production asset is raised as a finding.

Assessment questions

1. Do two published review lanes operate with stated routing rules and service levels of 5 and 15 business days, with adherence measured and reported per lane and persistent breaches treated as a capacity problem? Evidence this in the Infrastructure domain against a full-lane decision on an internet-reachable inference endpoint.
2. Is the review checklist published so teams self-assess before submitting, covering network exposure justified and minimised, egress allowlist defined, no standing human access with time-bound elevation i, and does every review produce a written decision with named reviewer, rationale and tracked conditions?
3. Do review triggers fire from inventory, deployment and change signals rather than team initiative, and does monthly bypass reconciliation raise every unreviewed production asset as a finding with an owner? In Infrastructure, ask to see a full-lane decision on an internet-reachable inference endpoint.

Common pitfalls

• ❌ Running review without a published service level, which teams respond to by planning around review rather than through it, invisibly.
• ❌ Approving with conditions that are never tracked, which converts a conditional approval into an unconditional one.
• ❌ Relying on teams to request review, which means the practice only ever sees the designs of teams already inclined to ask.
• ❌ Never reconciling production against review records, so the practice measures the designs it saw and not the ones it missed.
• ❌ Treating persistent service-level breaches as a communication problem rather than as evidence that review capacity is short.

Level 2, Comprehensive

Calibrate infrastructure design review depth and reviewer authority to asset tier, and make review decisions traceable to the threats and requirements they turned on.

Activities

A) Derive review depth and reviewer authority from tier — Publish what each tier receives: at Critical, a full-lane review with a named architect, mandatory threat-model and requirements-evidence review, and a decision recorded where the sponsor can see it; at High, full lane where a deviation exists and fast lane otherwise; at Medium, fast lane; at Low, a checklist pass with sampling. Derive the lane from the inventory tier automatically rather than from reviewer judgment at intake, because intake routing under queue pressure drifts toward the fast lane and does so without anyone deciding to lower the bar. Publish a reviewer-load ceiling per tier, and when it is breached add reviewers rather than shortening reviews, since the alternative is a gate that approves at the rate the queue demands. In this domain the designs that genuinely need architect judgment are new exposure topologies and tenancy models, while an additional replica within an established pattern does not.

B) Make decisions traceable to threats and requirements — Require every full-lane decision to cite the specific threat identifiers from the TA library and requirement identifiers from the SR pack that it assessed, and to state which ones drove each condition or refusal. This converts review from personal judgment into reconstructable reasoning, which matters for three reasons: a second reviewer can reach the same conclusion, an auditor can follow it, and the practice can tell which threats and requirements are actually doing work in review. Report the threats and requirements most frequently driving conditions, since that is direct evidence of where designs are systematically weak and belongs in the architecture and education practices rather than only in the review log. Automated design criteria read infrastructure-as-code plans before apply, which is the highest-leverage automated review point in the model because the plan states the exposure exactly.

C) Govern deviations and make Critical decisions visible — Operate deviation approval as a distinct decision with its own record: what pattern or requirement is not met, why, what compensating control applies, who approved it at an authority proportionate to tier, and when it expires. Review the deviation register monthly and report Critical-tier deviations to the sponsor alongside the approvals, because a review practice that reports only what it approved and not what it excused presents a misleading picture of the estate. Track deviations later implicated in incidents as a distinct measure, which is the only honest feedback on whether deviation judgment is sound rather than merely procedurally correct. Forward tracing in this domain most often implicates approvals granted on a design whose exposure was later widened operationally without returning to review.

Outcome metrics

Success criteria

• Review lane and reviewer seniority are derived from asset tier, with at least 95% of routing tier-derived rather than judged at intake.
• Every Critical-tier design receives a full-lane review with a named architect and a sponsor-visible decision.
• At least 90% of full-lane decisions cite the specific threat and requirement identifiers that drove each condition or refusal.
• Deviation approvals carry a compensating control, tier-appropriate authority and an expiry, with none past expiry.
• A per-tier reviewer-load ceiling is published and breaching it adds capacity rather than shortening reviews.

Assessment questions

1. Is review lane and reviewer seniority derived automatically from asset tier rather than judged at intake under queue pressure, with at least 95% tier-derived routing and a published per-tier reviewer-load ceiling? Evidence this in the Infrastructure domain against a full-lane decision on an internet-reachable inference endpoint.
2. Do at least 90% of full-lane decisions cite the specific TA threat identifiers and SR requirement identifiers that drove each condition or refusal, making the reasoning reconstructable by a second reviewer and an auditor? For Infrastructure, the assessor should be shown a full-lane decision on an internet-reachable inference endpoint.
3. Do deviation approvals carry a compensating control, tier-appropriate approval authority and an expiry with none past expiry, and are Critical-tier deviations reported to the sponsor alongside approvals? In Infrastructure, ask to see a full-lane decision on an internet-reachable inference endpoint.

Common pitfalls

• ❌ Routing at intake by reviewer judgment, which under queue pressure drifts toward the fast lane without anyone deciding to lower the bar.
• ❌ Recording decisions without citing what drove them, which makes review personal judgment and leaves a second reviewer unable to reproduce it.
• ❌ Reporting approvals to the sponsor without reporting deviations, which presents a systematically flattering picture of the estate.
• ❌ Responding to reviewer overload by shortening reviews rather than adding reviewers, producing a gate that approves at queue rate.
• ❌ Never tracking whether deviations were later implicated in incidents, which leaves deviation judgment unevaluated.

Level 3, Industry-Leading

Shift infrastructure design review left into the pipeline, reserve human review for genuine novelty, and measure whether review decisions were right rather than only timely.

Activities

A) Automate the machine-checkable criteria — Convert every review criterion that can be evaluated from an artifact into an automated check running against the design representation, whether that is infrastructure-as-code, a configuration manifest, a service definition or a system card. A design meeting all automated criteria and adopting a sanctioned pattern clears without human review at Medium and Low tier, and arrives at Critical and High review pre-verified so the reviewer spends attention on what is novel rather than on confirming what is conformant. Publish the list of criteria that cannot be automated and review it quarterly for reduction, because that list is the honest boundary of the automation. Keep a human veto and sampling at every tier, since a wholly automated gate encodes yesterday's criteria permanently and nothing in the system notices. In this domain the designs that genuinely need architect judgment are new exposure topologies and tenancy models, while an additional replica within an established pattern does not. Novelty in this domain means a new exposure or tenancy model, not another replica within an approved topology.

B) Measure decision quality, not throughput — Trace review decisions forward. For every approval, record whether the asset subsequently generated an incident, an implementation-review finding or a security-test failure attributable to something the review should have caught. Report false-negative rate, being approvals later implicated, and investigate false-positive pressure, being conditions imposed that evidence later showed unnecessary, since a gate that over-conditions trains teams to argue rather than comply. Throughput and service-level adherence remain as health metrics but are explicitly not the practice's outcome measure, because a review function optimising for turnaround will achieve it by approving, and the metric will look excellent throughout. Publish decision-quality measures to the sponsor alongside throughput. Automated design criteria read infrastructure-as-code plans before apply, which is the highest-leverage automated review point in the model because the plan states the exposure exactly.

C) Calibrate reviewers and concentrate attention on novelty — Run blind double-review on a sample of Critical-tier designs at least quarterly and report the disagreement rate, because a gate whose outcome depends on which architect drew the review is a lottery rather than a control, and disagreement is invisible without deliberately measuring it. Use the disagreement cases to refine criteria rather than to rank reviewers. Reserve human review explicitly for novelty: designs using an unestablished pattern, a new archetype, a new model capability class or an unfamiliar threat surface, and route everything else to automated verification with sampling. Track the share of reviewer hours spent on novel designs, which should rise steadily and is the clearest measure that the automation is doing its job. Forward tracing in this domain most often implicates approvals granted on a design whose exposure was later widened operationally without returning to review.

Outcome metrics

Success criteria

• At least 75% of machine-checkable criteria run automatically, with the non-automatable remainder listed and shrinking.
• Medium- and Low-tier conformant designs clear without human review while sampling continues, and a human veto is retained at every tier.
• Decision quality is measured by forward tracing to incidents and findings, and reported to the sponsor alongside throughput.
• Blind double-review runs quarterly on Critical-tier designs with a disagreement rate at or below 15%, used to refine criteria rather than rank reviewers.
• At least 60% of reviewer hours are spent on genuinely novel designs, and rising.

Assessment questions

1. Are at least 75% of machine-checkable review criteria evaluated automatically with the non-automatable remainder listed explicitly, and do conformant Medium- and Low-tier designs clear without human review while sampling and a human veto are retained? Evidence this in the Infrastructure domain against a full-lane decision on an internet-reachable inference endpoint.
2. Is decision quality measured by tracing approvals forward to incidents, implementation findings and test failures, reported to the sponsor alongside throughput and explicitly rather than throughput as the outcome measure? For Infrastructure, the assessor should be shown a full-lane decision on an internet-reachable inference endpoint.
3. Does blind double-review run at least quarterly on Critical-tier designs with disagreement at or below 15% used to refine criteria rather than rank reviewers, and is at least 60% of reviewer time spent on genuinely novel designs? In Infrastructure, ask to see a full-lane decision on an internet-reachable inference endpoint.

Common pitfalls

• ❌ Optimising the practice for turnaround, which a review function achieves by approving while the metric looks excellent throughout.
• ❌ Removing the human veto and sampling once automation is in place, which encodes yesterday's criteria permanently with nothing to notice.
• ❌ Never measuring reviewer disagreement, leaving a gate whose outcome depends on which architect drew the review.
• ❌ Using double-review disagreement to rank reviewers, which produces defensive convergence rather than better criteria.
• ❌ Leaving the non-automatable criteria list unpublished, so the boundary of the automation is unknown and assumed wider than it is.

Implementation Review (IR)

Objective: Review what was actually built against what was designed and required for infrastructure AI assets, at go-live, with findings tracked to closure.

A program at this level checks the built system rather than the intended one. Every infrastructure AI asset gets an implementation review before go-live, working from a published checklist that examines network exposure as deployed against what was approved, egress allowlists actually enforced rather than logged, standing access genuinely absent rather than merely discouraged, model provenance verified in the serving path, per-tenant quotas configured, and secrets sourced from the vault rather than from environment variables. The distinction from design review is the entire point: design review assesses what a team intends, implementation review assesses what exists, and the gap between the two is where most AI security failures actually live. Findings carry an owner, a severity and a date, and go-live is conditional on the blocking ones being closed.

Level 1, Foundational

Review what was actually built against what was designed and required for infrastructure AI assets, at go-live, with findings tracked to closure.

Activities

A) Publish the implementation review checklist and run it at go-live — Publish a checklist per archetype that examines the built artifact rather than its documentation. In this domain the review looks at network exposure as deployed against what was approved, egress allowlists actually enforced rather than logged, standing access genuinely absent rather than merely discouraged, model provenance verified in the serving path, per-tenant quotas configured, and secrets sourced from the vault rather than from environment variables. The discipline that makes this practice worth running is verifying presence and effect rather than presence alone: a logging configuration that exists but emits nothing, an allowlist enforced in audit mode, a kill switch wired to nothing, all pass a documentation check and fail a real one. Run the review before go-live for every asset, and make go-live conditional on blocking findings being closed. Where the review cannot verify a control because the environment does not expose it, record that explicitly as unverifiable rather than as passed, since silence is not a pass.

B) Compare the built system against the approved design — Reconcile what was built against the design review decision and the requirements pack: every requirement in scope for the asset's archetype gets a verified, not-verified or unverifiable disposition, and every condition attached to the design approval gets confirmed as met. Record the design-to-implementation gap as a measure in its own right, because a gap that is never measured is assumed to be zero and never is. Where implementation diverges from the approved design, treat it as a finding against the change rather than as a re-opening of the design decision, and route material divergence back to design review rather than absorbing it in the implementation record.

C) Track findings to closure with severity and ownership — Give every finding a severity, an owner, a remediation date and a disposition of remediated, accepted or deferred. An accepted finding requires a named acceptor with authority proportionate to the severity, because a finding that is neither fixed nor formally accepted leaves residual risk that nobody owns, which is the most common way implementation review findings evaporate. Feed all findings into the issue-management backlog rather than maintaining a separate review register, so that AI findings age and escalate on the same rails as everything else. Report finding density by archetype and by checklist item quarterly, which tells the architecture and education practices where implementations systematically diverge rather than where reviewers happened to look.

Outcome metrics

Oversight metrics — Share of implementation reviews recording at least one finding before sign-off (detects: rubber-stamping at implementation review, where sign-off is recorded without the review having examined anything, which is the default outcome when review is a release-checklist item) · Findings accepted as risk at sign-off rather than remediated, with a named acceptor (detects: silent acceptance, where a finding is noted and neither fixed nor formally accepted, so nobody owns the residual risk)

Success criteria

• Every AI asset reaching production has an implementation review against the published per-archetype checklist.
• Controls are verified as present and effective, with controls the environment cannot expose recorded as unverifiable rather than passed.
• Every in-scope requirement receives a verified, not-verified or unverifiable disposition at review.
• Blocking findings are closed before go-live, and every accepted finding has a named acceptor with proportionate authority.
• Findings flow into the shared issue backlog rather than a separate review register.

Assessment questions

1. Does every infrastructure AI asset receive an implementation review before go-live against a published per-archetype checklist that verifies controls are present and effective rather than documented, with controls the environment cannot expose recorded as unverifiable rather than passed?
2. Is the built system reconciled against the approved design and requirements pack, with every in-scope requirement given a verified, not-verified or unverifiable disposition and the design-to-implementation gap measured as a metric in its own right? For Infrastructure, the assessor should be shown the go-live review record showing applied state rather than the approved plan.
3. Do all findings carry an owner, severity, date and disposition in the shared issue backlog, with blocking findings closed before go-live and every accepted finding carrying a named acceptor with proportionate authority? In Infrastructure, ask to see the go-live review record showing applied state rather than the approved plan.

Common pitfalls

• ❌ Verifying that a control is configured rather than that it works, which passes logging that emits nothing and allowlists running in audit mode.
• ❌ Recording a control the environment cannot expose as passed, which converts an observability gap into a false assurance.
• ❌ Keeping findings in a separate review register, where they age without the escalation the shared backlog would give them.
• ❌ Allowing findings to be neither fixed nor formally accepted, which leaves residual risk owned by nobody.
• ❌ Treating implementation divergence as a chance to re-open the design decision rather than as a finding against the change.

Level 2, Comprehensive

Calibrate infrastructure implementation review cadence to asset tier and detect configuration drift continuously, so controls verified at go-live stay verified.

Activities

A) Set re-review cadence by tier and material change — Publish the re-review cadence per tier: Critical at go-live, semi-annually and on every material change; High at go-live, annually and on material change; Medium at go-live and annually; Low at go-live. Define material change concretely for this domain, being a security group widened for a debugging session and left open, a quota raised under load and never restored, provenance verification bypassed for a hotfix, and a vault integration replaced by an environment variable, and wire the triggers to change records and deployment events so the obligation is raised by the system rather than remembered. The calendar is the weakest trigger available and should be the floor rather than the mechanism, because an asset that changed materially in month two carries a verified-once record for ten months and the record looks identical to an accurate one.

B) Detect configuration drift continuously on high-tier assets — Stand up continuous drift detection for Critical- and High-tier assets covering the controls whose silent disablement would matter most. In this domain the drift that recurs is a security group widened for a debugging session and left open, a quota raised under load and never restored, provenance verification bypassed for a hotfix, and a vault integration replaced by an environment variable. Detect it from configuration state, infrastructure-as-code divergence, policy enforcement mode and runtime telemetry, and alert on change rather than on schedule. Drift detection is what converts implementation review from a point-in-time assertion into a continuing one, and it is the single highest-value activity at this level, because implementations rarely fail review, they pass and then change. Publish a detection service level of 5 business days at Critical tier and route detections into the issue backlog with tier-appropriate severity rather than into a report.

C) Apply tier-proportionate service levels and deferral authority — Set remediation service levels by finding severity and asset tier, tighter at Critical, and report adherence rather than only backlog size. Require deferral authority to rise with tier and severity, so a deferred Critical-tier finding needs an executive acceptance with an expiry and a quarterly review, while a Low-tier one needs the practice lead. Report deferred Critical-tier findings to the sponsor alongside remediated ones, because a practice that reports only closures presents a picture that flatters it. Track findings that were deferred and later implicated in an incident as a distinct measure, since that is the only feedback available on whether deferral judgment is sound rather than merely authorised.

Outcome metrics

Oversight metrics — Reviewer finding rate per review, compared across reviewers on comparable assets (detects: reviewer drift, where the same implementation would pass with one reviewer and fail with another, making sign-off depend on assignment rather than on the code) · Implementation reviews per reviewer per week against the published ceiling (detects: reviewer overload, where depth falls as volume rises and the review continues to be recorded as complete) · Critical-tier findings deferred past their remediation window with a recorded acceptance authority (detects: deferral becoming the default disposition under release pressure, which converts a blocking finding into an advisory one without a decision)

Success criteria

• Re-review cadence is published per tier and driven by material-change triggers wired to change and deployment events.
• At least 90% of Critical-tier assets are under continuous drift detection, with detection within 5 business days.
• Findings carry tier and severity service levels with adherence at or above 90% per tier.
• Deferral authority rises with tier and severity, and deferred Critical findings carry executive acceptance with expiry.
• Deferred findings later implicated in incidents are tracked as a distinct measure of deferral judgment.

Assessment questions

1. Is re-review cadence published per tier and driven by material-change triggers wired to change records and deployment events, with the calendar serving as a floor rather than the mechanism? Evidence this in the Infrastructure domain against the go-live review record showing applied state rather than the approved plan.
2. Are at least 90% of Critical-tier assets under continuous configuration drift detection with a 5 business day detection service level, and are detections routed into the shared backlog with tier-appropriate severity? For Infrastructure, the assessor should be shown the go-live review record showing applied state rather than the approved plan.
3. Do remediation service levels vary by tier and severity with adherence at or above 90% per tier, does deferral authority rise with tier, and are deferred findings later implicated in incidents tracked as a distinct measure? In Infrastructure, ask to see the go-live review record showing applied state rather than the approved plan.

Common pitfalls

• ❌ Relying on annual re-review, which leaves an asset that changed in month two carrying a record indistinguishable from an accurate one for ten months.
• ❌ Detecting drift on a schedule rather than on change, which reintroduces exactly the point-in-time weakness the activity exists to remove.
• ❌ Routing drift detections into a report rather than the backlog, where they age without escalation and without an owner.
• ❌ Reporting closures to the sponsor without deferrals, which systematically flatters the practice.
• ❌ Authorising deferral without ever checking whether deferred findings later caused incidents, leaving the judgment unevaluated.

Level 3, Industry-Leading

Verify infrastructure implementation continuously in the pipeline and at runtime, and measure whether review judgment was right rather than only whether review happened.

Activities

A) Assert controls continuously in pipeline and runtime — Express each verifiable control as an assertion that runs both in the delivery pipeline against the change and against the running system on a continuous schedule. Pipeline assertions prevent a regression from merging; runtime assertions catch what was changed outside the pipeline, and both are needed because the second is where drift actually originates. Give each assertion a tier-proportionate severity, block merges on Critical-tier failures, and route runtime failures to the backlog automatically. Publish the list of controls that cannot be asserted and review it quarterly with a reduction target, because that list is the true boundary of the verification and is otherwise assumed to be empty. Assertions in this domain read live cloud configuration, admission policy, network reachability and identity state, and are the closest thing in the model to fully automatable implementation review.

B) Concentrate human review on the unassertable — Reserve human implementation review for the controls automation cannot assert and for the judgment calls automation cannot make, such as whether a control's implementation is adequate to its intent rather than merely present. Retain sampling on automatically verified assets so that the assertions themselves are checked, because an assertion that has silently stopped evaluating reports success indistinguishably from one that is passing. Track the share of reviewer hours spent on unassertable controls, which should rise as automation extends, and use it as the measure that the automation is releasing capacity rather than merely adding a second process alongside the first. What automation cannot assert here is whether a break-glass path is genuinely usable under incident conditions, which only an exercise establishes.

C) Measure review judgment by forward tracing — Trace every sign-off forward. Where a production incident is attributable to something implementation review examined and passed, record it as a review false negative and investigate whether the checklist, the assertion or the judgment was at fault. Report the false-negative rate to the sponsor alongside coverage and throughput, and treat it as the practice's primary quality measure, because coverage and timeliness are both achievable by a review function that examines nothing carefully. Run blind double-review on a sample of Critical-tier implementations quarterly and report the disagreement rate, using disagreements to refine checklists and assertions rather than to rank reviewers. Forward tracing in this domain most often implicates sign-offs taken against an approved plan rather than against the applied state, which diverge more often than teams expect.

Outcome metrics

Oversight metrics — Automated verification failures overridden by a human, with the override sustained across builds (detects: override becoming the routine path around a check that is inconvenient rather than wrong, which is how automated verification quietly becomes advisory) · Sign-offs later contradicted by a production incident attributable to what the review examined (detects: review that is procedurally complete and substantively wrong, which only forward tracing to incidents can reveal) · Blind double-review disagreement rate on Critical-tier implementations (detects: reviewer drift at the tier where it matters most, invisible unless deliberately sampled)

Success criteria

• At least 80% of verifiable controls carry continuous assertions in both pipeline and runtime, with unassertable controls listed and shrinking.
• Critical-tier assertion failures block merges or reach an owner within the day.
• Review false-negative rate is measured by forward tracing to incidents and reported to the sponsor as the practice's primary quality measure.
• At least 60% of reviewer hours go to controls automation cannot assert, and rising.
• Sampling continues on automatically verified assets so that a silently stopped assertion is detectable.

Assessment questions

1. Do at least 80% of verifiable controls carry continuous assertions running both in the pipeline and against the running system, with the unassertable remainder listed explicitly and reviewed quarterly for reduction? Evidence this in the Infrastructure domain against the go-live review record showing applied state rather than the approved plan.
2. Is review false-negative rate measured by tracing sign-offs forward to incidents and reported to the sponsor as the primary quality measure, rather than coverage and timeliness which a review examining nothing carefully can also achieve? For Infrastructure, the assessor should be shown the go-live review record showing applied state rather than the approved plan.
3. Is sampling maintained on automatically verified assets so that an assertion which has silently stopped evaluating is detectable, and does at least 60% of reviewer time go to controls automation cannot assert? In Infrastructure, ask to see the go-live review record showing applied state rather than the approved plan.

Common pitfalls

• ❌ Asserting only in the pipeline, which misses the drift that originates outside it and is where most silent disablement happens.
• ❌ Assuming a passing assertion is evaluating, when a stopped assertion reports success indistinguishably.
• ❌ Dropping sampling once automation is in place, removing the only check on the checks.
• ❌ Reporting coverage and timeliness as quality, both of which a review function that examines nothing carefully achieves easily.
• ❌ Leaving the unassertable control list unpublished, so the verification boundary is assumed empty and is not.

Security Testing (ST)

Objective: Baseline a foundational security test battery per infrastructure archetype that asserts the controls actually hold, and run it before every release.

A program at this level tests the AI-specific controls rather than assuming them. Each archetype has a foundational battery, and every test asserts a control holds rather than that a feature works. The battery covers an exposure test asserting that inference endpoints are unreachable from outside their declared boundary, an egress test asserting that a host cannot reach destinations outside its allowlist, a provenance test asserting that an unsigned or unregistered model artifact cannot be served, an isolation test asserting one tenant cannot observe or affect another, a quota and spend-limit test asserting the declared at-limit behaviour occurs, and a secrets test asserting credentials are not retrievable from the runtime environment. The discipline that distinguishes this practice from general quality assurance is that each test must be able to fail for the right reason: a test that passes because the condition it checks never arises provides assurance that is indistinguishable from real assurance on every dashboard.

Level 1, Foundational

Baseline a foundational security test battery per infrastructure archetype that asserts the controls actually hold, and run it before every release.

Activities

A) Build the foundational battery per archetype — Author a foundational test battery per archetype covering an exposure test asserting that inference endpoints are unreachable from outside their declared boundary, an egress test asserting that a host cannot reach destinations outside its allowlist, a provenance test asserting that an unsigned or unregistered model artifact cannot be served, an isolation test asserting one tenant cannot observe or affect another, a quota and spend-limit test asserting the declared at-limit behaviour occurs, and a secrets test asserting credentials are not retrievable from the runtime environment. Write each test as an assertion about a control rather than as a check on behaviour, and verify that each test can fail: introduce the condition deliberately once and confirm the test catches it, which is the only way to distinguish a test that asserts something from one that passes because its condition never arises. Keep the battery small enough to run on every build at the tiers that need it, because a battery too slow to run continuously will be moved to a nightly job and then to a weekly one and then to nowhere. Version the battery alongside the reference patterns so that a pattern change and its test change travel together.

B) Map tests to threats and publish the coverage gaps — Map each test to the threat identifiers from the TA library that it exercises, and publish the threats that have no test. The coverage map is what converts a test suite into an assurance argument: without it the battery asserts that the tests pass, which is not the same claim as that the threats are addressed. Report threat coverage rather than test count, since test count rises with effort while coverage rises only with effect. Treat an untested threat at a high-risk archetype as an open finding with an owner rather than as a backlog item, and feed coverage gaps back to the threat assessment practice so that the library and the battery evolve together.

C) Run before release, with a governed waiver path — Run the battery before every release, and trigger it on the changes that actually invalidate prior results, being an exposure or security-group change, a capacity or tenancy change, a promotion-pipeline change, and a base image or runtime upgrade. Failures block release for Critical- and High-tier assets and warn for the rest. Provide a governed waiver path, because an unwaivable gate is bypassed in ways nobody sees: a waiver needs a named authority, a stated reason, an expiry and automatic notification. Review waivers weekly and treat a test waived across more than one release cycle as a finding, since the common failure here is not a single waiver but a test quietly disabled to unblock a release and never re-enabled, which reduces coverage without anyone deciding to reduce it.

Outcome metrics

Oversight metrics — Failing tests waived to permit a release, with a named waiver authority (detects: waiver becoming the routine release path, where a failing security test is treated as a scheduling obstacle rather than a result) · Tests marked as skipped or quarantined for more than one release cycle (detects: silent test decay, where a test is disabled to unblock a release and never re-enabled, so coverage falls without any decision being taken)

Success criteria

• A foundational battery exists for at least 85% of inventoried infrastructure archetypes and runs before every release.
• Every test has been verified as able to fail by deliberately introducing its condition once.
• At least 75% of threats from the library are exercised by a test, with untested threats published as gaps carrying owners.
• Failures block release for Critical- and High-tier assets, with a governed waiver path carrying authority, reason and expiry.
• A test waived across more than one release cycle is raised as a finding rather than tolerated.

Assessment questions

1. Does a foundational security test battery exist for at least 85% of inventoried infrastructure archetypes, with each test written as an assertion about a control and verified as able to fail by deliberately introducing its condition once?
2. Is every test mapped to the TA threat identifiers it exercises, with threats having no test published as gaps and threat coverage reported rather than test count? For Infrastructure, the assessor should be shown the exposure test and the spend-limit at-limit behaviour result.
3. Does the battery run before every release and on the change types that invalidate prior results, with failures blocking Critical- and High-tier releases and a governed waiver path carrying named authority, reason and expiry? In Infrastructure, ask to see the exposure test and the spend-limit at-limit behaviour result.

Common pitfalls

• ❌ Writing tests that pass because their condition never arises, which are indistinguishable from real assurance on every dashboard.
• ❌ Reporting test count rather than threat coverage, since count rises with effort while coverage rises only with effect.
• ❌ Building a battery too slow to run on every build, which migrates to nightly, then weekly, then nowhere.
• ❌ Providing no waiver path, which produces bypasses nobody sees rather than compliance.
• ❌ Tolerating tests quietly disabled to unblock a release, which reduces coverage without any decision being taken.

Level 2, Comprehensive

Calibrate the infrastructure test battery to asset tier, run it continuously in CI, and add adversarial testing where the automated battery cannot reach.

Activities

A) Calibrate battery depth and gating to tier — Publish what each tier receives: Critical takes the full battery on every build with failures blocking, plus quarterly adversarial testing; High takes the full battery in CI with failures blocking; Medium takes a defined subset; Low takes spot checks. Derive the applicable battery from the inventory tier automatically rather than letting teams select, because selection under release pressure converges on the fastest subset. Publish gating behaviour per tier so that nobody negotiates it at release time. Where battery runtime makes per-build execution impractical at Critical tier, treat that as a test-performance problem to be solved rather than a reason to reduce frequency, since reduced frequency is how continuous testing becomes periodic testing. Adversarial work in this domain concentrates on reachability that the topology says should not exist and on credential paths through the control plane, which are structural findings automation can assert only where someone anticipated them.

B) Add adversarial testing where automation cannot reach — Schedule adversarial testing by people against Critical-tier assets at least quarterly, scoped from the threat model and executed by testers who did not build the asset. The purpose is specifically to find what the automated battery cannot: an automated suite tests the techniques it encodes and therefore converges, passing continuously while the attack surface moves underneath it. Measure the adversarial exercise by what it found that the battery missed, and feed every such finding back into the battery as a new test, so that the exercise permanently raises the automated floor rather than producing a report. Scope must be set by the testing party rather than the tested team, because a team that steers scope away from what it knows is weak produces a clean result that means nothing. The regression trigger that matters most here is any infrastructure-as-code change, because exposure is a property of the plan and changes without warning.

C) Keep corpora fresh and remove tests that assert nothing — Refresh regression corpora on a defined cadence, targeting 90 days, drawing new cases from published research, incidents, adversarial findings and threat-intelligence updates. A prompt-injection or adversarial-input corpus that has not changed in a year tests the techniques of a year ago and passes with increasing confidence as the real surface moves. Separately, audit the battery for tests that have never failed across a full quarter and examine each: some are genuinely stable controls and some assert nothing at all, and the two are indistinguishable from a dashboard. Remove or strengthen the latter. Trigger regression runs on an exposure or security-group change, a capacity or tenancy change, a promotion-pipeline change, and a base image or runtime upgrade in addition to the scheduled cadence. Infrastructure test harnesses contribute well because the platforms are shared and an exposure assertion transfers directly.

Outcome metrics

Oversight metrics — Critical-tier test waivers, with the waiver authority and expiry recorded (detects: waiver authority drifting downward under release pressure, so the decision to ship with a failing security test is taken by whoever is available) · Waived tests later implicated in a production incident (detects: waiver judgment that is procedurally correct and substantively wrong, which only forward tracing can reveal) · Test suites whose pass rate is 100% across every run for a full quarter (detects: tests that assert nothing, which are indistinguishable from passing tests on every dashboard and provide false assurance at exactly the tiers that need real assurance)

Success criteria

• Battery depth and gating behaviour are published per tier and derived automatically from the inventory.
• Every Critical-tier asset is exercised by adversarial testing at least annually, scoped by the testing party rather than the tested team.
• Every adversarial finding becomes a new automated test within 30 days, raising the automated floor permanently.
• Regression corpora are refreshed within 90 days, and triggered by the change types that invalidate prior results.
• Tests that have never failed across a quarter are audited for assertion strength and removed or strengthened.

Assessment questions

1. Is battery depth and gating behaviour published per tier and derived automatically from the inventory rather than selected by teams under release pressure? Evidence this in the Infrastructure domain against the exposure test and the spend-limit at-limit behaviour result.
2. Is every Critical-tier asset exercised by adversarial testing at least annually with scope set by the testing party, and does every adversarial finding become a new automated test within 30 days so the automated floor rises permanently? For Infrastructure, the assessor should be shown the exposure test and the spend-limit at-limit behaviour result.
3. Are regression corpora refreshed within 90 days and triggered by invalidating change types, and are tests that have never failed across a quarter audited for assertion strength rather than assumed to be stable controls? In Infrastructure, ask to see the exposure test and the spend-limit at-limit behaviour result.

Common pitfalls

• ❌ Letting the tested team scope the adversarial exercise, which produces a clean result that means nothing.
• ❌ Treating adversarial findings as a report rather than as new automated tests, so the same class of finding recurs next cycle.
• ❌ Leaving regression corpora static, which tests last year's techniques with rising confidence as the real surface moves.
• ❌ Reducing test frequency when the battery gets slow, rather than treating runtime as a performance problem to solve.
• ❌ Assuming a test that never fails is a stable control, when it is indistinguishable from one that asserts nothing.

Level 3, Industry-Leading

Run continuous adversarial testing against infrastructure AI assets, measure the battery against what people can still find, and contribute test corpora back to the field.

Activities

A) Run adversarial testing continuously — Move from scheduled engagements to continuous adversarial operation against Critical-tier assets, whether by a standing internal team, a contracted continuous service or an automated adversarial harness that evolves its own cases. Measure the practice by time-to-find rather than by engagement count, and track the surprise rate, being the share of findings the automated battery should have caught but did not. A falling surprise rate indicates the battery is converging on reality; a flat one at high volume indicates the battery has converged on itself. Feed every finding into the battery within days rather than within the 30-day window that L2 allowed, since continuous operation makes the slower loop the binding constraint. Adversarial work in this domain concentrates on reachability that the topology says should not exist and on credential paths through the control plane, which are structural findings automation can assert only where someone anticipated them.

B) Close the automation gap deliberately — Treat the gap between what automation catches and what humans still find as the practice's primary work item rather than as an inevitable residue. For each human finding, ask specifically why the battery missed it: a missing case, a missing technique class, an assertion too weak, or a surface the harness cannot reach. Record the category, because the distribution tells you whether to extend the corpus, strengthen assertions or rebuild the harness, and these are very different investments. Publish the gap measure alongside coverage so that a rising coverage number with a static gap is visible as the illusion it represents. The regression trigger that matters most here is any infrastructure-as-code change, because exposure is a property of the plan and changes without warning.

C) Contribute corpora and harnesses back — Contribute at least four substantive artifacts per year, weighted toward runnable form: adversarial corpora, test harnesses, assertion libraries and reproducible technique demonstrations. Shared adversarial corpora are the scarcest resource in AI security and the one where contribution most directly raises the whole field's floor, because a corpus a peer can run is worth far more than a paper describing what it contains. Work through OWASP projects, MITRE ATLAS, OpenSSF AI and sector bodies. Anonymise, legally review and coordinate disclosure with affected providers before release. Count only accepted artifacts, and prioritise contributing the cases that most often catch real defects internally, since those carry the most evidence. Infrastructure test harnesses contribute well because the platforms are shared and an exposure assertion transfers directly.

Outcome metrics

Oversight metrics — Adversarial findings that the automated battery should have caught but did not (detects: a battery that has converged on what it already tests, which passes continuously while the real attack surface moves) · Automated test results overridden at merge, with the override sustained across builds (detects: override becoming the normal path around an inconvenient check, which converts a gate into advice without any decision) · Share of red-team scope determined by the team being tested (detects: scope capture, where the tested team steers the exercise away from what it knows is weak, which makes a clean result meaningless)

Success criteria

• Critical-tier assets are under continuous adversarial testing, measured by time-to-find rather than engagement count.
• The surprise rate is tracked and falling, and reported alongside coverage so that rising coverage with a static gap is visible.
• Human findings reach the automated battery within 5 business days.
• New public techniques reach the corpus within 14 days at the 90th percentile.
• At least four accepted external contributions per year, the majority runnable corpora or harnesses, disclosure-coordinated.

Assessment questions

1. Is adversarial testing continuous against Critical-tier assets rather than scheduled, measured by time-to-find and surprise rate rather than by engagement count? Evidence this in the Infrastructure domain against the exposure test and the spend-limit at-limit behaviour result.
2. Is the automation gap treated as the primary work item, with each human finding categorised as a missing case, missing technique class, weak assertion or unreachable surface so that investment goes to the right remedy? For Infrastructure, the assessor should be shown the exposure test and the spend-limit at-limit behaviour result.
3. Do new public techniques reach the corpus within 14 days at the 90th percentile, and does the organisation contribute at least four accepted artifacts per year in majority-runnable form with disclosure coordinated? In Infrastructure, ask to see the exposure test and the spend-limit at-limit behaviour result.

Common pitfalls

• ❌ Measuring adversarial testing by engagement count, which rewards activity and says nothing about whether defects are found faster.
• ❌ Treating the automation gap as an inevitable residue rather than as the work, which leaves the battery converged on itself.
• ❌ Reporting coverage without the surprise rate, so a rising coverage number conceals a static real-world gap.
• ❌ Contributing papers rather than runnable corpora, which peers read and cannot execute.
• ❌ Releasing technique demonstrations touching a provider's systems without coordinating disclosure first.

Business Function: Operations

Environment Hardening (EH)

Objective: Publish and apply a hardening baseline for each infrastructure AI archetype, so the operating environment enforces the controls rather than documenting them.

A program at this level has a published hardening baseline per archetype and applies it. The baseline covers no standing human access with time-bound elevation, egress allowlisting from inference hosts, per-tenant isolation, signed artifacts enforced at promotion, secrets in a vault with automated rotation, host and container hardening baselines, and control-plane access restricted to a named break-glass path. The distinction that matters is enforcement: a control in enforcing mode changes what can happen, and a control in audit mode changes only what is recorded, and the two are frequently confused in reporting. Consumption limits are part of the baseline rather than an operational afterthought, because an environment with no limit has no defence against the one attack that leaves every availability signal green.

Level 1, Foundational

Publish and apply a hardening baseline for each infrastructure AI archetype, so the operating environment enforces the controls rather than documenting them.

Activities

A) Publish the hardening baseline per archetype — Author a hardening baseline per archetype covering no standing human access with time-bound elevation, egress allowlisting from inference hosts, per-tenant isolation, signed artifacts enforced at promotion, secrets in a vault with automated rotation, host and container hardening baselines, and control-plane access restricted to a named break-glass path. Express the baseline as configuration that can be applied and checked rather than as prose, and publish it alongside the reference architecture patterns so that the secure configuration travels with the secure design. State for every control whether it is enforcing or advisory, and default to enforcing, because a baseline whose enforcement status is unstated is assumed to be enforcing by everyone reporting on it and is frequently not. Where a control genuinely cannot be enforced in the current environment, record it as a known limitation rather than listing it in the baseline as though it were active.

B) Apply the baseline and measure conformance against the running environment — Apply the baseline to every asset of the archetype and measure conformance from the environment rather than from a checklist. Read configuration state, policy enforcement mode and runtime behaviour, and report conformance per control rather than as a single score, because one aggregate number lets a fully conformant set of easy controls conceal a missing hard one. Report separately the count of controls running in audit rather than enforcing mode, which is the most common way a baseline is nominally met and practically absent. Reconcile monthly and treat any asset with no baseline applied as a finding with an owner rather than as an onboarding backlog item.

C) Configure consumption limits and govern exceptions — Configure per-tenant request and spend limits with a declared at-limit behaviour as part of the baseline for every asset, expressed relative to per 1k inference requests so that the limit bears a defensible relationship to the work the asset actually does. Declare the behaviour at the limit explicitly, being degrade, queue or refuse, and never silently continue, because a limit that is exceeded without effect is a reporting artefact rather than a control. Operate a baseline exception process: any control not applied requires an owner, a compensating control, an approver and an expiry, reviewed monthly, with expired exceptions treated as findings. Exceptions are healthy when visible and corrosive when absorbed into the environment as the new normal.

Outcome metrics

Oversight metrics — Baseline exceptions approved without a compensating control recorded (detects: exception approval becoming a formality, where the control is waived and nothing replaces it, which is indistinguishable from not having the control at all) · Controls switched from enforcing to audit mode, with the change approved and time-bounded (detects: silent downgrade, where a control is moved to audit mode to resolve an operational problem and left there, so the baseline reports as met while enforcing nothing)

Consumption metrics — Assets with a consumption limit configured and a declared at-limit behaviour per per production asset · Configured limit against observed consumption per per 1k inference requests · Limit breaches where the declared at-limit behaviour did not occur per per limit breach event

Success criteria

• A hardening baseline is published per archetype as applicable configuration rather than prose, and applied to at least 90% of production infrastructure AI assets.
• At least 90% of baseline controls run in enforcing mode, with every audit-mode control recorded as an exception with an owner.
• Every production asset has a consumption limit configured with an explicit, declared at-limit behaviour.
• Conformance is reported per control rather than as an aggregate score that can conceal a missing hard control.
• Every baseline exception carries an owner, compensating control, approver and expiry, with none past expiry.

Assessment questions

1. Is a hardening baseline published per infrastructure archetype as applicable configuration rather than prose, applied to at least 90% of production assets, and measured against the running environment rather than a checklist?
2. Are at least 90% of baseline controls in enforcing rather than audit mode, with audit-mode controls recorded as exceptions, and is conformance reported per control rather than as an aggregate that can conceal a missing hard control? For Infrastructure, the assessor should be shown the identity review showing no standing human access to a production fleet.
3. Does every production asset have per-tenant request and spend limits with a declared at-limit behaviour configured with an explicit declared at-limit behaviour, and does every baseline exception carry an owner, compensating control, approver and expiry?

Common pitfalls

• ❌ Leaving enforcement status unstated, so controls in audit mode are reported and believed to be enforcing.
• ❌ Reporting a single aggregate conformance score, which lets a set of easy controls conceal a missing hard one.
• ❌ Measuring conformance from a checklist rather than from the running environment, which records intent rather than state.
• ❌ Configuring a limit without declaring the behaviour at the limit, producing a threshold that is crossed without effect.
• ❌ Allowing exceptions without compensating controls, which is indistinguishable from not having the control.

Level 2, Comprehensive

Calibrate infrastructure hardening intensity to asset tier, detect and remediate baseline drift continuously, and make consumption limits tier-proportionate and tested.

Activities

A) Publish tier-calibrated baselines — Publish a baseline variant per tier rather than one baseline with exceptions. Critical adds per-tenant isolation, no standing human access with time-bound elevation, shorter secret rotation intervals, stricter egress allowlisting and full logging; Low takes single sign-on and the essential controls. Deriving the applicable variant from the inventory tier removes the negotiation that otherwise happens per asset and resolves toward the lighter variant. Publish the reasoning alongside the variants so that a team subject to the Critical baseline understands why, because a baseline that arrives without justification is complied with minimally and circumvented where possible. Drift in this domain most often appears as a security group widened during an incident and a quota raised under load, both defensible at the time and neither routinely reverted.

B) Detect and remediate drift continuously — Stand up continuous drift detection across the baseline controls for Critical- and High-tier assets, reading configuration state, policy enforcement mode and runtime behaviour, and alerting on change rather than on schedule. Publish remediation windows by tier and severity, tighter at Critical, and route detections into the shared issue backlog with tier-appropriate severity rather than into a dashboard. Drift is the defining failure of hardening at scale because environments are hardened once and then changed for operational reasons that are individually reasonable: a rule widened for a debugging session, a quota raised under load, an enforcement mode relaxed to unblock a deployment. Each is defensible and none is reverted unless something notices. Limit exercises in this domain drive an endpoint to its per-tenant spend limit and confirm shedding or refusal actually occurs, which is the only defence against a drain that leaves availability untouched.

C) Make limits tier-proportionate and exercise them — Set consumption limits proportionate to tier and to the asset's observed workload, expressed relative to per 1k inference requests. Exercise the at-limit behaviour at least annually for Critical- and High-tier assets by driving the asset to its limit in a controlled test and confirming that the declared behaviour actually occurs, because a limit whose behaviour has never been observed is an assumption. Record the exercise result. Treat a cost-driven change to a limit, a logging level, a retention period or an enforcement mode on a Critical-tier asset as a material change requiring approval, since these are precisely the controls that cost pressure erodes and the erosion is invisible on any security dashboard. Cost pressure in this domain lands first on reserved capacity and telemetry sampling, the second of which makes monitoring statistical at the tier least able to afford it.

Outcome metrics

Oversight metrics — Time-bound elevations to Critical-tier environments that expired without being reviewed (detects: elevation becoming effectively standing access, where a time bound exists on paper and is renewed automatically without anyone assessing whether it is still needed) · Drift remediation deferrals at Critical tier, with a named acceptance authority (detects: deferral becoming the default response to drift under operational pressure, which converts continuous detection into continuous reporting) · Enforcement-mode relaxations approved without an expiry (detects: a temporary relaxation becoming permanent because nothing in the approval obliges anyone to revisit it)

Consumption metrics — Critical- and High-tier assets whose at-limit behaviour has been exercised and recorded per per Critical or High tier asset · Limit headroom: configured limit against observed peak consumption, by tier per per 1k inference requests · Cost-driven relaxations of limits, logging or retention on Critical-tier assets per per approved change

Success criteria

• A baseline variant per tier is published with its reasoning and derived automatically from the inventory.
• At least 95% of Critical-tier assets are under continuous drift detection, with detection within 3 business days.
• Standing human access to Critical-tier AI environments is zero, with time-bound elevation only.
• Every Critical-tier asset's at-limit behaviour has been exercised within 12 months and the result recorded.
• Cost-driven changes to limits, logging, retention or enforcement mode at Critical tier require approval as material changes.

Assessment questions

1. Is a baseline variant published per tier with its reasoning and derived automatically from the inventory, rather than one baseline with per-asset exceptions that resolve toward the lighter option? Evidence this in the Infrastructure domain against the identity review showing no standing human access to a production fleet.
2. Are at least 95% of Critical-tier assets under continuous drift detection with a 3 business day detection window, with detections routed into the shared backlog rather than a dashboard, and is standing human access to Critical-tier environments zero? For Infrastructure, the assessor should be shown the identity review showing no standing human access to a production fleet.
3. Has every Critical-tier asset's at-limit behaviour been exercised within 12 months, and do cost-driven changes to limits, logging, retention or enforcement mode require approval as material changes? In Infrastructure, ask to see the identity review showing no standing human access to a production fleet.

Common pitfalls

• ❌ Running one baseline with per-asset exceptions rather than tier variants, which turns every asset into a negotiation resolving toward the lighter option.
• ❌ Detecting drift on a schedule rather than on change, which reintroduces the point-in-time weakness the activity exists to remove.
• ❌ Renewing time-bound elevations automatically, which produces standing access with a time bound written on it.
• ❌ Declaring limits without exercising them, so their behaviour is first learned during the event they exist for.
• ❌ Exempting cost-driven changes from approval, which is how hardened environments degrade with everyone's agreement.

Level 3, Industry-Leading

Make infrastructure AI environments self-correcting, with baselines applied at provisioning and drift remediated automatically where it is safe to do so.

Activities

A) Apply baselines at provisioning — Move baseline application to provisioning time so that an asset is conformant from creation rather than hardened afterwards. Implement the baseline as infrastructure-as-code modules, admission policies and platform defaults, so the conformant configuration is the default path and a non-conformant asset requires deliberate effort to create. Block provisioning of Critical-tier assets that would not meet baseline. Measure the time an asset spends non-conformant after creation and drive it to zero, since that window is the one period in an asset's life when it is both unhardened and unmonitored, and it is the window most exploitation of new infrastructure actually uses. Provisioning-time conformance in this domain is the most complete in the model, since admission policy can simply refuse a non-conformant deployment before it exists.

B) Automate remediation within enumerated bounds — Enumerate explicitly the drift classes that may be remediated automatically, being those where the correct state is unambiguous and reversion carries no operational risk, such as re-enabling logging, restoring an enforcement mode, or closing an egress rule not in the allowlist. Everything outside that list alerts to a human. Log every automated action with what changed, what it was reverted to and why, and make the log reviewable, because self-correction that is silent can mask a genuine operational need indefinitely and the team whose change was reverted will simply make it again. Track repeated remediation of the same control on the same asset as a signal that the baseline is wrong or the operational need is real, and route it to review rather than continuing to revert. Safe automatic remediation in this domain covers closing an egress rule outside the allowlist and restoring an enforcement mode, both of which are frequently widened during incidents and rarely reverted.

C) Govern adaptive limits and publish the automation boundary — Allow consumption limits to adapt to observed workload within governed bounds, so that a limit tracks the asset's real demand rather than remaining static and becoming either irrelevant or obstructive. Set a hard ceiling that adaptation cannot cross without human approval, because an adaptive limit with no ceiling accommodates an attack as readily as it accommodates growth, and that is precisely the failure mode denial-of-wallet exploits. Express bounds relative to per 1k inference requests. Publish the list of controls that cannot be automatically applied or remediated and review it quarterly for reduction, so that the boundary of the automation is known rather than assumed to be complete. Adaptive limits in this domain track cost per thousand inference requests, with a hard ceiling that a genuine traffic increase must cross by approval rather than by growth.

Outcome metrics

Oversight metrics — Automated remediations reverted by a human within 24 hours (detects: automation acting against a genuine operational need, which indicates the enumerated safe class was wrongly scoped and is invisible unless reversions are tracked) · Adaptive limit ceiling increases approved, with the approver and justification recorded (detects: ceiling creep, where each individual increase is justified and the cumulative effect is an unbounded limit nobody decided to create) · Alerts for non-automatable drift acknowledged but not remediated within the tier window (detects: alert fatigue on the drift classes that specifically require human judgment, which are the ones automation deliberately left to people)

Consumption metrics — Adaptive limits operating within a governed hard ceiling per per asset with an adaptive limit · Cumulative ceiling increase over the trailing year, per asset per per 1k inference requests · Drift events on consumption-limit configuration, and time spent drifted per per limit configuration drift event

Success criteria

• At least 95% of new assets are provisioned already conformant, with non-conformance windows at or under one hour.
• The drift classes eligible for automatic remediation are enumerated explicitly, and everything outside alerts to a human.
• Every automated remediation is logged with change, reversion and reason, and is reviewable.
• Repeated remediation of the same control is treated as a signal and routed to review rather than continuing to revert.
• Adaptive limits operate within a governed hard ceiling that requires human approval to cross.

Assessment questions

1. Are at least 95% of new assets provisioned already conformant with non-conformance windows at or under one hour, and is provisioning of non-conformant Critical-tier assets blocked? Evidence this in the Infrastructure domain against the identity review showing no standing human access to a production fleet.
2. Are the drift classes eligible for automatic remediation enumerated explicitly with everything else alerting to a human, and is every automated action logged with change, reversion and reason and reviewable rather than silent? For Infrastructure, the assessor should be shown the identity review showing no standing human access to a production fleet.
3. Is repeated automatic remediation of the same control treated as a signal routed to baseline or operational review, and do adaptive limits operate within a governed hard ceiling requiring human approval to cross? In Infrastructure, ask to see the identity review showing no standing human access to a production fleet.

Common pitfalls

• ❌ Hardening after provisioning rather than at it, leaving a window where the asset is both unhardened and unmonitored.
• ❌ Automating remediation without enumerating the safe classes, which eventually reverts something that mattered operationally.
• ❌ Making self-correction silent, so the team whose change was reverted simply makes it again and nobody learns.
• ❌ Treating repeated automatic remediation as a success metric rather than as evidence the baseline or the operational need is wrong.
• ❌ Allowing adaptive limits without a hard ceiling, which accommodates an attack as readily as growth.

Issue Management (IM)

Objective: Run one backlog for infrastructure AI issues with an AI-aware severity taxonomy and published service levels, so AI findings age and escalate like everything else.

A program at this level handles AI issues on the same rails as every other security issue, with a severity taxonomy extended to cover the failure modes AI actually produces. The taxonomy names an inference endpoint exposed beyond its boundary, an unregistered model served in production, control-plane credential compromise, shadow accelerator workloads, and denial-of-wallet at the serving layer, because a severity scheme built for classical vulnerabilities rates these either as nothing or as whatever the assessor improvises. One backlog matters more than it sounds: AI findings kept in a separate register age quietly, escalate to nobody, and are rediscovered at the next assessment.

Level 1, Foundational

Run one backlog for infrastructure AI issues with an AI-aware severity taxonomy and published service levels, so AI findings age and escalate like everything else.

Activities

A) Extend the severity taxonomy to AI failure modes — Extend the existing severity taxonomy rather than writing a new one, adding the AI-specific classes that a classical scheme cannot rate: an inference endpoint exposed beyond its boundary, an unregistered model served in production, control-plane credential compromise, shadow accelerator workloads, and denial-of-wallet at the serving layer. For each class, state what makes it Critical, High, Medium or Low, using the dimensions that actually vary, being the data exposed, the reversibility of the effect, the population affected and whether the asset acts or advises. Include denial-of-wallet as a named class with its own criteria, because it degrades no availability signal and a scheme keyed to availability rates it as nothing while the budget drains. Publish worked examples per class, since a taxonomy without examples is applied inconsistently and its inconsistency is invisible in the aggregate.

B) Run one backlog with wired-in sources and service levels — Route every AI finding into the shared security backlog rather than a separate AI register. Wire the sources so findings arrive automatically: implementation review, security testing, monitoring detections, adversarial exercises, vendor advisories and self-reports. Assign a severity, an owner and a service level at creation rather than at triage, because an issue that waits for triage to acquire a service level is an issue whose clock has not started. Publish service levels by severity and report adherence rather than backlog size, since backlog size is a function of discovery rate as much as of remediation and improves when discovery stops.

C) Document containment for AI-specific incidents — Write containment actions for each AI-specific incident class before they are needed, covering withdrawing the endpoint from the load balancer, rotating control-plane credentials, isolating the affected namespace, and applying an emergency spend or rate limit. AI containment differs from classical containment in a way that catches teams out: the fastest effective action is usually to remove the system's ability to act rather than to take the system offline, because an AI asset that is merely unavailable stops delivering value while an asset whose tool access is revoked stops causing harm and may keep delivering. Name who is authorised to invoke each action at what hour, and verify the technical means exists and works. An action documented but never exercised is discovered to be unavailable at the worst moment.

Outcome metrics

Oversight metrics — Issues closed without a recorded remediation or acceptance (detects: closure as backlog hygiene rather than resolution, where an issue is marked done because it is old rather than because anything changed) · Severity downgrades applied after creation, with the downgrading authority recorded (detects: severity drift under service-level pressure, where reclassification is used to make a clock more comfortable rather than to correct an assessment)

Consumption metrics — Denial-of-wallet incidents detected, with time from onset to containment per per denial-of-wallet incident · Consumption anomaly cost accrued before containment per per 1k inference requests · AI incidents whose containment action was a consumption limit rather than a service withdrawal per per contained AI incident

Success criteria

• All AI issues are managed in the shared security backlog with the same ageing and escalation as other security issues.
• The severity taxonomy names the AI-specific classes including denial-of-wallet, with published criteria and worked examples.
• At least 95% of issues carry a severity, owner and service level at creation rather than at triage.
• At least 80% of issues arrive from a wired source rather than manual entry.
• Containment actions are documented per AI incident class and technically verified as available.

Assessment questions

1. Does the severity taxonomy extend the existing scheme with AI-specific classes including an inference endpoint exposed beyond its boundary and denial-of-wallet, with published criteria keyed to data exposed, reversibility, population affected and whether the asset acts or advises, plus worked examples per class?
2. Are all AI issues managed in the shared security backlog with at least 95% carrying severity, owner and service level at creation, and do at least 80% arrive from a wired source rather than manual entry? For Infrastructure, the assessor should be shown an incident record showing endpoint withdrawal and emergency limit application.
3. Are containment actions documented for every AI incident class, covering withdrawing the endpoint from the load balancer and the others, with named invocation authority and technical availability verified rather than assumed?

Common pitfalls

• ❌ Keeping AI findings in a separate register, where they age quietly, escalate to nobody and are rediscovered at the next assessment.
• ❌ Using a severity scheme keyed to availability, which rates denial-of-wallet as nothing while the budget drains.
• ❌ Assigning severity at triage rather than creation, so the issue's clock does not start until someone looks at it.
• ❌ Reporting backlog size rather than service-level adherence, a number that improves when discovery stops.
• ❌ Documenting containment actions without verifying the technical means exists, which is discovered at the worst moment.

Level 2, Comprehensive

Calibrate infrastructure AI issue service levels to asset tier, run rehearsed AI incident response, and close the loop from root cause to the practice that owns it.

Activities

A) Calibrate service levels and authority to tier — Publish remediation service levels by severity and asset tier, tighter at Critical, and report adherence per tier rather than in aggregate, because an aggregate figure can be met while every Critical-tier breach is missed. Make deferral and risk-acceptance authority rise with tier and severity, so a deferred Critical finding requires executive acceptance with an expiry and quarterly review. Report deferrals to the sponsor alongside closures, since a practice reporting only what it closed describes a backlog that does not exist. Track findings deferred and later implicated in an incident as a distinct measure of whether deferral judgment is sound rather than merely authorised. Tier-proportionate response in this domain is shaped by blast radius, since a control-plane compromise reaches every asset the plane commands and the clock should reflect that rather than the asset count.

B) Rehearse AI incident response and instrument the notification clocks — Write and exercise incident playbooks for each AI-specific class, covering detection signals, containment using withdrawing the endpoint from the load balancer, rotating control-plane credentials, isolating the affected namespace, and applying an emergency spend or rate limit, evidence preservation, communication and recovery. Exercise at least semi-annually with the people who would actually respond, including out of hours, and measure the exercise by the gaps it finds rather than by its completion. Instrument the regulatory notification clocks explicitly: identify which incident classes trigger EU AI Act serious-incident reporting or GDPR breach notification, what starts the clock, who decides, and what evidence the notification requires. A team that discovers a 72-hour obligation 40 hours into an incident has lost most of the window to a question it could have answered in advance. Playbook exercises in this domain rehearse withdrawing an endpoint and rotating control-plane credentials, which is the action most likely to be technically available and organisationally frightening.

C) Route root causes to the owning practice — Require every incident reaching a root cause to name the practice that owns the fix and to open a work item there, rather than closing the incident where it was found. An incident caused by a missing architectural control belongs to secure architecture, one caused by an untested threat belongs to security testing, one caused by a knowledge gap belongs to education. Closing it in the incident record alone guarantees recurrence, and recurrence is the single clearest signal that a response practice is functioning as a repair shop rather than as a feedback mechanism. Track the share of incidents whose root cause was already known and untreated, which separates a discovery problem from an action problem and is usually uncomfortable reading. Root causes in this domain most often belong to environment hardening, specifically drift that was detected and deferred.

Outcome metrics

Oversight metrics — Incidents closed without a root cause recorded, by severity (detects: closure without diagnosis, which guarantees recurrence and is most common under load at exactly the times recurrence is least affordable) · Risk acceptances at Critical tier, with the accepting authority and expiry recorded (detects: acceptance authority drifting downward under pressure, so the decision to live with a Critical risk is taken by whoever is available) · Exercise findings closed without a change to the playbook or the environment (detects: rehearsal treated as a compliance activity rather than as a source of correction, which is visible only by tracking what the exercise changed)

Consumption metrics — Denial-of-wallet incidents contained within their tier service level per per denial-of-wallet incident · Cost accrued between detection and containment, by tier per per 1k inference requests · Emergency consumption limits applied during incidents and subsequently reverted per per applied emergency limit

Success criteria

• Service levels vary by tier and severity with adherence at or above 90% reported per tier rather than in aggregate.
• Every AI incident playbook has been exercised within 12 months, including an out-of-hours exercise, and measured by the gaps it found.
• Regulatory notification clocks are instrumented with owners and evidence requirements identified in advance.
• At least 90% of incidents reaching root cause open a work item in the practice that owns the fix.
• Incidents whose root cause was already known and untreated are tracked openly and falling.

Assessment questions

1. Do service levels vary by asset tier and severity with adherence reported per tier rather than in an aggregate that can be met while every Critical-tier breach is missed? Evidence this in the Infrastructure domain against an incident record showing endpoint withdrawal and emergency limit application.
2. Have AI incident playbooks been exercised within 12 months including out of hours, measured by the gaps found, and are regulatory notification clocks instrumented in advance with owners and evidence requirements identified? For Infrastructure, the assessor should be shown an incident record showing endpoint withdrawal and emergency limit application.
3. Do at least 90% of incidents reaching root cause open a work item in the owning practice rather than closing where they were found, and is the share whose root cause was already known and untreated tracked openly? In Infrastructure, ask to see an incident record showing endpoint withdrawal and emergency limit application.

Common pitfalls

• ❌ Reporting service-level adherence in aggregate, which can be met while every Critical-tier breach is missed.
• ❌ Writing playbooks without exercising them, so their gaps are found during the incident rather than before it.
• ❌ Discovering a notification obligation mid-incident, by which point most of the window is gone.
• ❌ Closing incidents where they were found, which guarantees the same root cause returns.
• ❌ Reporting closures without deferrals, which describes a backlog that does not exist.

Level 3, Industry-Leading

Automate infrastructure AI issue triage and containment where it is safe, measure response by outcome rather than activity, and contribute incident learning back to the field.

Activities

A) Automate triage and bounded first-line containment — Automate severity assignment, ownership routing and duplicate detection from the detection signal, so an issue arrives with its clock started and its owner assigned. Enumerate explicitly the containment actions that may fire automatically, being those where the correct response is unambiguous and reversible, such as applying an emergency consumption limit, revoking a specific tool credential or blocking a destination. Everything else alerts. Log every automated action with what was done, on what evidence, and how to reverse it. Publish the human-judgment classes explicitly, because automated containment is safest when its boundary is stated rather than assumed, and the assumption is always that it covers more than it does. Automatic containment in this domain is the best bounded in the model: withdrawing an endpoint from a load balancer, applying an emergency limit and closing an egress rule are all unambiguous, fast and reversible.

B) Measure response by outcome rather than activity — Shift the primary measures from throughput to outcome: recurrence rate by root-cause class, harm avoided estimated from the consumption ledger and the affected population, and time from detection to containment rather than to closure. Ticket throughput and backlog size remain as health indicators but are explicitly not the practice's outcome measures, because both are dominated by discovery rate and both improve when discovery degrades. Report recurrence to the sponsor as the headline, since a response practice that closes quickly and recurs constantly is failing at the thing it exists for while every throughput metric looks excellent. Recurrence in this domain clusters around exposure changes made during incidents and never reverted, which points remediation at environment hardening rather than at detection.

C) Contribute anonymised incident learning back — Contribute at least four substantive artifacts per year: anonymised incident case studies, detection and containment patterns, playbooks others can adopt, and technique observations submitted to MITRE ATLAS where they fit. The public corpus of real AI security incidents is thin, disproportionately so relative to the deployment base, which means most organisations learn only from their own incidents, which is the slowest and most expensive way to learn anything. Contributing changes that for everyone including the contributor. Anonymise carefully, review legally, coordinate disclosure with any affected vendor or provider, and count only accepted artifacts. Prefer contributing the incidents that were most surprising internally, since those carry the most information for others. Infrastructure incident write-ups contribute well as exposure and drift case studies with the configuration detail included.

Outcome metrics

Oversight metrics — Automated containment actions reversed by a human within one hour (detects: automation acting on a misread signal, which indicates the enumerated class was wrongly scoped and is only visible if reversals are tracked separately from actions) · Incidents in human-judgment classes acknowledged but not actioned within the tier window (detects: alert fatigue concentrating precisely on the classes automation deliberately left to people because they need judgment) · Recurrence accepted without a corresponding work item in the owning practice (detects: recurrence normalised as a cost of doing business, which is how a response practice becomes a repair shop)

Consumption metrics — Automated emergency consumption limits applied within the detection window per per denial-of-wallet detection · Cost accrued between detection and automated containment per per 1k inference requests · Harm avoided from contained consumption incidents, computed from burn rate and time to contain per per contained consumption incident

Success criteria

• At least 85% of issues arrive auto-triaged with severity, owner and clock assigned.
• Automatic containment classes are enumerated explicitly, everything else alerts, and every automated action is logged with evidence and a reversal path.
• Recurrence rate by root-cause class is the practice's headline measure, reported to the sponsor, rather than throughput or backlog size.
• Harm avoided is estimated from the consumption ledger and affected population and reported annually.
• At least four accepted external incident-learning contributions per year, anonymised and disclosure-coordinated.

Assessment questions

1. Are at least 85% of issues auto-triaged with severity, owner and clock assigned, and are automatic containment classes enumerated explicitly with everything else alerting and every action logged with evidence and a reversal path? Evidence this in the Infrastructure domain against an incident record showing endpoint withdrawal and emergency limit application.
2. Is recurrence rate by root-cause class the practice's headline outcome measure reported to the sponsor, rather than throughput and backlog size which are dominated by discovery rate and improve when discovery degrades? For Infrastructure, the assessor should be shown an incident record showing endpoint withdrawal and emergency limit application.
3. Does the organisation contribute at least four accepted anonymised incident-learning artifacts per year with disclosure coordinated, prioritising the incidents that were most surprising internally? In Infrastructure, ask to see an incident record showing endpoint withdrawal and emergency limit application.

Common pitfalls

• ❌ Automating containment without enumerating the safe classes, which eventually acts on a misread signal in a way that matters.
• ❌ Measuring the practice by throughput, which is dominated by discovery rate and improves when discovery degrades.
• ❌ Tracking automated actions without tracking reversals, which hides automation that is acting wrongly.
• ❌ Normalising recurrence as a cost of doing business, which is how a response practice becomes a repair shop.
• ❌ Withholding incident learning, which keeps the whole field, including the withholder, learning only from its own incidents.

Monitoring & Logging (ML)

Objective: Establish a logging baseline per infrastructure archetype and a detection set covering the top threats, with retention set by the longest applicable obligation.

A program at this level can see what its AI is doing. Each archetype has a logging baseline covering inference requests with tenant and model identity, model registry and promotion events, accelerator scheduling and utilisation, control-plane administrative actions, egress connections from inference hosts, and secrets access events, and a detection set targeting the threats the threat library rates highest. Retention is set by the longest applicable regulatory obligation rather than by storage cost, because the retention decision is usually made on cost grounds by people who do not know the obligation and is discovered to be wrong during an investigation.

Level 1, Foundational

Establish a logging baseline per infrastructure archetype and a detection set covering the top threats, with retention set by the longest applicable obligation.

Activities

A) Publish and verify the logging baseline per archetype — Define a logging baseline per archetype covering inference requests with tenant and model identity, model registry and promotion events, accelerator scheduling and utilisation, control-plane administrative actions, egress connections from inference hosts, and secrets access events. Verify that each stream is actually emitting rather than merely configured, by asserting the presence of recent events per stream per asset, because a logging configuration that exists and emits nothing passes every configuration check and fails the one investigation that needs it. Include the identity and context fields that make a log answerable: who initiated, what asset, which model and version, which session. A log that records that something happened without recording which model version did it cannot answer the question that follows any output-integrity incident. Publish the baseline alongside the hardening baseline so telemetry travels with configuration.

B) Build the detection set from the threat library — Build detections for the threats the TA library rates highest, covering in this domain an inference endpoint becoming reachable beyond its boundary, an unregistered artifact served, control-plane action outside a change window, egress to a destination outside the allowlist, and inference cost per thousand requests departing from the endpoint baseline. Map each detection to the threat identifiers it addresses and publish the threats with no detection, because coverage is the claim that matters and detection count is not a proxy for it. Route every detection to the shared issue backlog with a severity from the IM taxonomy rather than to a dashboard, since a detection whose only destination is a dashboard has no owner and no clock. Include at least one consumption-anomaly detection from the outset, expressed against a denominated metric, because it is the cheapest detection available and the only one that covers denial-of-wallet.

C) Set retention by obligation and make logs usable — Set retention per log class by the longest applicable regulatory, contractual or investigative requirement, and record which obligation drives each period so the decision is reviewable. Retention is routinely set on storage cost by people who do not hold the obligation, and the error surfaces during an investigation or a regulator request when it can no longer be corrected. Apply redaction at write rather than at read so that sensitive content is not retained in the clear pending a query-time control. Verify that logs are actually queryable at the volumes an investigation would need, because a retention period that cannot be searched within a working day is not meaningfully retained.

Outcome metrics

Oversight metrics — Detections acknowledged but closed without an investigation note (detects: alert closure as queue management rather than as analysis, which is invisible in any alert-volume metric and is the normal response to a noisy detection set) · Detections muted or suppressed for more than one week, with the suppressing authority recorded (detects: silent suppression, where a noisy detection is muted to clear a queue and never re-enabled, so coverage falls with no decision taken)

Consumption metrics — Consumption-anomaly detections active against a denominated metric per per archetype detection set · Detection threshold expressed against consumption per unit of work rather than absolute spend per per 1k inference requests · Logging and retention cost per retained record, by log class per per retained record

Success criteria

• A logging baseline is defined per archetype and verified as emitting for at least 85% of infrastructure archetypes.
• At least 70% of top-rated library threats are covered by a detection, with uncovered threats published as gaps.
• Every log class has a retention period traced to a named obligation rather than to storage cost.
• Every active detection routes to the shared issue backlog with an IM severity rather than to a dashboard.
• At least one consumption-anomaly detection operates against a denominated metric from the outset.

Assessment questions

1. Is a logging baseline defined per infrastructure archetype covering identity, asset, model and version context, and verified as actually emitting rather than merely configured, for at least 85% of archetypes?
2. Do detections cover at least 70% of the top-rated threats from the library with uncovered threats published as gaps, and does every detection route to the shared issue backlog with an IM severity rather than to a dashboard? For Infrastructure, the assessor should be shown the inference and egress logs and the exposure-change detection.
3. Is retention per log class traced to a named obligation rather than to storage cost, with redaction applied at write, and are log streams verified as queryable within a working day at investigation volumes? In Infrastructure, ask to see the inference and egress logs and the exposure-change detection.

Common pitfalls

• ❌ Verifying that logging is configured rather than that it is emitting, which passes a stream that has been silent for months.
• ❌ Omitting model and version from the log context, which makes every output-integrity investigation start from scratch.
• ❌ Setting retention on storage cost, an error discovered during an investigation when it can no longer be corrected.
• ❌ Routing detections to a dashboard, where they have no owner and no clock.
• ❌ Building a detection set with no consumption anomaly, leaving the one attack class that produces no availability signal entirely uncovered.

Level 2, Comprehensive

Calibrate infrastructure detection and logging depth to asset tier, tune for alert quality, and establish behavioural baselines so departure is detectable.

Activities

A) Calibrate logging and detection depth to tier — Publish what each tier receives: Critical gets full logging, the complete detection set tuned for the asset, behavioural baselining and the shortest alerting path; Low gets baseline logging and the shadow-AI detections. Derive the applicable set from the inventory tier automatically. Publish the retention and sampling position per tier explicitly, because sampling is where cost pressure lands first and a Critical-tier asset whose telemetry is sampled has monitoring that is statistical at exactly the tier where specific answers are needed. Treat a sampling or retention reduction at Critical tier as a material change requiring approval rather than an operational tuning decision. Behavioural baselining in this domain covers request distribution per tenant, accelerator utilisation and cost per thousand requests, where the last is frequently the earliest indicator of everything else.

B) Establish behavioural baselines per Critical-tier asset — Build a behavioural baseline per Critical-tier asset covering its normal operating envelope: request and tool-call volumes, the distribution of actions taken, typical session shapes, identity and access patterns, and consumption per unit of work. Alert on departure rather than only on signature, because the AI failure modes that matter most produce traffic that is individually well-formed and collectively abnormal, and a signature-based set cannot see an agent competently doing something nobody asked for. Refresh baselines on a defined cadence and after any material change to the asset, and record baseline shifts explicitly, since an unexplained baseline shift is itself the signal and is lost if baselines are refreshed silently. Detection-as-code in this domain is the most stable in the model, because cloud audit and network telemetry schemas change slowly and predictably.

C) Tune for alert quality and report precision with recall — Measure detection quality using analyst disposition labels, and report precision and recall together. Precision alone rewards a detector that fires twice a year and misses everything, which is exactly the detector a tuning programme under alert-fatigue pressure will produce. Set a false-positive budget per detection and tune or retire detections that exceed it, because analysts who learn that a detection is usually wrong stop reading it and the coverage is then notional. Assert continuously that each detection is still evaluating and each stream still emitting, alerting within hours, since a stopped detection and a clean environment are indistinguishable in every dashboard. Infrastructure detection content contributes well since the underlying platforms are shared, so a rule transfers with minimal adaptation.

Outcome metrics

Oversight metrics — Alert disposition distribution per analyst, compared across analysts on comparable queues (detects: disposition drift, where the same alert would be escalated by one analyst and closed by another, making detection outcome depend on shift rather than on evidence) · Alerts per analyst per shift against the published ceiling (detects: analyst overload, where triage quality falls as volume rises while closure rates continue to look healthy) · Critical-tier detections closed as false positive and later implicated in an incident (detects: dismissal under load of exactly the alerts that mattered, visible only by tracing closures forward)

Consumption metrics — Consumption-anomaly detections operating against a per-asset behavioural baseline per per 1k inference requests · Time from a consumption anomaly beginning to its detection per per consumption anomaly · Telemetry cost per retained record at Critical tier, tracked against sampling decisions per per retained record

Success criteria

• Logging and detection depth are published per tier and derived automatically from the inventory.
• At least 90% of Critical-tier assets have a current behavioural baseline, refreshed on cadence and after material change, with shifts recorded.
• Every active detection reports precision and recall together, with a false-positive budget enforced by tuning or retirement.
• A silent log stream or stopped detection is detected within 4 hours.
• Sampling or retention reductions at Critical tier are approved as material changes rather than treated as operational tuning.

Assessment questions

1. Are logging and detection depth published per tier and derived automatically from the inventory, with sampling and retention positions stated per tier and reductions at Critical tier treated as material changes? Evidence this in the Infrastructure domain against the inference and egress logs and the exposure-change detection.
2. Do at least 90% of Critical-tier assets carry a current behavioural baseline alerting on departure rather than only on signature, with baseline shifts recorded explicitly rather than refreshed silently? For Infrastructure, the assessor should be shown the inference and egress logs and the exposure-change detection.
3. Is detection quality measured with precision and recall reported together using analyst disposition labels, with a false-positive budget enforced, and is a stopped detection or silent stream detected within 4 hours? In Infrastructure, ask to see the inference and egress logs and the exposure-change detection.

Common pitfalls

• ❌ Reporting precision without recall, which rewards a detector that fires twice a year and catches nothing.
• ❌ Relying on signature detections alone, which cannot see an agent competently doing something nobody asked for.
• ❌ Refreshing behavioural baselines silently, which discards the baseline shift that was itself the signal.
• ❌ Allowing sampling at Critical tier as an operational tuning decision, making monitoring statistical where specific answers are needed.
• ❌ Assuming a quiet detection is a clean environment, when a stopped detection looks identical.

Level 3, Industry-Leading

Engineer infrastructure detections as versioned, tested code, close the loop from incident to detection continuously, and contribute detection content back to the field.

Activities

A) Engineer detections as versioned, tested code — Move detection content into version control with the same discipline as application code: each detection carries an owner, a version, a documented intent, and test cases that assert it fires on known-positive events and does not fire on known-negative ones. Run the tests continuously against replayed or synthetic events, because the most common silent failure in monitoring is a detection that still exists, still evaluates and no longer matches anything due to a log-format change upstream, and nothing in a conventional monitoring stack reports that. Deploy through a pipeline with review, so a detection change is as traceable as a code change and can be reverted as easily. Behavioural baselining in this domain covers request distribution per tenant, accelerator utilisation and cost per thousand requests, where the last is frequently the earliest indicator of everything else. Detection tests in this domain replay cloud audit and network events, which are stable enough that a detection broken by schema change is genuinely rare here.

B) Close the incident-to-detection loop continuously — Require every incident and every adversarial finding to be assessed for detection coverage within days: did a detection fire, should one have, and if so why did it not. Where no detection existed, author one and deploy it within 10 business days, with its test cases. Where one existed and did not fire, treat that as a detection defect with an owner rather than as an incident detail. Track the share of incidents detected internally before external report as the practice's headline measure, since it is the one number that expresses whether monitoring is working, and report it alongside coverage rather than instead of it. Detection-as-code in this domain is the most stable in the model, because cloud audit and network telemetry schemas change slowly and predictably.

C) Contribute detection content back — Contribute at least four substantive artifacts per year in deployable form: detection rules, behavioural baselining approaches, log schemas for AI telemetry and the test cases that verify a detection works. Shared detection content is scarce in AI security while the failure modes are largely common across organisations, which makes this among the highest-leverage contributions available, since a peer can deploy a rule and cannot deploy a description of one. Work through OWASP projects, MITRE ATLAS, sector information-sharing bodies and detection-engineering communities. Anonymise, review legally, coordinate disclosure where content reveals a provider weakness, and prioritise contributing the detections that most often produce actioned findings internally. Infrastructure detection content contributes well since the underlying platforms are shared, so a rule transfers with minimal adaptation.

Outcome metrics

Oversight metrics — Detection changes deployed without review, by severity of the detection (detects: detection content changed directly in the console, which is untraceable, unrevertable and the most common way coverage is lost without a decision) · Detections whose tests have been failing for more than one week without action (detects: test failure normalised, which restores exactly the silent-failure mode the tests were introduced to remove) · Incidents detected externally rather than internally, with a coverage assessment completed (detects: external detection accepted without asking why internal detection missed it, which forfeits the single most informative signal monitoring produces)

Consumption metrics — Consumption-anomaly detections held as tested code with known-positive cases per per consumption detection · Time from consumption anomaly onset to internal detection per per 1k inference requests · Telemetry and detection platform cost per monitored asset per per monitored asset

Success criteria

• At least 85% of active detections are held in version control with an owner, documented intent and test cases.
• Detection tests run continuously against replayed or synthetic events, with failures alerting within the day.
• Every incident and adversarial finding is assessed for detection coverage within 5 business days, and uncovered cases produce a deployed detection within 10.
• Internal detection share is reported as the practice's headline measure alongside coverage.
• At least four accepted external contributions per year in deployable form.

Assessment questions

1. Are at least 85% of active detections held in version control with an owner, documented intent and test cases running continuously against replayed or synthetic events, so a detection broken by an upstream log change is caught by its own test? Evidence this in the Infrastructure domain against the inference and egress logs and the exposure-change detection.
2. Is every incident and adversarial finding assessed for detection coverage within 5 business days, with uncovered cases producing a deployed detection and its tests within 10, and detections that existed but did not fire treated as defects with owners? For Infrastructure, the assessor should be shown the inference and egress logs and the exposure-change detection.
3. Is internal detection share reported as the headline outcome measure alongside coverage, and does the organisation contribute at least four accepted detection artifacts per year in deployable rather than descriptive form? In Infrastructure, ask to see the inference and egress logs and the exposure-change detection.

Common pitfalls

• ❌ Leaving detections in a console rather than version control, which makes changes untraceable and coverage loss undecided.
• ❌ Deploying detections without test cases, which leaves the silent-failure mode after an upstream log change entirely uncovered.
• ❌ Normalising failing detection tests, which restores exactly the problem the tests were introduced to remove.
• ❌ Accepting external detection of an incident without asking why internal detection missed it, forfeiting the most informative signal available.
• ❌ Contributing descriptions of detections rather than deployable rules, which peers read and cannot run.

Part V, Assessment workbook

The full questionnaires for this domain are in ../questionnaires/, one per practice, nine questions each. Work them in dependency order from Part III.

Scoring

Fully Mature 1.0 · Implemented 0.67 · Partial 0.33 · Not Implemented 0.0. A level is achieved when all three of its questions score 0.67 or better. Levels are derived, never self-assigned, and every claim is graded at its weakest evidence grade (E0–E3).

Generated by tools/render.py from the Infrastructure domain cell records.