Security Testing (ST)

Endpoints Domain - HAIAMM v3.5

Generated from the canonical cell records in cells/. Do not edit by hand; edit cells/ST-Endpoints-L*.cell.json and re-run tools/render.py.

---

Practice Overview

Business Function: Verification

Objective: Baseline a foundational security test battery per endpoints archetype that asserts the controls actually hold, and run it before every release.

Description: A program at this level tests the AI-specific controls rather than assuming them. Each archetype has a foundational battery, and every test asserts a control holds rather than that a feature works. The battery covers an injection test asserting that content the user did not author cannot drive the assistant to act, an allowlist test asserting an unsanctioned capability cannot be installed or enabled, a device-posture test asserting access is refused from a non-compliant device, an input data-loss-prevention test asserting regulated data is blocked at paste, a per-user and per-session limit test asserting the declared at-limit behaviour occurs, and a logging test asserting sessions are recorded at the declared level. The discipline that distinguishes this practice from general quality assurance is that each test must be able to fail for the right reason: a test that passes because the condition it checks never arises provides assurance that is indistinguishable from real assurance on every dashboard.

---

Maturity Level 1, Foundational

Objective: Baseline a foundational security test battery per endpoints archetype that asserts the controls actually hold, and run it before every release.

A program at this level tests the AI-specific controls rather than assuming them. Each archetype has a foundational battery, and every test asserts a control holds rather than that a feature works. The battery covers an injection test asserting that content the user did not author cannot drive the assistant to act, an allowlist test asserting an unsanctioned capability cannot be installed or enabled, a device-posture test asserting access is refused from a non-compliant device, an input data-loss-prevention test asserting regulated data is blocked at paste, a per-user and per-session limit test asserting the declared at-limit behaviour occurs, and a logging test asserting sessions are recorded at the declared level. The discipline that distinguishes this practice from general quality assurance is that each test must be able to fail for the right reason: a test that passes because the condition it checks never arises provides assurance that is indistinguishable from real assurance on every dashboard.

Dependencies

• Required: SM-Endpoints-L1, TA-Endpoints-L1, SR-Endpoints-L1, SA-Endpoints-L1
• Alignment (not blocking): Existing quality assurance and CI practice, so security tests run on the same rails as everything else
• Supports / unblocks: IR-Endpoints-L1; EH-Endpoints-L1; IM-Endpoints-L1; ML-Endpoints-L1; Required by: ST-Endpoints-L2

Desired Outcomes

• Every endpoints archetype has a foundational security test battery that runs before release.
• Each test asserts a control holds rather than that a feature works, and can fail for the right reason.
• Test failures block release at the tiers where they matter, rather than being recorded and passed over.
• Test coverage maps to the threats from the threat library, so gaps are visible rather than implicit.
• A waived test failure has a named authority and an expiry rather than being absorbed into the release.

Activities

A) Build the foundational battery per archetype

Author a foundational test battery per archetype covering an injection test asserting that content the user did not author cannot drive the assistant to act, an allowlist test asserting an unsanctioned capability cannot be installed or enabled, a device-posture test asserting access is refused from a non-compliant device, an input data-loss-prevention test asserting regulated data is blocked at paste, a per-user and per-session limit test asserting the declared at-limit behaviour occurs, and a logging test asserting sessions are recorded at the declared level. Write each test as an assertion about a control rather than as a check on behaviour, and verify that each test can fail: introduce the condition deliberately once and confirm the test catches it, which is the only way to distinguish a test that asserts something from one that passes because its condition never arises. Keep the battery small enough to run on every policy or client release ring at the tiers that need it, because a battery too slow to run continuously will be moved to a nightly job and then to a weekly one and then to nowhere. Version the battery alongside the reference patterns so that a pattern change and its test change travel together.

B) Map tests to threats and publish the coverage gaps

Map each test to the threat identifiers from the TA library that it exercises, and publish the threats that have no test. The coverage map is what converts a test suite into an assurance argument: without it the battery asserts that the tests pass, which is not the same claim as that the threats are addressed. Report threat coverage rather than test count, since test count rises with effort while coverage rises only with effect. Treat an untested threat at a high-risk archetype as an open finding with an owner rather than as a backlog item, and feed coverage gaps back to the threat assessment practice so that the library and the battery evolve together.

C) Run before release, with a governed waiver path

Run the battery before every release, and trigger it on the changes that actually invalidate prior results, being an extension update or permission change, a client or agent upgrade, a policy mode change from enforcing to audit, and a new capability enabled inside an approved product. Failures block release for Critical- and High-tier assets and warn for the rest. Provide a governed waiver path, because an unwaivable gate is bypassed in ways nobody sees: a waiver needs a named authority, a stated reason, an expiry and automatic notification. Review waivers weekly and treat a test waived across more than one release cycle as a finding, since the common failure here is not a single waiver but a test quietly disabled to unblock a release and never re-enabled, which reduces coverage without anyone deciding to reduce it.

Outcome Metrics (L1)

Metric	Baseline	L1 Target	Source
Endpoints archetypes with a baselined foundational security test battery	first battery build	≥85% of inventoried archetypes within 9 months	Test registry joined to the asset inventory
Tests verified as able to fail, by deliberately introducing the condition once	first verification pass	100% of tests in the battery	Test verification records
Threats from the TA library exercised by at least one test	first coverage mapping	≥75%, with untested threats published as gaps	Threat-to-test coverage map
Releases of Critical- and High-tier assets proceeding with a failing battery test and no waiver	first quarter of release telemetry	0	CI results joined to release records
Test waivers carrying a named authority, reason and expiry	first waiver review	100%, with none past expiry	Waiver log

Process Metrics (leading)

• Battery triggered by the change types that invalidate prior results rather than only on a schedule.
• Waivers reviewed weekly, with any test waived across more than one release cycle raised as a finding.
• Threat-to-test coverage map refreshed whenever the threat library changes.
• Battery runtime monitored so it stays fast enough to run on every policy or client release ring at high tiers.

Effectiveness Metrics (business value)

• Findings caught by the battery rather than in production rise as a share of total findings.
• Time from a threat entering the library to a test exercising it falls.
• Release delays attributable to late-discovered security defects fall as testing moves earlier.

Oversight Metrics (the HAI control, measured)

Metric	Oversight failure it detects	Source
Failing tests waived to permit a release, with a named waiver authority	waiver becoming the routine release path, where a failing security test is treated as a scheduling obstacle rather than a result	Test waiver log joined to release records
Tests marked as skipped or quarantined for more than one release cycle	silent test decay, where a test is disabled to unblock a release and never re-enabled, so coverage falls without any decision being taken	CI test status history

Level of Effort (incremental over the level below)

• Stand-up: 180 hours · Ongoing: 120 hours/quarter
• AppSec / SecEng — primary, 34 days/year
• AI/ML Engineer — regular, 14 days/year
• DevOps / Platform Engineer — regular, 10 days/year
• Security Architect — low, 6 days/year
• Material new tooling: Test harness capable of exercising AI-specific controls in CI

Success Criteria

• A foundational battery exists for at least 85% of inventoried endpoints archetypes and runs before every release.
• Every test has been verified as able to fail by deliberately introducing its condition once.
• At least 75% of threats from the library are exercised by a test, with untested threats published as gaps carrying owners.
• Failures block release for Critical- and High-tier assets, with a governed waiver path carrying authority, reason and expiry.
• A test waived across more than one release cycle is raised as a finding rather than tolerated.

---

Maturity Level 2, Comprehensive

Objective: Calibrate the endpoints test battery to asset tier, run it continuously in CI, and add adversarial testing where the automated battery cannot reach.

At this level the battery runs continuously rather than at release, and its depth follows tier. Critical-tier assets carry the full battery plus scheduled adversarial testing by people, because an automated suite converges on what it already tests while the attack surface moves. Regression corpora are kept fresh, since a prompt-injection corpus that has not changed in a year tests last year's techniques. Waivers are tier-governed, and waived tests later implicated in incidents are tracked, which is the only feedback on whether waiver judgment is sound.

Dependencies

• Required: ST-Endpoints-L1, SM-Endpoints-L2, TA-Endpoints-L2, IR-Endpoints-L2
• Alignment (not blocking): Release engineering, since continuous testing runs on their pipeline and gates their releases
• Supports / unblocks: EH-Endpoints-L2; IM-Endpoints-L2; ML-Endpoints-L2; Required by: SR-Endpoints-L3, ST-Endpoints-L3, TA-Endpoints-L3

Desired Outcomes

• Battery depth follows asset tier, and the difference is published rather than negotiated per release.
• Critical-tier assets are exercised by people as well as by the automated battery.
• Regression corpora stay fresh, so the battery tests current techniques rather than historical ones.
• Waiver authority rises with tier, and waived tests later implicated in incidents are tracked.
• Test results are trusted, because tests that assert nothing are identified and removed.

Activities

A) Calibrate battery depth and gating to tier

Publish what each tier receives: Critical takes the full battery before each policy release ring and on every extension or client update, with failures blocking the rollout, plus quarterly adversarial testing; High takes the full battery in the endpoint policy release process with failures blocking; Medium takes a defined subset; Low takes spot checks. Derive the applicable battery from the inventory tier automatically rather than letting teams select, because selection under release pressure converges on the fastest subset. Publish gating behaviour per tier so that nobody negotiates it at release time. Where battery runtime makes per-build execution impractical at Critical tier, treat that as a test-performance problem to be solved rather than a reason to reduce frequency, since reduced frequency is how continuous testing becomes periodic testing. Adversarial work in this domain concentrates on content the user did not author reaching an assistant that can act, and on extensions acquiring permissions after approval, both of which need a real browser and a real device to exercise.

B) Add adversarial testing where automation cannot reach

Schedule adversarial testing by people against Critical-tier assets at least quarterly, scoped from the threat model and executed by testers who did not build the asset. The purpose is specifically to find what the automated battery cannot: an automated suite tests the techniques it encodes and therefore converges, passing continuously while the attack surface moves underneath it. Measure the adversarial exercise by what it found that the battery missed, and feed every such finding back into the battery as a new test, so that the exercise permanently raises the automated floor rather than producing a report. Scope must be set by the testing party rather than the tested team, because a team that steers scope away from what it knows is weak produces a clean result that means nothing. The regression trigger that matters most here is an extension or client update, which changes the attack surface without any action by the organisation.

C) Keep corpora fresh and remove tests that assert nothing

Refresh regression corpora on a defined cadence, targeting 90 days, drawing new cases from published research, incidents, adversarial findings and threat-intelligence updates. A prompt-injection or adversarial-input corpus that has not changed in a year tests the techniques of a year ago and passes with increasing confidence as the real surface moves. Separately, audit the battery for tests that have never failed across a full quarter and examine each: some are genuinely stable controls and some assert nothing at all, and the two are indistinguishable from a dashboard. Remove or strengthen the latter. Trigger regression runs on an extension update or permission change, a client or agent upgrade, a policy mode change from enforcing to audit, and a new capability enabled inside an approved product in addition to the scheduled cadence. Endpoint test content contributes well as extension permission test cases and injection corpora for browser-delivered content.

Outcome Metrics (L2)

Metric	Baseline	L2 Target	Source
Assets running the battery depth derived from their tier	first tier-derivation pass	100% of Critical and High tier assets	Battery assignment joined to inventory tier
Critical-tier assets exercised by adversarial testing in the last 12 months	first adversarial cycle	100%, scoped by the testing party	Adversarial engagement register
Adversarial findings fed back into the automated battery as new tests	first adversarial cycle	100% within 30 days of the finding	Test registry change log joined to engagement findings
Regression corpus entries with a last-refresh date within 90 days	first freshness audit	100%	Corpus metadata
Tests with a 100% pass rate across a full quarter, audited for assertion strength	first audit round	100% audited, with non-asserting tests removed or strengthened	Test audit records
Waived Critical-tier tests carrying an authority proportionate to tier and an expiry	first tiered waiver review	100%, with none past expiry	Waiver log joined to asset tier

Process Metrics (leading)

• Battery assignment re-derived automatically when an asset's tier changes.
• Corpus refresh cadence honoured at 90 days, with the refresh date published.
• Adversarial scope set by the testing party and recorded, with the tested team's input logged separately.
• Never-failing tests audited quarterly for assertion strength.

Effectiveness Metrics (business value)

• Adversarial findings the battery missed fall over time, indicating the automated floor is rising.
• Time from a new technique appearing publicly to a corpus entry testing it falls.
• Waived tests later implicated in incidents fall, which evaluates waiver judgment rather than waiver process.
• Testing effort concentrates on Critical and High tiers relative to L1.

Oversight Metrics (the HAI control, measured)

Metric	Oversight failure it detects	Source
Critical-tier test waivers, with the waiver authority and expiry recorded	waiver authority drifting downward under release pressure, so the decision to ship with a failing security test is taken by whoever is available	Test waiver log joined to asset tier
Waived tests later implicated in a production incident	waiver judgment that is procedurally correct and substantively wrong, which only forward tracing can reveal	Incident root-cause analysis joined to the waiver log
Test suites whose pass rate is 100% across every run for a full quarter	tests that assert nothing, which are indistinguishable from passing tests on every dashboard and provide false assurance at exactly the tiers that need real assurance	CI test result history with assertion-strength review

Level of Effort (incremental over the level below)

• Stand-up: 200 hours · Ongoing: 200 hours/quarter
• AppSec / SecEng — primary, 40 days/year
• Security Analyst / SOC — regular, 16 days/year
• AI/ML Engineer — regular, 12 days/year
• DevOps / Platform Engineer — low, 8 days/year
• Material new tooling: Adversarial testing capability, internal or contracted, Regression corpus management with freshness tracking

Success Criteria

• Battery depth and gating behaviour are published per tier and derived automatically from the inventory.
• Every Critical-tier asset is exercised by adversarial testing at least annually, scoped by the testing party rather than the tested team.
• Every adversarial finding becomes a new automated test within 30 days, raising the automated floor permanently.
• Regression corpora are refreshed within 90 days, and triggered by the change types that invalidate prior results.
• Tests that have never failed across a quarter are audited for assertion strength and removed or strengthened.

---

Maturity Level 3, Industry-Leading

Objective: Run continuous adversarial testing against endpoints AI assets, measure the battery against what people can still find, and contribute test corpora back to the field.

At this level testing is continuous and its quality is measured against human adversaries rather than against itself. The gap between what automation catches and what people still find is the practice's primary quality signal, and closing it is the work. Test corpora and harnesses are contributed back, which matters disproportionately in AI security because shared adversarial corpora are the field's scarcest resource and the one where a single organisation's contribution measurably raises everyone's floor.

Dependencies

• Required: ST-Endpoints-L2, TA-Endpoints-L3, ML-Endpoints-L2
• Alignment (not blocking): Red team function, internal or contracted, operating continuously rather than per engagement
• Supports / unblocks: Every other Endpoints practice at L3, which inherits continuously tested assurance

Desired Outcomes

• Adversarial testing runs continuously against Critical-tier assets rather than in scheduled campaigns.
• The gap between automated catch and human find is measured and closing.
• New public techniques reach the corpus within days rather than at the next refresh.
• Test corpora and harnesses are contributed back in runnable form.
• Test quality is measured against adversaries rather than against pass rates.

Activities

A) Run adversarial testing continuously

Move from scheduled engagements to continuous adversarial operation against Critical-tier assets, whether by a standing internal team, a contracted continuous service or an automated adversarial harness that evolves its own cases. Measure the practice by time-to-find rather than by engagement count, and track the surprise rate, being the share of findings the automated battery should have caught but did not. A falling surprise rate indicates the battery is converging on reality; a flat one at high volume indicates the battery has converged on itself. Feed every finding into the battery within days rather than within the 30-day window that L2 allowed, since continuous operation makes the slower loop the binding constraint. Adversarial work in this domain concentrates on content the user did not author reaching an assistant that can act, and on extensions acquiring permissions after approval, both of which need a real browser and a real device to exercise.

B) Close the automation gap deliberately

Treat the gap between what automation catches and what humans still find as the practice's primary work item rather than as an inevitable residue. For each human finding, ask specifically why the battery missed it: a missing case, a missing technique class, an assertion too weak, or a surface the harness cannot reach. Record the category, because the distribution tells you whether to extend the corpus, strengthen assertions or rebuild the harness, and these are very different investments. Publish the gap measure alongside coverage so that a rising coverage number with a static gap is visible as the illusion it represents. The regression trigger that matters most here is an extension or client update, which changes the attack surface without any action by the organisation.

C) Contribute corpora and harnesses back

Contribute at least four substantive artifacts per year, weighted toward runnable form: adversarial corpora, test harnesses, assertion libraries and reproducible technique demonstrations. Shared adversarial corpora are the scarcest resource in AI security and the one where contribution most directly raises the whole field's floor, because a corpus a peer can run is worth far more than a paper describing what it contains. Work through OWASP projects, MITRE ATLAS, OpenSSF AI and sector bodies. Anonymise, legally review and coordinate disclosure with affected providers before release. Count only accepted artifacts, and prioritise contributing the cases that most often catch real defects internally, since those carry the most evidence. Endpoint test content contributes well as extension permission test cases and injection corpora for browser-delivered content.

Outcome Metrics (L3)

Metric	Baseline	L3 Target	Source
Critical-tier assets under continuous adversarial testing rather than scheduled campaigns	L2 quarterly cadence	100%	Adversarial operation register
Surprise rate: findings the automated battery should have caught but did not	first quarter of continuous operation	tracked and falling year over year	Adversarial findings classified against battery coverage
Human findings fed back into the automated battery	first quarter of continuous operation	100% within 5 business days	Test registry change log joined to findings
Time from a technique appearing publicly to a corpus entry exercising it	L2 90-day refresh cadence	≤14 days at the 90th percentile	Corpus change log joined to publication dates
Accepted external contributions per year in runnable form	0	≥4 accepted, majority runnable corpora or harnesses	Contribution log with acceptance references

Process Metrics (leading)

• Automation-gap categories recorded per human finding, and the distribution reviewed quarterly.
• Public technique monitoring continuous, with corpus entries raised within days.
• Disclosure coordinated with affected providers before any release touching their systems.
• Contribution pipeline carries at least two runnable artifacts in draft or review.

Effectiveness Metrics (business value)

• Surprise rate falls, which is the honest measure that automation is converging on reality rather than on itself.
• Time-to-find for a newly introduced defect falls toward the continuous testing cycle.
• External adoption of contributed corpora and harnesses by peer organisations.
• Production incidents attributable to a technique the corpus already contained approach zero.

Oversight Metrics (the HAI control, measured)

Metric	Oversight failure it detects	Source
Adversarial findings that the automated battery should have caught but did not	a battery that has converged on what it already tests, which passes continuously while the real attack surface moves	Red-team findings joined to the test registry
Automated test results overridden at merge, with the override sustained across builds	override becoming the normal path around an inconvenient check, which converts a gate into advice without any decision	Pipeline override log
Share of red-team scope determined by the team being tested	scope capture, where the tested team steers the exercise away from what it knows is weak, which makes a clean result meaningless	Red-team engagement scoping records

Level of Effort (incremental over the level below)

• Stand-up: 260 hours · Ongoing: 260 hours/quarter
• AppSec / SecEng — primary, 44 days/year
• Security Analyst / SOC — regular, 24 days/year
• AI/ML Engineer — regular, 12 days/year
• Security Architect — low, 6 days/year
• Material new tooling: Continuous adversarial testing capability, Evolving adversarial harness with case generation

Success Criteria

• Critical-tier assets are under continuous adversarial testing, measured by time-to-find rather than engagement count.
• The surprise rate is tracked and falling, and reported alongside coverage so that rising coverage with a static gap is visible.
• Human findings reach the automated battery within 5 business days.
• New public techniques reach the corpus within 14 days at the 90th percentile.
• At least four accepted external contributions per year, the majority runnable corpora or harnesses, disclosure-coordinated.

---

Key Success Indicators

Level 1:

• A foundational battery exists for at least 85% of inventoried endpoints archetypes and runs before every release.
• Every test has been verified as able to fail by deliberately introducing its condition once.
• At least 75% of threats from the library are exercised by a test, with untested threats published as gaps carrying owners.
• Failures block release for Critical- and High-tier assets, with a governed waiver path carrying authority, reason and expiry.
• A test waived across more than one release cycle is raised as a finding rather than tolerated.

Level 2:

• Battery depth and gating behaviour are published per tier and derived automatically from the inventory.
• Every Critical-tier asset is exercised by adversarial testing at least annually, scoped by the testing party rather than the tested team.
• Every adversarial finding becomes a new automated test within 30 days, raising the automated floor permanently.
• Regression corpora are refreshed within 90 days, and triggered by the change types that invalidate prior results.
• Tests that have never failed across a quarter are audited for assertion strength and removed or strengthened.

Level 3:

• Critical-tier assets are under continuous adversarial testing, measured by time-to-find rather than engagement count.
• The surprise rate is tracked and falling, and reported alongside coverage so that rising coverage with a static gap is visible.
• Human findings reach the automated battery within 5 business days.
• New public techniques reach the corpus within 14 days at the 90th percentile.
• At least four accepted external contributions per year, the majority runnable corpora or harnesses, disclosure-coordinated.

---

Common Pitfalls

Level 1:

• ❌ Writing tests that pass because their condition never arises, which are indistinguishable from real assurance on every dashboard.
• ❌ Reporting test count rather than threat coverage, since count rises with effort while coverage rises only with effect.
• ❌ Building a battery too slow to run on every policy or client release ring, which migrates to nightly, then weekly, then nowhere.
• ❌ Providing no waiver path, which produces bypasses nobody sees rather than compliance.
• ❌ Tolerating tests quietly disabled to unblock a release, which reduces coverage without any decision being taken.

Level 2:

• ❌ Letting the tested team scope the adversarial exercise, which produces a clean result that means nothing.
• ❌ Treating adversarial findings as a report rather than as new automated tests, so the same class of finding recurs next cycle.
• ❌ Leaving regression corpora static, which tests last year's techniques with rising confidence as the real surface moves.
• ❌ Reducing test frequency when the battery gets slow, rather than treating runtime as a performance problem to solve.
• ❌ Assuming a test that never fails is a stable control, when it is indistinguishable from one that asserts nothing.

Level 3:

• ❌ Measuring adversarial testing by engagement count, which rewards activity and says nothing about whether defects are found faster.
• ❌ Treating the automation gap as an inevitable residue rather than as the work, which leaves the battery converged on itself.
• ❌ Reporting coverage without the surprise rate, so a rising coverage number conceals a static real-world gap.
• ❌ Contributing papers rather than runnable corpora, which peers read and cannot execute.
• ❌ Releasing technique demonstrations touching a provider's systems without coordinating disclosure first.

---

Practice Maturity Questions

Level 1:

1. Does a foundational security test battery exist for at least 85% of inventoried endpoints archetypes, with each test written as an assertion about a control and verified as able to fail by deliberately introducing its condition once?
2. Is every test mapped to the TA threat identifiers it exercises, with threats having no test published as gaps and threat coverage reported rather than test count? For Endpoints, the assessor should be shown the injection test against unauthored content and the allowlist bypass test.
3. Does the battery run before every release and on the change types that invalidate prior results, with failures blocking Critical- and High-tier releases and a governed waiver path carrying named authority, reason and expiry? In Endpoints, ask to see the injection test against unauthored content and the allowlist bypass test.

Level 2:

1. Is battery depth and gating behaviour published per tier and derived automatically from the inventory rather than selected by teams under release pressure? Evidence this in the Endpoints domain against the injection test against unauthored content and the allowlist bypass test.
2. Is every Critical-tier asset exercised by adversarial testing at least annually with scope set by the testing party, and does every adversarial finding become a new automated test within 30 days so the automated floor rises permanently? For Endpoints, the assessor should be shown the injection test against unauthored content and the allowlist bypass test.
3. Are regression corpora refreshed within 90 days and triggered by invalidating change types, and are tests that have never failed across a quarter audited for assertion strength rather than assumed to be stable controls? In Endpoints, ask to see the injection test against unauthored content and the allowlist bypass test.

Level 3:

1. Is adversarial testing continuous against Critical-tier assets rather than scheduled, measured by time-to-find and surprise rate rather than by engagement count? Evidence this in the Endpoints domain against the injection test against unauthored content and the allowlist bypass test.
2. Is the automation gap treated as the primary work item, with each human finding categorised as a missing case, missing technique class, weak assertion or unreachable surface so that investment goes to the right remedy? For Endpoints, the assessor should be shown the injection test against unauthored content and the allowlist bypass test.
3. Do new public techniques reach the corpus within 14 days at the 90th percentile, and does the organisation contribute at least four accepted artifacts per year in majority-runnable form with disclosure coordinated? In Endpoints, ask to see the injection test against unauthored content and the allowlist bypass test.

---

Lightweight Questions (E0 rung)

Answerable yes/no by one person in under 30 seconds, with nothing open.

Level 1:

• Is there a security test that would fail if your AI controls stopped working?
• Has anyone checked that your security tests can actually fail?
• Would a failing security test stop a release?

Level 2:

• Has a person tried to break your highest-risk AI asset this year?
• When did your prompt-injection test cases last change?
• Do you know which of your security tests have never once failed?

Level 3:

• Is someone continuously trying to break your highest-risk AI?
• Do you know what your automated tests keep missing?
• Have you published an adversarial corpus others can run?

---

Tracking

Level 1 — Jira Epic, labels haiamm-st-endpoints-l1, haiamm-verification

• Done when: Foundational battery coverage at or above 85% of archetypes with threat coverage at or above 75% and zero unwaived failing releases at Critical or High tier, sustained two consecutive quarters.
• Git artifacts: security/tests/endpoints/, security/tests/registry.json, security/threat-test-coverage.json
• Git signal: threat-test-coverage.json is regenerated by joining the test registry to the threat library rather than maintained by hand, and the battery runs as a required pipeline stage.
• CI check: security-test-gate

Level 2 — Jira Epic, labels haiamm-st-endpoints-l2, haiamm-verification

• Done when: Every Critical-tier asset adversarially tested within the trailing year with all findings converted to automated tests within 30 days and corpus freshness at 100%, sustained two quarters.
• Git artifacts: security/adversarial-register.json, security/corpora/, security/tests/registry.json
• Git signal: Corpus files carry a last-refresh date checked by CI against the 90-day threshold, and adversarial-register.json records the scoping party separately from the tested team.
• CI check: corpus-freshness-check

Level 3 — Jira Epic, labels haiamm-st-endpoints-l3, haiamm-verification

• Done when: All Critical-tier assets under continuous adversarial testing with human findings reaching the battery within 5 business days and technique-to-corpus latency within 14 days, sustained two quarters.
• Git artifacts: security/adversarial-findings.json, security/corpora/, governance/contribution-log.json
• Git signal: adversarial-findings.json records an automation-gap category per finding, and corpus metadata records the publication date of each technique alongside its corpus entry date.
• CI check: adversarial-gap-tracking

---

Mappings

Level	HAI TTP	ATLAS	OWASP LLM 2025	Compliance
L1	AGH, TM	AML.T0051, AML.T0053	LLM01, LLM05, LLM10	EU AI Act Art. 15; EU AI Act Art. 17; NIST AI RMF MEASURE 2.5; ISO/IEC 42001 8.1; ISO/IEC 27001 A.8.29
L2	AGH, TM	AML.T0051, AML.T0053	LLM01, LLM05, LLM10	EU AI Act Art. 15; EU AI Act Art. 17; EU AI Act Art. 72; NIST AI RMF MEASURE 2.7; ISO/IEC 42001 9.1
L3	AGH, TM	AML.T0051, AML.T0053	LLM01, LLM05, LLM10	EU AI Act Art. 15; EU AI Act Art. 72; NIST AI RMF MEASURE 2.7; ISO/IEC 42001 9.1

---

Document Version: HAIAMM v3.5 (generated) / Practice: Security Testing (ST) / Domain: Endpoints