> **v3.0 framing:** The canonical source-of-truth for Security Testing (ST) in the Software domain is [`../practices/ST-Software-OnePager.md`](../practices/ST-Software-OnePager.md). Outcome metrics, activities, and success criteria below are verbatim from that one-pager. Canonical subject and through-lines: [`../HAIAMM-v3.0-Framing.md`](../HAIAMM-v3.0-Framing.md).

---

# Security Testing (ST) - Software Domain
## HAIAMM Assessment Questionnaire v3.0

**Practice:** Security Testing (ST)
**Domain:** Software
**Purpose:** Assess organizational maturity in running adversarial test batteries and regression corpora against AI/HAI software artifacts, from foundational per-archetype batteries in CI through continuous automated adversarial testing
**Scoring Model:** Evidence + Outcome Metrics (see Scoring Methodology below)

---

## Instructions

- Answer each question honestly based on **current, implemented practices** (not plans or aspirations)
- Each question has two components: **Evidence** (what you did) and **Outcome Metrics** (how well it worked)
- **Scoring uses 4 tiers:** Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
- **Answer progressively** - Complete all Level 1 questions before Level 2
- **Level progression** - Achieve ALL questions at lower level before advancing
- **Baseline first** - Record current metric values before setting targets

---

## Scoring Methodology

| Score | Label | Criteria |
|-------|-------|----------|
| 1.0 | Fully Mature | Evidence complete + ≥3 outcome metrics meet targets |
| 0.67 | Implemented | Evidence complete + 2 outcome metrics meet targets |
| 0.33 | Partial | Evidence partially complete + <2 outcome metrics meet targets |
| 0.0 | Not Implemented | No evidence of the activity |

**Level Score** = average of the 3 question scores for that level
**Overall ST-Software Score** = weighted average: L1 × 0.5 + L2 × 0.3 + L3 × 0.2

---

## Maturity Level 1
**Objective:** Establish a foundational per-archetype test battery and regression corpora that run in CI on every PR, and verify that every AI/HAI software artifact reaches production with a passed go-live battery on record

---

### Question 1: Per-Archetype Test Battery

**Q1.1:** Is a per-archetype foundational test battery published for all seven AI/HAI software archetypes (LLM-integrated app, autonomous agent, RAG pipeline, fine-tuning/training workload, eval/red-team harness, model-serving service, classical ML model), with each test class tied to a TA-Software archetype threat (HAI TTP + ATLAS tactic ID) and an SR-Software requirement, defined inputs/outputs/pass-fail criteria, and an evidence artifact?

**Evidence Required:**
- [ ] Test battery document published per archetype; covers per-archetype probes including prompt-injection corpus (ATLAS TA0001/TA0003), training-data leakage canary (ATLAS TA0013), tool-scope boundary test (ATLAS TA0004), agent goal-hijack probe (ATLAS TA0001/TA0003), kill-switch test, HITL gate test, RAG injection test, eval isolation test, model-serving rollback test, and classical ML drift test as applicable per archetype
- [ ] Each test class records: inputs, expected output, pass/fail criteria, evidence artifact (log snippet, CI run link, or trace ID)
- [ ] Each test class is mapped to a TA-Software threat (HAI TTP + ATLAS tactic ID) and an SR-Software requirement
- [ ] Battery linked from SM inventory record and DR/IR artifacts for each AI/HAI software artifact
- [ ] Named battery owner per archetype documented; quarterly re-run scheduled in advance
- [ ] 100% of AI/HAI software artifacts reaching production in the last 90 days have a passed go-live battery on record

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % of AI/HAI software artifacts reaching production with a passed go-live battery on record | ___% | ___% | ≥90% within 12 months; 100% for Critical/High-tier | ☐ | |
| % archetype threat library entries covered by at least one test or corpus entry | ___% | ___% | ≥80% by end of year 1 | ☐ | |
| % test failures routed to IM within 1 business day | ___% | ___% | 100% | ☐ | |
| CI automation coverage of battery items (% running without human intervention) | ___% | ___% | ≥60% | ☐ | |

**Metric Collection Guidance:**
- **Go-live battery coverage**: Query SM inventory registry for artifacts promoted to production in the last 90 days; cross-reference with the test-run registry. Formula: `artifacts_with_passed_battery / artifacts_reaching_production × 100`
- **Threat coverage**: For each archetype's TA-Software library entry, check whether at least one test class or corpus entry maps to it. Formula: `covered_threats / total_archetype_threats × 100`. Source: TA library × test metadata mapping
- **IM routing rate**: For test failures generated in the last quarter, check IM ticket creation timestamps. Formula: `failures_with_IM_ticket_within_1BD / total_failures × 100`. Source: CI telemetry × IM system
- **CI automation**: Count battery items with a CI trigger vs. total battery items. Source: battery registry + CI configuration

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No evidence of per-archetype test battery)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 2: Regression Corpora in CI

**Q2.1:** Are four regression corpora (jailbreak, prompt-injection, agent goal-hijack, tool-misuse) versioned in source control, running in CI on every PR for Critical/High-tier artifacts, with a named corpus owner, a monthly refresh cadence from internal and external sources, and a CI token-spend budget cap?

**Evidence Required:**
- [ ] Four regression corpora published in source control: jailbreak corpus (30–100 entries covering role-override, persona-switch, authority-claim, encoding-bypass), prompt-injection corpus (30–100 direct and indirect entries), agent goal-hijack corpus (20–60 multi-turn sequences), and tool-misuse corpus (20–60 argument-smuggling payloads)
- [ ] Each corpus entry includes: input, expected safe output pattern, threat tag (HAI TTP + ATLAS tactic ID), OWASP reference, source, date added
- [ ] CI wiring confirmed: corpus runs on every PR for Critical/High-tier artifacts; failure is a blocking check for Critical/High-tier
- [ ] Named corpus owner per corpus; corpus changes go through PR review
- [ ] Monthly refresh cadence evidenced: change-log showing updates from internal observations (IR findings, IM incidents), external sources (OWASP LLM Top 10, HackAPrompt, ATLAS examples), and jailbreak research repositories
- [ ] CI token-spend budget cap configured and enforced; evidence of cap limit in CI configuration
- [ ] New PR for AI/HAI software that changes archetype triggers a corpus completeness check

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| Regression corpora published (jailbreak, prompt-injection, AGH, tool-misuse) | ___/4 | ___/4 | 4/4 | ☐ | |
| % PR merges for Critical/High-tier artifacts that ran the regression corpus and passed | ___% | ___% | ≥95% | ☐ | |
| Corpus refresh cadence, months since last update per corpus | ___ | ___ | ≤1 month | ☐ | |
| % archetype threat library entries covered by at least one corpus entry | ___% | ___% | ≥80% by end of year 1 | ☐ | |

**Metric Collection Guidance:**
- **Corpus publication**: Count corpora present in source control with versioned entries and a named owner. Source: corpus registry in VCS
- **PR corpus run rate**: Pull CI run history for Critical/High-tier artifact PRs; cross-reference with corpus run results. Formula: `PRs_with_corpus_run_and_pass / total_Critical_High_PRs × 100`. Source: CI telemetry
- **Refresh cadence**: Inspect git log for each corpus; compute days since last commit adding new entries. Source: VCS change-log
- **Threat coverage**: TA library entries mapped to corpus entries vs. total archetype library entries. Source: TA library × corpus metadata

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No regression corpora wired into CI)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 3: Go-Live Battery and IM Wire

**Q3.1:** Is the go-live battery operated with defined re-run triggers (pre-production, post-model-update, post-incident, quarterly), and are all test failures routed to IM-Software within 1 business day with a severity tag and named owner?

**Evidence Required:**
- [ ] Go-live battery process document defines re-run triggers: pre-production (blocks Sanctioned status), post-model-update within 14 days (Critical-tier: 7 days), post-incident before incident closure, and quarterly for all active artifacts
- [ ] Go-live test records linked from SM inventory and PC intake artifacts for artifacts promoted in the last 90 days
- [ ] Test failure to IM routing confirmed: sample of recent failures shows IM tickets created within 1 business day with severity tag (Blocker/Critical/High/Medium/Low), named owner, and test battery reference
- [ ] Severity rubric published and applied consistently to test failure triage
- [ ] Battery re-run cadence evidenced: quarterly re-run records on file for active artifacts with named battery owner
- [ ] Post-model-update re-run triggered and completed within SLA for any model-version changes in the last 6 months

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % test failures routed to IM within 1 business day | ___% | ___% | 100% | ☐ | |
| % post-model-update re-runs completed within declared SLA (7/14 days by tier) | ___% | ___% | ≥90% | ☐ | |
| % active artifacts with a quarterly battery re-run completed in last 90 days | ___% | ___% | ≥90% | ☐ | |
| % of AI/HAI software artifacts reaching production with a passed go-live battery on record | ___% | ___% | 100% for Critical/High-tier | ☐ | |

**Metric Collection Guidance:**
- **IM routing rate**: Compare test failure log timestamps to IM ticket creation timestamps. Source: CI failure log × IM system timestamp query
- **Post-model-update SLA**: For model-version changes in last 6 months, check re-run date vs. change date by tier. Source: model registry changelog × test-run registry
- **Quarterly re-run rate**: Count active artifacts with a battery run record dated within the last 90 days. Source: test-run registry × SM inventory active artifact list
- **Go-live coverage**: SM inventory × test-run registry cross-reference for artifacts reaching production in last 90 days

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No go-live battery or IM wiring in place)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

## Maturity Level 2
**Objective:** Calibrate test depth per risk tier using the SM L2 tier-treatment matrix, run scheduled red-team exercises per tier using TA L2 deep threat models, and test cross-archetype compositions for Critical-tier artifacts

---

### Question 4: Tier-Calibrated Battery and Corpus Depth

**Q4.1:** Is per-tier corpus calibration enforced in CI (Critical-tier: all 4 corpora on every PR + daily output-integrity regression; High-tier: all 4 corpora on merge; Low-tier: jailbreak corpus on merge), and does each Critical-tier artifact have a separately tuned corpus derived from its TA L2 per-artifact deep threat model?

**Evidence Required:**
- [ ] Per-tier test treatment published and aligned to SM L2 tier-treatment matrix: Critical (full battery + all 4 corpora every PR + daily output-integrity + logging-completeness quarterly), High (full battery + all 4 corpora on merge + weekly output-integrity), Medium (subset battery + jailbreak + prompt-injection on merge), Low (spot-check + jailbreak on merge)
- [ ] CI pipeline configuration confirms tier-differentiated corpus assignments, not the same corpus applied to all tiers
- [ ] Critical-tier corpus entries are tuned to the artifact's specific tool set, retrieval sources, and data classes from the TA L2 per-artifact threat model
- [ ] Daily output-integrity regression run logs on file for Critical-tier artifacts; drift alert wired to IM
- [ ] Logging-completeness verified quarterly for Critical-tier; semi-annually for High-tier; evidence on file
- [ ] Per-tier SLA adherence tracked and reported to program sponsor

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier artifacts with all 4 corpora running on every PR | ___% | ___% | 100% | ☐ | |
| Per-tier SLA adherence for testing activities (go-live battery, model-update re-run, red-team cadence) | ___% | ___% | ≥90% per tier | ☐ | |
| % Critical-tier artifacts with a separately tuned corpus from TA L2 threat model | ___% | ___% | 100% | ☐ | |
| % Critical-tier artifacts with daily output-integrity regression active | ___% | ___% | 100% | ☐ | |

**Metric Collection Guidance:**
- **Corpus per-PR rate**: CI telemetry for Critical-tier PRs; verify all 4 corpus runs appear in each PR's check suite. Source: CI telemetry
- **SLA adherence**: Program telemetry across go-live battery turnaround, model-update re-run dates, and red-team calendar adherence. Formula: `activities_completed_within_SLA / total_activities × 100`
- **Tuned corpus coverage**: Count Critical-tier artifacts with a corpus file containing artifact-specific entries vs. total Critical-tier artifacts. Source: corpus registry
- **Daily regression**: Check CI schedule configuration for output-integrity job; verify last-run timestamp per Critical-tier artifact is within 24 hours. Source: CI job run history

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No tier-calibrated corpus differentiation in place)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 5: Scheduled Red-Team Exercises

**Q5.1:** Are 100% of Critical-tier AI/HAI software artifacts red-teamed at least quarterly, and 100% of High-tier semi-annually, with scope derived from TA L2 per-artifact deep threat models covering prompt-injection chains, indirect-prompt-injection via RAG, agent tool abuse, multi-turn agent goal-hijack probes, data-egress canaries, and cross-tenant isolation, with findings routed to IM and remediation tracked?

**Evidence Required:**
- [ ] Red-team schedule on calendar covering all Critical-tier (quarterly) and High-tier (semi-annual) artifacts; no Critical-tier artifact skipped in the last 12 months
- [ ] Red-team scope documented per exercise: written rules of engagement, test plan reviewed with artifact owner, scope derived from TA L2 per-artifact deep threat model (not archetype snapshot)
- [ ] ATLAS tactic IDs referenced in scope documentation: TA0001 Reconnaissance, TA0003 Initial Access, TA0004 ML Model Access, TA0012 ML Attack Staging, TA0013 Exfiltration
- [ ] Red-team execution log and structured findings report on file: severity, root cause, ATLAS tactic ID, SR requirement traced, remediation pairing
- [ ] Findings routed to IM-Software with severity tag and named artifact owner as assignee; remediation tracked
- [ ] Scheduled red-team exercises confirmed for L2 cadence: quarterly for Critical, semi-annual for High

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier artifacts red-teamed in last 90 days | ___% | ___% | 100% | ☐ | |
| % High-tier artifacts red-teamed in last 180 days | ___% | ___% | 100% | ☐ | |
| % red-team findings (Critical/High severity) converted to corpus entries within 30 days | ___% | ___% | ≥90% | ☐ | |
| Per-tier SLA adherence for red-team cadence | ___% | ___% | ≥90% | ☐ | |

**Metric Collection Guidance:**
- **Critical red-team rate**: Count Critical-tier artifacts with a red-team report dated within the last 90 days vs. total Critical-tier artifacts. Source: ST records
- **High red-team rate**: Same logic for 180 days. Source: ST records
- **Finding to corpus conversion**: For Critical/High severity findings, count those with a corresponding corpus entry committed within 30 days. Formula: `findings_with_corpus_entry / total_Critical_High_findings × 100`. Source: finding → corpus pipeline telemetry
- **SLA adherence**: Compare scheduled red-team dates to actual execution dates across all tiers. Source: program calendar × ST records

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No scheduled red-team function in place)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 6: Cross-Archetype Composition Tests

**Q6.1:** Are cross-archetype composition tests (agent + RAG, fine-tune + model-serving, multi-agent orchestration) documented and executed for all Critical-tier composite artifacts, with each Critical/High-severity red-team finding producing a corpus entry within 30 days?

**Evidence Required:**
- [ ] Cross-archetype composition test plan published for each Critical-tier artifact with composite archetypes; reviewed by named architect
- [ ] Agent + RAG composition test: indirect-injection via RAG path into agent goal-hijack chain exercised and documented
- [ ] Fine-tune + model-serving composition test: membership-inference probe and canary-extraction probe against the served fine-tuned model documented
- [ ] Multi-agent orchestration test: sub-agent goal-hijack via orchestrator response; scope-inheritance test (sub-agent cannot exceed parent scope) documented
- [ ] Finding to TA library-gap pipeline: Critical gaps closed within 30 days; High within 60 days; tracked tickets on file
- [ ] Regression corpus growth rate evidenced: Critical-tier corpora show ≥1 new entry per month from red-team or incident findings

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier composite artifacts with documented cross-archetype composition tests | ___% | ___% | 100% | ☐ | |
| Regression corpus growth rate, Critical-tier corpora (new entries per month) | ___ | ___ | ≥1 per month | ☐ | |
| % red-team findings (Critical/High severity) converted to corpus entries within 30 days | ___% | ___% | ≥90% | ☐ | |
| % TA library gaps (Critical tier) closed within 30 days | ___% | ___% | 100% | ☐ | |

**Metric Collection Guidance:**
- **Composition test coverage**: Count Critical-tier artifacts with a composite archetype that have a documented composition test plan + execution record vs. total composite Critical-tier artifacts. Source: ST records
- **Corpus growth**: Git log of Critical-tier corpus files; count commits adding new entries per calendar month. Source: VCS change-log per corpus file
- **Finding to corpus conversion**: Finding log × corpus commit-log cross-reference. Source: IM findings × corpus change-log
- **TA gap closure**: TA library-gap ticket backlog filtered to Critical tier; measure open-to-close duration. Source: issue tracker

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No cross-archetype composition tests in place)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

## Maturity Level 3
**Objective:** Operate continuous automated adversarial testing for Critical-tier artifacts, publish regression corpora and findings as open artifacts, and contribute discovered TTPs to MITRE ATLAS, AVID, and OWASP LLM/Agentic Top 10

---

### Question 7: Continuous Automated Adversarial Testing Harness

**Q7.1:** Are ≥80% of Critical-tier AI/HAI software artifacts under continuous automated adversarial testing with daily probe execution, using prompt-injection generators, indirect-injection seeders, tool-misuse generators, and output-integrity monitors, with novel TTPs triaged into the TA-Software library within 14 days and high-severity automated findings routed to IM within 24 hours?

**Evidence Required:**
- [ ] Automated adversarial testing harness deployed and producing daily probe results against Critical-tier artifacts: prompt-injection generator (mutation + template + jailbreak-ladder generation), indirect-injection seeder (poisoned-document payloads seeded into retrieval path), tool-misuse generator (argument-smuggling variants per declared tool type), output-integrity monitor (daily golden test set run against production endpoints)
- [ ] Continuous adversarial testing telemetry: harness health dashboard showing % Critical-tier artifacts with a fresh probe result within the last 24 hours; on-call paged when a feed goes stale >24 hours
- [ ] Finding triage process: named ST owner reviewing automated findings at least weekly; novel TTP patterns forwarded to TA L3 auto-proposal pipeline within 14 days
- [ ] High-severity automated findings route to IM within 24 hours; evidence: IM ticket timestamps vs. harness alert timestamps
- [ ] ATLAS tactic walk records: harness exercises TA0001, TA0003, TA0004, TA0012, TA0013 technique coverage daily
- [ ] Output-integrity drift alert wired to IM; P1 alert triggers full regression corpus re-run

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier artifacts under continuous automated adversarial testing (daily probe execution) | ___% | ___% | ≥80% | ☐ | |
| New-TTP ingestion lead time (automated finding to TA library entry) | ___ days | ___ days | ≤14 days | ☐ | |
| % high-severity automated findings routed to IM within 24 hours | ___% | ___% | 100% | ☐ | |
| Continuous harness health (% Critical-tier artifacts with fresh probe result in last 24 hours) | ___% | ___% | ≥95% | ☐ | |

**Metric Collection Guidance:**
- **Continuous coverage**: Harness telemetry, count Critical-tier artifacts with a probe result in the last 24 hours vs. total Critical-tier. Source: ST harness telemetry
- **TTP ingestion lead time**: For novel patterns identified by the harness, measure time from harness alert to TA library entry. Source: harness to TA pipeline telemetry
- **IM routing within 24h**: Compare harness alert timestamp to IM ticket creation timestamp for high-severity findings. Source: harness alert log × IM system
- **Harness health**: Monitoring dashboard showing last-probe-time per Critical-tier artifact; stale feeds trigger on-call page

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No continuous automated adversarial testing harness in place)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 8: Industry Contributions

**Q8.1:** Has the program contributed ≥4 anonymized, legally-vetted findings per year to MITRE ATLAS, AVID, or OWASP LLM/Agentic Top 10, with at least one accepted as a new or refined technique, and are ≥4 open regression corpora published under a permissive license and maintained upstream?

**Evidence Required:**
- [ ] Contribution log on file: ≥4 submissions per year to MITRE ATLAS (novel technique observations with ATLAS tactic IDs), AVID (structured vulnerability disclosures), or OWASP LLM/Agentic Top 10 (real-world telemetry evidence during revision cycles)
- [ ] At least one submission accepted by ATLAS, AVID, or OWASP as a new or refined technique; evidence: acknowledgment or technique ID assignment
- [ ] Legal-vetting record for each contribution: org identity scrubbed; coordinated disclosure completed for any third-party component involvement
- [ ] ≥4 open regression corpora published under a permissive license (jailbreak, prompt-injection, AGH, tool-misuse); published versions scrubbed of org-specific tool names, data classes, and artifact identifiers
- [ ] Published corpora maintained upstream with documented update cadence; internal corpora are a superset of published versions
- [ ] Industry-contribution pipeline: at least one anonymized finding in-preparation, in-legal-review, or submitted at all times

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| Industry contributions per year (MITRE ATLAS / AVID / OWASP) | ___ | ___ | ≥4 | ☐ | |
| Open regression corpora published and maintained upstream | ___/4 | ___/4 | ≥4 corpora published | ☐ | |
| Contributions accepted as new or refined techniques | ___ | ___ | ≥1 per year | ☐ | |
| Published corpus recency, months since last upstream update | ___ | ___ | ≤1 month | ☐ | |

**Metric Collection Guidance:**
- **Contribution count**: Contribution log filtered to the last 12 months; count submissions with submission confirmation. Source: contribution log
- **Open corpus publication**: Confirm ≥4 corpora exist in the designated public repository under a permissive license. Source: external repository
- **Acceptance rate**: Cross-reference contribution log with ATLAS/AVID/OWASP acknowledgment records. Source: contribution log × external acknowledgments
- **Corpus recency**: Last commit date to the public repository per corpus. Source: public VCS

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No industry contributions or open corpus publication)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

### Question 9: Industry-Shared Exercises

**Q9.1:** Has the program hosted at least 1 industry-shared red-team exercise per year and participated in ≥2 additional cross-org exercises, with documented cross-org detection-benchmark improvement data from participants?

**Evidence Required:**
- [ ] Exercise log on file: ≥1 hosted exercise per year (OWASP AI chapter, ATLAS practitioner table, or sector ISAC AI red-team); ≥2 additional cross-org exercises participated
- [ ] Hosted exercise documented: agenda, participant list, benchmark methodology, measurement of detection improvement before and after the exercise
- [ ] Cross-org detection-benchmark improvement data collected from participants and documented in exercise report
- [ ] Industry-exercise calendar: next hosted or co-hosted exercise scheduled at least 60 days in advance
- [ ] Participation in AISI Inspect evaluation benchmarks, HELM safety evaluations, or sector ISAC AI red-team exercises with anonymized findings on record
- [ ] Published regression corpora cited or adopted by at least one external organization; adoption documented

**Outcome Metrics:**
| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| Industry-shared exercises per year (hosted + participated) | ___ | ___ | ≥1 hosted + ≥2 participated | ☐ | |
| Cross-org detection-benchmark improvement documented | No / Yes | No / Yes | Yes | ☐ | |
| Open regression corpora published and maintained upstream | ___/4 | ___/4 | ≥4 corpora published | ☐ | |
| Industry-shared exercises per year | ___ | ___ | ≥1 hosted + ≥2 participated | ☐ | |

**Metric Collection Guidance:**
- **Exercise count**: Exercise log filtered to the last 12 months; count hosted and participated exercises. Source: exercise log
- **Improvement documentation**: Hosted exercise report; participant pre/post detection benchmark data. Source: exercise reports
- **External adoption**: External organizations that have cited or forked the public corpora; check repository forks, community citations. Source: public repository analytics + community references
- **Calendar check**: Confirm a specific future exercise date ≥60 days out is on the program calendar. Source: program calendar

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No industry-shared exercises conducted or planned)

**Evidence Location:** _______________
**Validation Date:** _______________
**Notes:** _______________

---

## Summary Scorecard

| Level | Q# | Question | Score | Weight |
|-------|-----|----------|-------|--------|
| L1 | Q1 | Per-Archetype Test Battery | ___ | |
| L1 | Q2 | Regression Corpora in CI | ___ | |
| L1 | Q3 | Go-Live Battery and IM Wire | ___ | |
| **L1 Score** | | | **___** | 0.5 |
| L2 | Q4 | Tier-Calibrated Battery and Corpus Depth | ___ | |
| L2 | Q5 | Scheduled Red-Team Exercises | ___ | |
| L2 | Q6 | Cross-Archetype Composition Tests | ___ | |
| **L2 Score** | | | **___** | 0.3 |
| L3 | Q7 | Continuous Automated Adversarial Testing Harness | ___ | |
| L3 | Q8 | Industry Contributions | ___ | |
| L3 | Q9 | Industry-Shared Exercises | ___ | |
| **L3 Score** | | | **___** | 0.2 |
| **Overall ST-Software Score** | | | **___** | |

**Maturity Level Achieved:** ☐ L1 ☐ L2 ☐ L3

**Assessment Date:** _______________
**Assessor:** _______________
**Next Review Date:** _______________

---

**Document Version:** HAIAMM v3.0
**Practice:** Security Testing (ST)
**Domain:** Software
**Questionnaire Authored:** 2026-05-15
**Author:** Verifhai