Threat Assessment (TA)

Software Domain - HAIAMM v3.0

Practice Overview

Objective: Build and maintain a reusable threat library for AI/HAI software the organization builds, one archetype-level threat model per software artifact type, so every intake into the SM inventory produces a threat snapshot in minutes rather than a blank-page exercise.

Description: TA-Software catalogs the threats specific to AI/HAI software the organization ships, not tools it uses to do security. At L1 the library covers one threat model per software archetype (LLM-integrated app, autonomous agent, RAG pipeline, fine-tune/training workload, evaluation harness, model-serving service, classical ML) mapped to HAIAMM's HAI-specific TTPs (EA, AGH, TM, RA), to MITRE ATLAS tactics (TA0001–TA0014), and to OWASP LLM / Agentic Top 10. Each new artifact registered in SM's inventory generates a threat snapshot by pulling the archetype model and adding artifact-specific deltas (specific tool list, specific retrieval sources, specific fine-tune data classes). L2 layers per-artifact deep models for Critical-tier cases and red-teams the library quarterly. L3 contributes discovered TTPs back to MITRE ATLAS, OWASP, and AVID.

Context: Engineering teams shipping AI/HAI software face failure modes that classic AppSec threat modeling was not designed to enumerate, prompt injection at the application boundary, tool-scope overreach in agent loops, retrieval poisoning in RAG pipelines, training-data leakage in fine-tune workloads, output-integrity regression when the underlying model family changes. These are first-party risks owned by the teams that write the code and train the models. TA-Software closes the gap by making AI/HAI-specific threats a first-class library that threat modelers pull from at every intake, and by tying every archetype threat to a specific ATLAS tactic so the walk from attacker capability to artifact exposure is concrete, not narrative.

Maturity Level 1

Objective: Build the AI/HAI software archetype threat library, integrate a threat snapshot into every SM intake, and ensure every artifact's threat surface is documented before production landing

At this level, the organization gives every threat modeler and AppSec reviewer a reusable, archetype-keyed library that maps AI/HAI software failure modes to HAI TTPs and ATLAS tactics, so no AI/HAI software artifact enters production without a documented threat view.

Dependencies

• SM-Software L1 (required): the AI/HAI software inventory defines which archetypes exist and what artifacts need threat models; without it, TA operates on guesswork about scope.
• PC-Software L1 (required): the priority compliance map identifies which regulatory obligations (EU AI Act Art. 26, GDPR Art. 22, EU AI Act Annex III high-risk systems) must be reflected in the threat library.
• EG-Software L1 (required for reviewer activity): reviewers must be able to recognize AI/HAI-specific archetypes and TTPs before they can produce a credible snapshot.
• Supports / unblocks: SR-Software L1 (requirements derive from archetype threats), SA-Software L1 (reference patterns cover archetype threat surfaces), ST-Software L1 (test battery targets archetype threats), ML-Software L1 (detections prioritized from threat library), IM-Software L1 (incident classifications derive from the threat library).

Desired Outcomes

• Every AI/HAI software artifact reaching SM intake gets a threat snapshot within one business day, pulled from the archetype library and adapted to the specific artifact's tool list, data classes, and retrieval sources.
• HAIAMM's HAI-specific TTPs, EA (Excessive Agency), AGH (Agent Goal Hijack), TM (Tool Misuse), RA (Rogue Agents), are tagged to each archetype's threats; reviewers can explain the tag and its implications for the specific artifact.
• MITRE ATLAS tactics (TA0001 Reconnaissance through TA0014 Impact) are walked for each archetype at intake; a tactic with no relevant technique is an explicit exclusion, not a gap.
• The threat library is versioned, owned, and refreshed on a documented cadence, it does not rot as new AI attack research emerges.
• Downstream practices (SR, SA, ST, ML, IM) inherit the library rather than re-deriving threats per artifact.

Activities

A) Build the AI/HAI software archetype threat library

Author one threat model per AI/HAI software archetype. Each archetype model is concise (target ≤2 pages), explicitly scoped to first-party artifacts the organization builds, and maps threats to HAI TTPs, ATLAS tactic IDs, OWASP LLM/Agentic Top 10 references, and the PC-Software priority compliance map.

Archetypes to cover at L1 (from SM-Software's inventory schema):

• LLM-integrated application or feature, a product or internal tool that calls an LLM provider API to generate, classify, or summarize content.
• Autonomous AI agent, a tool-using, sometimes multi-step orchestrator that reads and writes to org or customer systems on behalf of a user or autonomously.
• RAG / retrieval-augmented generation pipeline, an artifact that retrieves from a vector store or document corpus and passes retrieved context to a model for generation.
• Fine-tuning or model-training workload, a workload that trains or fine-tunes a model on org data (customer data, internal corpora, proprietary annotations).
• Evaluation / red-team harness, an artifact that runs test inputs through a model and scores outputs for quality, safety, or performance.
• Model-serving service, an internal or externally-exposed API that puts a model (own-hosted or vendor-fine-tuned) behind production traffic.
• Classical ML model in a product surface, a non-LLM ML model (classifier, recommender, anomaly detector) integrated into a customer or employee-facing workflow.

Per-archetype threat content (minimum):

• EA, Excessive Agency patterns: tool scopes defined more broadly than the narrowest function that satisfies the use case; agent permission model wider than any individual human operator's permissions for the same action; side-effects reaching systems outside the stated scope (sending email, modifying records, calling external APIs, provisioning resources).
• AGH, Agent Goal Hijack patterns: indirect prompt injection via retrieved document, tool response payload, multi-turn conversation history, or shared memory store; system-prompt exfiltration via crafted user inputs; persona or instruction override via jailbreak framing; cross-agent instruction injection in multi-agent workflows.
• TM, Tool Misuse patterns: argument smuggling (attacker-controlled values passed as tool parameters); unexpected tool combinations (chaining two safe tools to achieve a prohibited effect); recursive or looping tool invocation that amplifies scope or cost; crafted parameters that exploit missing input validation on the tool side.
• RA, Rogue Agents patterns: long-session goal drift as context accumulates; reflective-loop divergence when an agent evaluates its own outputs and compounds errors; multi-agent miscoordination where one agent's output becomes a goal-hijack vector for another; autonomous continuation past a natural stop condition.
• Additional archetype-specific threats:
• LLM-integrated app: prompt injection at the application input boundary (direct, via form fields, document uploads); output misrepresentation (hallucination treated as authoritative by downstream logic); inference from completion revealing training data.
• RAG pipeline: retrieval poisoning (attacker-controlled documents in the corpus containing injected instructions); retrieval source provenance gaps (no lineage from retrieved context to source); cross-tenant retrieval leakage in multi-tenant stores.
• Fine-tune/training workload: training-data leakage (the fine-tuned model memorizes and regurgitates regulated data); poisoned training data (adversarially labeled examples embedded in the training corpus); consent-base gap (data used for fine-tuning outside its original consent scope or GDPR lawful basis); model backdoor introduced during training.
• Agent: kill-switch absence or bypass; human-in-the-loop gate failure (automation flag set, gate skipped); credential or secret exposed in tool-call log.

• Model-serving service: output-integrity regression on silent model-family swap; model version rollback without stakeholder awareness; denial-of-inference via resource exhaustion.

• ATLAS tactic walk per archetype: For each archetype, walk the full ATLAS tactic sequence and document which techniques apply and which are excluded with rationale:

• TA0001 Reconnaissance, how an adversary would learn about the artifact's inputs, retrieval sources, tool surfaces.
• TA0002 Resource Development, staging hostile documents, poisoned fine-tune datasets, adversarial evaluation inputs.
• TA0003 Initial Access, prompt injection as the primary initial-access vector for all LLM-based archetypes.
• TA0004 Execution, tool invocation, code execution via output handling, agent loop execution.
• TA0005 Persistence, backdoor in fine-tuned model, injected instruction surviving context window turns.
• TA0006 Privilege Escalation, EA patterns; agent gaining access to systems outside its scope by chaining tools.
• TA0007 Defense Evasion, obfuscated injection payloads; adversarial inputs designed to evade safety layers.
• TA0008 Credential Access, secrets exposed in tool-call logs; credentials in system prompt; API key exfiltration via completion.
• TA0009 Discovery, model interrogation to infer system-prompt content, training data, or internal tool schemas.
• TA0010 Collection, training-data extraction via membership inference; RAG corpus harvesting via repeated retrieval.
• TA0040 ML Attack Staging, adversarial examples, evasion inputs, model inversion probes.
• TA0011 Exfiltration, data extracted via model completions, tool outputs, or side-channel timing.

• TA0043 Impact, harmful content generation; denial of inference; integrity regression in decision-affecting outputs; downstream process manipulation via rogue agent.

• OWASP LLM / Agentic Top 10 cross-references per archetype: tag each archetype's dominant threats to the relevant OWASP LLM Top 10 (2025) item. Agents map primarily to LLM01 (Prompt Injection), LLM06 (Excessive Agency), LLM07 (System Prompt Leakage), and the Agentic Top 10. Fine-tune workloads map primarily to LLM03 (Training Data Poisoning) and LLM08 (Vector and Embedding Weaknesses). RAG pipelines map to LLM02 (Insecure Output Handling), LLM08, and LLM01 (indirect injection via retrieval).

• Compliance linkage: tag each threat to the PC-Software priority compliance item it activates, EU AI Act Art. 26 deployer duties (accuracy, robustness, cybersecurity under Art. 15), GDPR Art. 22 (automated decisioning safeguards), EU AI Act Annex III high-risk use triggers.

Owner: named TA-Software library steward; cadence: reviewed quarterly; versioned in a single location linked from the SM inventory record for every artifact.

B) Produce a per-intake threat snapshot for every SM inventory registration

Bind TA into the SM intake flow, every new artifact registration emits a threat snapshot before the Sanctioned status is issued; Provisional-status artifacts receive a snapshot within five business days of registration.

Snapshot contents (designed to fit one screen):

• Which archetype(s) apply (an artifact may be composite, e.g., a customer-facing agent that also does fine-tuning from interaction logs is both agent and fine-tune/training workload).
• Artifact-specific deltas over the archetype model: specific tool list and scope; specific retrieval sources and their trust level; specific fine-tune data classes and their consent basis; specific output-integrity-critical decision paths.
• Top-5 threats for this artifact, each with: HAI TTP tag, ATLAS tactic ID, OWASP reference, and compliance linkage.
• Controls already evident from the design or existing architecture vs. gaps for SR/SA follow-up.
• Reviewer, date, expiry (re-snapshot on model-family swap, new tool addition, new retrieval source, scope change, or material code change).

Time target: ≤1 business day per intake with the library available. Most threats come pre-written in the archetype model; the reviewer adapts rather than invents.

C) Author the shadow-AI-in-engineering threat view

Shadow AI in engineering, LLM calls shipped without intake, agents wired into products behind feature flags, fine-tunes run from researcher notebooks, has its own threat surface distinct from sanctioned artifacts. Author a standalone shadow-AI-in-engineering threat document covering:

• Entry vectors: unannounced LLM API calls in new feature branches; direct SDK imports without security review; researcher-run fine-tuning from Jupyter notebooks; agents wired to production APIs behind feature flags; developer-built eval harnesses consuming production data.
• Elevated threats for shadow artifacts: no threat model (TA snapshot missing, this whole library was never applied); no SR requirements pack; no design review; no security testing; deployer duties under EU AI Act Art. 26 are unmet because the artifact is unknown to the deployer-duty owners.
• Specific failure modes: training on regulated production data (customer PII reaching a fine-tuning job outside GDPR lawful basis); agent with tool access to production APIs deployed without kill-switch; RAG pipeline over confidential documents with no retrieval-source provenance.
• Detections available at L1 (from SM discovery sources): source-code signals (LLM SDK imports in feature branches); CI/CD telemetry (jobs training models or calling LLM providers); runtime egress (outbound calls to AI provider domains from services without corresponding inventory entries); cloud-spend signals (untagged AI API usage).

Output: a "Shadow AI in Engineering, Threat View" one-pager reviewed by the program sponsor and feeding the ML-Software detection backlog and the IM-Software triage playbook.

Outcome Metrics (L1)

Process Metrics (leading)

• Threat library review cadence, quarterly archetype refresh recorded; no quarter with zero updates.
• New-archetype lead time, from "first intake in new software category" to "archetype model published" ≤30 days.
• Snapshot-to-SR linkage, % of snapshots whose top-5 threats are referenced by at least one downstream SR-Software requirement (expected to rise once SR-Software L1 is in place).
• Library steward named and active, single owner, not a committee.

Effectiveness Metrics (business value)

• Threats that converted to prevented production issues, documented cases where a snapshot-identified threat caused a control to be added before production landing (e.g., kill-switch added after agent snapshot flagged EA; RAG retrieval-source allow-listing added after snapshot flagged retrieval poisoning).
• Reviewer consistency, EG-Software calibration exercises use live threat snapshots; inter-reviewer threat-identification drift stays within target.
• Downstream reuse, SR, SA, ST artifacts cite snapshot threats in ≥80% of cases rather than re-deriving independently.

Success Criteria

• Seven archetype threat models published (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), each tagged to HAI TTPs, ATLAS tactic IDs, OWASP LLM/Agentic Top 10 references, and the PC-Software priority compliance map.
• Threat snapshot gate live in the SM intake flow, 100% of newly Sanctioned AI/HAI software artifacts in the last 90 days have a snapshot attached.
• Shadow-AI-in-engineering threat view published, reviewed by the program sponsor, and feeding the ML-Software detection backlog.
• Named library steward and quarterly refresh cadence operating.
• ≥90% of active AI/HAI software artifacts in the inventory carry a current-year snapshot.

Maturity Level 2

Objective: Layer per-artifact deep threat models on top of archetype snapshots for Critical-tier artifacts, integrate external AI security threat intelligence, and red-team the threat library quarterly against novel real-world attack patterns

At this level, threat assessment stops being "snapshot plus go" for high-stakes artifacts. Critical-tier artifacts (per SM L2's risk-tier rubric) receive full per-artifact deep threat models with adversarial-ML overlays. External threat intel (MITRE ATLAS updates, AVID submissions, OWASP LLM/Agentic Top 10 revisions, sector ISACs, academic adversarial-ML research) is wired in and triaged quarterly. The library is stress-tested by running a quarterly red-team probe against real in-scope integrations to surface what the library catches and what it misses.

Dependencies

• TA-Software L1 (required): archetype threat library and per-intake snapshot gate.
• SM-Software L2 (required): the risk-tier rubric determines where per-artifact deep modeling is required (Critical) vs. archetype-only (Medium/Low); without L2 tiers, every artifact appears equally important and deep modeling cannot be calibrated.
• ST-Software L2 (required for red-team-the-library activity): the red-team capability to probe real integrations comes from ST.
• Supports / unblocks: SR-Software L2 (per-tier threat depth drives per-tier requirements), SA-Software L2 (threat depth drives pattern complexity), DR-Software L2 (scenario-based design reviews need per-artifact models), ML-Software L2 (detections tuned to per-artifact threats).

Desired Outcomes

• Every Critical-tier AI/HAI software artifact has a current-year per-artifact deep threat model, not a recycled archetype snapshot.
• High-tier artifacts receive archetype snapshot plus artifact-specific deltas and an ATLAS full tactic walk; no High-tier artifact on archetype-only.
• External AI-security threat intel is routinely consumed and reflected in the library; the library is not frozen at publication time.
• The library is proved against reality, quarterly red-team exercises show what it catches and misses; gaps are closed with owners and expiry dates.
• Per-tier threat-assessment depth is visibly differentiated, matching the SM L2 tier-treatment matrix.

Activities

A) Per-artifact deep threat modeling for Critical-tier artifacts

For every Critical-tier artifact in the SM inventory, produce a full per-artifact threat model covering:

• Attack trees beyond the archetype snapshot: artifact-specific tool list with per-tool abuse paths; artifact-specific retrieval sources with provenance chain and injection surface; specific fine-tune data classes with memorization risk quantified (data class, corpus size, approximate risk of extraction); artifact-specific output-integrity-critical paths with downstream consequence analysis.
• Abuse-case catalog: named adversary archetypes (external attacker, malicious insider, compromised subprocessor, compromised vendor model family) with concrete attack narratives for this specific artifact.
• Deployer-duty mapping: EU AI Act Art. 26 obligations mapped to the threat-control chain specific to this artifact (accuracy and robustness requirements under Art. 15, human oversight requirements under Art. 14, documentation under Art. 13).
• ATLAS full tactic walk for the artifact: all 14 ATLAS tactics enumerated; techniques selected from the AI-Attack-Taxonomy for this specific archetype, tooling, and data boundary; exclusions explicit with rationale.
• Refresh cadence: Critical semi-annual plus change-driven (model-family swap, new tool, new retrieval source, scope change); High annual plus change-driven.

B) External AI-security threat intelligence integration

Subscribe to and operationalize: - MITRE ATLAS updates (new technique additions, technique refinements, practitioner-submitted evidence). - AVID (AI Vulnerability Database), new entries for techniques relevant to the org's software archetypes. - OWASP LLM Top 10 and Agentic Top 10 revisions, new items, revised severity, real-world evidence notes. - Academic adversarial-ML venues (IEEE S&P, USENIX Security, NeurIPS ML Safety, AdvML workshops), early signal on novel attack classes before they reach ATLAS. - Sector ISAC AI working groups, operationally-observed attack patterns relevant to the org's industry.

Quarterly triage cadence: which new intel items change the archetype library, change per-artifact models, or change the SR or ST artifacts that depend on the library. Changes are change-logged; change-log reviewed by the library steward and the IM backlog owner.

C) Red-team the threat library itself

Each quarter, ST-Software runs an adversarial probe against an in-scope AI/HAI software artifact using ONLY the threat scenarios documented in the library for that archetype. Threats the red-team exercise identifies that are NOT in the library are library gaps, not passing findings.

Gap closure is a governance activity: every gap becomes a ticket with a named owner and an expiry date; Critical-tier gaps close within 30 days; High-tier gaps within 60 days. The gap rate per quarter trends down as the library matures. Gaps are also reviewed for SR and ST update implications, if a threat is not in the library, it is also likely not covered by a requirement or a test.

Outcome Metrics (L2)

Process Metrics (leading)

• Library change-log cadence, no quarter with zero changes.
• Per-artifact deep model age, no Critical model older than 180 days; no High model with archetype snapshot alone.
• Red-team-the-library exercise cadence, at least quarterly; exercise artifact on file.
• Gap closure SLA tracked; no Critical gap open past 30 days.

Effectiveness Metrics (business value)

• Incidents caught by pre-existing library entries vs. library gaps, ratio trends toward pre-existing over time.
• Downstream reuse at tier, SR, SA, DR, ST artifacts for Critical-tier artifacts cite per-artifact threats in ≥80% of cases rather than generic archetype content.
• Library becomes a named internal resource, engineering leads and product architects consult it before designing AI features, reducing DR send-backs caused by uncaptured threat surfaces.

Success Criteria

• Per-artifact deep threat models live for 100% of Critical-tier and ≥90% of High-tier artifacts, with refresh cadence met.
• External threat intel integrated with quarterly triage and documented change-log.
• Quarterly red-team-the-library exercise operating; gaps closed with named owners and expiry dates.
• Intel-to-library update lead time ≤30 days on Critical-impact items.

Maturity Level 3

Objective: Automate threat-library maintenance from telemetry and external feeds, and contribute discovered AI/HAI software TTPs back to MITRE ATLAS, OWASP, and AVID

At this level, the threat library is self-updating. Telemetry from ML-Software detections and incident patterns from IM-Software, combined with external signal feeds (ATLAS, AVID, OWASP revision pipelines), auto-propose library updates; human curators review and approve. The program contributes emerging first-party-observed TTPs, attack patterns discovered in own-built AI/HAI software, to MITRE ATLAS, the OWASP LLM/Agentic Top 10, and AVID, making the org a net contributor to the industry's AI threat knowledge base.

Dependencies

• TA-Software L2 (required): per-artifact models and external intel integration must be operational before automation is trustworthy, automation of a weak library produces fast-decaying noise.
• ML-Software L2+ (required): the detection telemetry that proposes library updates comes from the monitoring pipeline; without it, the auto-proposal loop has no signal.
• IM-Software L2+ (required): incident pattern data feeds update proposals for threat actors, TTPs, and impact scenarios.

Desired Outcomes

• Library staleness is measured in weeks, not quarters, material new attack patterns reach the library within 14 days of first observation in external feeds or internal telemetry.
• Program-sourced TTPs appear in MITRE ATLAS, AVID, and OWASP LLM/Agentic Top 10 revisions, the org is cited as a practitioner contributor.
• External threat-landscape shifts are reflected in the library before most peers who lack an automated signal pipeline.
• The org becomes a trusted node in AI-software-security threat sharing (sector ISACs, ATLAS practitioner network, OWASP working groups).

Activities

A) Telemetry-driven library updates

Wire the following signal sources into an auto-proposal pipeline: - ML-Software detections, alert patterns that do not map to any existing library entry are surfaced as candidate new threats. - IM-Software incident records, post-incident review records generate structured threat updates; the incident's ATLAS tactic walk is auto-ingested. - External feeds, ATLAS technique additions, AVID new entries, OWASP LLM/Agentic Top 10 revision drafts, sector-ISAC AI-specific advisories. - Academic publication scanning, weekly digest of adversarial-ML papers; new attack classes auto-tagged for human curator review.

Human curators approve, reject, or defer each auto-proposal. Change-log is machine-readable; downstream SR, SA, ST artifacts subscribe to the change feed and receive update-required notifications when a threat they reference changes.

B) Industry contribution

Contribute to: - MITRE ATLAS, new techniques observed in own-built AI/HAI software (prompt-injection variants, new agent-loop attack patterns, novel retrieval-poisoning mechanics); submissions follow ATLAS evidence-and-provenance requirements. - OWASP LLM Top 10 / Agentic Top 10, substantive comments and real-world telemetry evidence submitted during revision cycles. - AVID, structured disclosure submissions for newly discovered vulnerabilities in own-built artifacts or upstream dependencies (coordinated disclosure where third-party components are involved). - NIST AI RMF Playbook successor editions, practitioner input grounded in program experience.

Target: minimum 4 substantive contributions per year; quality-graded and legally vetted before submission; every contribution anonymized.

C) Shared threat-model artifacts

Publish anonymized archetype threat models (scrubbed of org-specific tool names and data classes) under a permissive license for peer-org adoption. Host or co-host at least one industry tabletop per year tied to the library (ATLAS practitioner table, OWASP AI chapter, sector ISAC AI working group).

Outcome Metrics (L3)

Process Metrics (leading)

• Auto-proposal pipeline health, ≥1 actionable auto-proposal per week; staleness alert if feed silent for 7 days.
• Contribution pipeline always ≥2 in-flight (draft, in-review, or being prepared).
• External tabletop cadence, at least 1 per year.
• Library change-log machine-readable and consumed by at least one downstream practice (SR or ST) for auto-update notifications.

Effectiveness Metrics (business value)

• Program cited in industry advisories as a practitioner contributor to ATLAS / OWASP / AVID.
• Time-to-defend shrinks for library-sourced threats, controls are in place before incidents because the library leads external disclosure timelines.
• Talent signal, adversarial-ML and AI security engineering talent is attracted by the program's external profile and contribution record.

Success Criteria

• Library auto-update pipeline operating with ≤14-day lead time from signal to update.
• ≥4 industry contributions per year; ≥2 recognized in external artifacts (ATLAS merge, AVID entry, OWASP revision).
• Anonymized archetype threat models published under permissive license with tracked adoption.
• Industry tabletop hosted or co-hosted in last 12 months.

Key Success Indicators

Level 1: - Seven archetype threat models published (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), each tagged to HAI TTPs (EA/AGH/TM/RA), ATLAS tactic IDs (TA0001–TA0014), OWASP LLM/Agentic Top 10 references, and the PC-Software priority compliance map. - Threat snapshot gate live in the SM intake flow, 100% of newly Sanctioned AI/HAI software artifacts in the last 90 days have a snapshot attached before Sanctioned status is issued. - Each snapshot documents: archetype(s), artifact-specific deltas (tool list, retrieval sources, data classes), top-5 threats with HAI TTP tags and ATLAS tactic IDs, controls evident, and gaps for SR/SA follow-up. - Shadow-AI-in-engineering threat view published, reviewed by the program sponsor, and linked to the ML-Software detection backlog and the IM-Software triage playbook. - Named library steward, quarterly refresh cadence, and ≥90% of active AI/HAI software artifacts carrying a current-year snapshot.

Level 2: - Per-artifact deep threat models live for 100% of Critical-tier artifacts and ≥90% of High-tier artifacts, with refresh cadence (Critical semi-annual, High annual) met. - External AI-security threat intel (ATLAS, AVID, OWASP LLM/Agentic, academic venues, sector ISACs) integrated with quarterly triage and a documented change-log; intel-to-library update ≤30 days for Critical-impact items. - Quarterly red-team-the-library exercise operating; every gap closed with a named owner and expiry date; Critical gaps ≤30 days, High gaps ≤60 days.

Level 3: - Library auto-update pipeline operating with ≤14-day lead time; ≥60% of changes auto-proposed; change-log machine-readable and consumed by downstream SR and ST practices. - ≥4 substantive industry contributions per year to MITRE ATLAS / AVID / OWASP LLM / Agentic Top 10, with ≥2 externally recognized. - Anonymized archetype threat models published under permissive license with tracked peer-org adoption; at least one industry tabletop hosted or co-hosted annually.

Common Pitfalls

Level 1: - ❌ Threat models describe "the AI" as the actor performing the security work (v2.0 framing) rather than describing the first-party AI software artifact as the subject being assessed, the library ends up cataloging what AI tools do rather than what threats face the software the org ships. - ❌ Archetype library covers only the archetypes currently in production; eval harnesses and fine-tune workloads are excluded because "they're not customer-facing", shadow AI in engineering gets no threat surface. - ❌ Threat snapshot is a checklist checkbox, not a living artifact, snapshots are marked complete without artifact-specific deltas (tool list, retrieval sources, data classes), making them useless for SR and ST follow-through. - ❌ ATLAS tactic walk is performed for narrative completeness but no technique IDs are assigned, the walk produces prose, not structured references that ST and IR can act on. - ❌ HAI TTPs (EA/AGH/TM/RA) are listed in the library header but not tagged per-threat, reviewers cannot triage which threats matter for a specific archetype's failure modes. - ❌ Library steward is unnamed, "the security team owns it", so the quarterly refresh calendar item is no one's job and the library drifts from current attack research within two quarters.

Level 2: - ❌ "Per-artifact deep model" is the archetype snapshot with the artifact name swapped in, no artifact-specific tool list, no retrieval-source provenance analysis, no per-data-class memorization risk; the depth is cosmetic. - ❌ External intel is subscribed but never triaged, ATLAS update emails pile up unread; the library is frozen at L1 publication while the external threat landscape evolves. - ❌ Red-team-the-library exercise is a threat-hunting session that adds entries to a finding log but never cross-checks findings against the library, gaps are never surfaced because the comparison was never made. - ❌ Critical-tier accepted gaps from the library red-team lack owners or expiry dates, gap backlog grows without accountability. - ❌ Deep modeling stops at Critical; High-tier artifacts remain on archetype-only snapshots despite carrying regulated data or significant agent scope.

Level 3: - ❌ Auto-proposal pipeline accepts signals without curation, false-positive detections from ML-Software pollute the library with phantom threats; downstream SR and ST artifacts generate incorrect requirements and tests. - ❌ "Contributions" to MITRE/AVID/OWASP are observer submissions (comments, +1s, conference talks) rather than technical artifacts with evidence, they appear in the contribution log but produce no substantive change. - ❌ Published anonymized archetype models are not maintained after release, external adopters build on a stale version while the internal library has advanced; the gap becomes an embarrassment when discrepancies are cited. - ❌ Telemetry-driven update loop fires on every minor model-configuration change, overwhelming the curation queue, engineering teams disable telemetry to stop the noise rather than tune the signal sensitivity.

Practice Maturity Questions

Level 1: 1. Are there published, versioned threat models for all seven AI/HAI software archetypes (LLM-integrated app, agent, RAG, fine-tune/training, eval harness, model-serving service, classical ML), each mapping archetype-specific threats to HAI TTPs (EA/AGH/TM/RA), ATLAS tactic IDs, OWASP LLM/Agentic Top 10 references, and PC-Software compliance items, with a named library steward and a documented quarterly refresh cadence? 2. Does every AI/HAI software artifact entering the SM inventory receive a threat snapshot (delivered within one business day of intake) that documents: the applicable archetype(s), artifact-specific deltas (tool list, retrieval sources, data classes, output-integrity-critical paths), top-5 threats with HAI TTP tags and ATLAS tactic IDs, and gaps for SR/SA follow-up, with 100% of newly Sanctioned artifacts carrying a snapshot in the last 90 days? 3. Is there a published shadow-AI-in-engineering threat view, reviewed by the program sponsor in the last 12 months, that documents entry vectors, elevated threat scenarios for unreviewed AI/HAI software artifacts, and the specific detections (from SM discovery sources) used to surface them?

Level 2: 1. Does every Critical-tier AI/HAI software artifact have a current-year per-artifact deep threat model (not an archetype snapshot) covering artifact-specific attack trees, an abuse-case catalog, deployer-duty mapping, and a full ATLAS tactic walk with technique-level specificity, with a semi-annual refresh cadence and change-driven updates on model swap, new tools, or scope change? 2. Is external AI-security threat intel (MITRE ATLAS updates, AVID, OWASP LLM/Agentic Top 10 revisions, sector ISACs, academic adversarial-ML venues) integrated with a quarterly triage cadence and a documented change-log, with intel-to-library update ≤30 days on Critical-impact items? 3. Do you run a quarterly red-team-the-library exercise that probes an in-scope AI/HAI software artifact using only library threats and surfaces misses as library gaps, with every gap carrying a named owner and an expiry date, Critical gaps closing within 30 days, and the gap rate trending down quarter over quarter?

Level 3: 1. Does the threat library auto-update from telemetry (ML-Software detections, IM-Software incidents) and external feeds (ATLAS, AVID, OWASP, academic) via a human-curated auto-proposal pipeline, with ≥60% of changes auto-proposed, a ≤14-day lead time from signal to update, and a machine-readable change-log consumed by downstream SR and ST practices? 2. Does the program contribute at least four substantive, evidence-backed technical artifacts per year to MITRE ATLAS / AVID / OWASP LLM/Agentic Top 10, with at least two externally recognized in published advisory or standard revisions? 3. Are anonymized archetype threat models published under a permissive license with tracked peer-org adoption, and does the program host or co-host at least one industry tabletop per year (ATLAS practitioner table, OWASP AI chapter, or sector ISAC AI working group) tied to the library?

Document Version: HAIAMM v3.0 Practice: Threat Assessment (TA) Domain: Software Last Updated: 2026-05-13 Author: Verifhai