Issue Management (IM)

Software Domain - HAIAMM v3.0

Practice Overview

Objective: Run the single unified backlog for AI/HAI issues across the Software domain, findings from TA snapshots, SR gaps, DR conditions, IR drifts, ST failures, ML detections, and external advisories, with a tier-calibrated incident playbook containing AI-specific containment plays, and regulatory SLA tracking (GDPR Art. 33, EU AI Act Art. 73, HIPAA, sector-specific).

Description: IM-Software is the clearinghouse for everything the other Software-domain practices produce. Every TA threat snapshot row that carries residual risk, every SR REM accepted gap with an owner and expiry, every DR approve-with-conditions item, every IR drift finding, every ST CI corpus failure or red-team finding, every ML detection that fires, and every external advisory (MITRE ATLAS updates, OWASP LLM updates, AVID entries, model-vendor advisories) flows into a single, prioritized backlog with named owners, tier-calibrated SLAs, and an unambiguous incident playbook. The playbook includes AI-specific containment plays, kill-switch execution for rogue agents, model rollback for training-data-leakage or output-integrity incidents, RAG-source disable for retrieval-poisoning events, tool-revoke for tool-scope violations, and shadow-AI egress-block for unsanctioned service emergence. Every Critical or blocker incident receives a post-incident review whose outputs flow back to SA-Software (pattern update), SR-Software (requirements pack update), EG-Software (training content update), and ML-Software (detection update). The regulatory SLA tracker ensures GDPR Art. 33 72-hour breach notification, EU AI Act Art. 73 serious-incident reporting, and HIPAA breach notification windows are never missed because of organizational diffusion.

Context: Without a unified backlog, AI/HAI software issues scatter across product Jira projects, security queues, legal trackers, privacy dashboards, and ML-platform alert channels. TA residual risks age without remediation owners. SR gaps renew silently past their expiry dates. An ML detection fires on a Friday and routes to nobody because the on-call rotation does not cover AI-specific alerts. An agent executes a rogue tool invocation and the first human to know is a customer reporting unexpected behavior two days later. The GDPR Art. 33 72-hour clock starts at the moment the organization becomes aware of a personal data breach, not when the responsible team processes the notification. IM-Software closes all of these gaps with a single backlog, a single triage rubric, and a named on-call path for every severity class.

Maturity Level 1

Objective: Operate a single unified AI/HAI software issue backlog with a standard triage rubric, AI-specific incident playbook including containment plays for the primary HAI incident classes, and regulatory SLA tracking for GDPR Art. 33, EU AI Act Art. 73, HIPAA, and sector-specific obligations

At this level, every AI/HAI software issue has a home, a severity, an owner, and an SLA, and incidents follow a named playbook with AI-specific containment actions rather than generic software incident response.

Dependencies

• SM-Software L1 (required): the inventory provides the affected-artifact and owning-team spine for every issue; without the inventory, the backlog cannot route issues to accountable owners.
• PC-Software L1 (required): priority compliance map anchors the regulatory SLA tracker (GDPR Art. 33, EU AI Act Art. 73, HIPAA, sector-specific); triage rubric severity definitions reference compliance exposure.
• TA-Software L1 (required): archetype threat library drives the incident classification taxonomy and the AI-specific incident classes in the playbook.
• ML-Software L1 (required): detections from ML-Software are the primary runtime input to the issue backlog; without ML L1, the detection-to-backlog feed does not exist and runtime incidents go unreported until they are externally visible.
• Supports / unblocks: ML-Software L1 (post-incident reviews feed detection-tuning back into ML); SA-Software L1 (post-incident reviews generate pattern-update requests); SR-Software L1 (post-incident reviews generate requirements-pack update requests); EG-Software L1 (incident trends feed training-content updates).

Desired Outcomes

• One backlog, one triage rubric, one incident playbook for all AI/HAI software issues, regardless of source practice.
• AI-specific incident classes are handled on named playbook entries with pre-assigned roles, containment steps, and SLA targets, not improvised at incident time.
• Regulatory SLA tracker is live and showing zero missed notification windows for GDPR Art. 33, EU AI Act Art. 73, HIPAA, NYDFS Part 500, and sector-specific obligations in the last 90 days.
• Post-incident reviews for Critical / blocker incidents produce named updates to SA, SR, EG, and ML within 14 days of incident closure, incidents generate organizational learning, not just closure tickets.
• Backlog aging is visible to the program sponsor monthly; a small number of aging buckets are actively managed with clear escalation paths.

Activities

A) Stand up the AI/HAI software issue backlog and triage rubric

One backlog with standardized metadata:

• Source, TA (threat snapshot residual risk) / SR (REM accepted gap) / DR (approve-with-conditions item) / IR (drift finding) / ST (CI corpus failure, red-team finding) / ML (detection alert) / External (MITRE ATLAS advisory, OWASP LLM update, AVID entry, model-vendor advisory, customer report).
• Affected artifact(s), linked to SM-Software inventory; archetype, tier, owning team.
• Severity, Critical / High / Medium / Low per the rubric below.
• Owner, named artifact owner from the SM-Software inventory; escalation path to program sponsor.
• SLA, severity-based (see below; at L1 the SLAs are published even before SM L2 tiering exists, the SLAs are severity-only at L1; they gain per-tier calibration at L2).
• Evidence, link to originating artifact (TA snapshot row, REM gap row, DR decision, IR finding, ST test result reference, ML alert ticket, external advisory URL).
• Regulatory flag, whether the issue carries a regulatory notification obligation (GDPR Art. 33 clock started, EU AI Act Art. 73 clock started, HIPAA breach-notification triggered, sector-specific).

Severity rubric (AI/HAI software specific):

• Critical, active exfiltration of data through an AI/HAI artifact; agent executing tool invocations with real-world damage (account deletions, unauthorized transactions, customer-record modifications outside declared scope); kill-switch failure in a production agent; personal data breach in AI/HAI software processing that triggers GDPR Art. 33; regulated-data exposure in a training or inference pipeline.
• High, confirmed control failure in a production AI/HAI artifact with potential for harm if not contained (output-integrity regression in a customer-facing decision-affecting artifact; tool-scope violation detected but no confirmed damage; HITL-gate bypass attempt in production; RAG retrieval-source poisoning confirmed but output not yet serving customers).
• Medium, confirmed gap in a non-production artifact or a production artifact with compensating controls active; no current active impact; SR REM accepted-gap past expiry with no renewal; IR drift finding on a Medium-tier artifact.
• Low, informational; non-urgent gap; recommendation from an external advisory not yet assessed; Low-tier artifact logging gap.

SLA targets (published at L1, per-tier calibrated at L2):

• Critical: acknowledge ≤4 hours / contain ≤48 hours / root-cause ≤30 days.
• High: acknowledge ≤24 hours / contain ≤7 days / root-cause ≤45 days.
• Medium: acknowledge ≤48 hours / remediate ≤14 days.
• Low: acknowledge ≤5 business days / remediate ≤30 days.

Triage cadence: daily review for Critical and new High; weekly queue review for Medium; monthly aging review for the full backlog.

B) Publish the AI-specific incident playbook

Publish playbook entries for the primary AI/HAI software incident classes. Each entry includes: trigger conditions, named roles (deployer-duty owner, AppSec on-call, Privacy/Legal contact, executive sponsor escalation path), step-by-step containment, artifacts to collect, evidence-capture instructions for the deployer-duty record, closure criteria, and SLA targets.

Playbook entries (at L1, publish all of the following):

Prompt-injection containment: Trigger: ML detection of prompt-injection success (output exfiltration patterns on a production artifact). Containment: (1) disable the affected user-input path or LLM integration via feature flag; (2) assess scope, which principals, which prompt sessions, what completions were affected; (3) check whether completions containing exfiltration patterns reached customers; (4) tune the detection's query to reduce false-positive coverage while the investigation is active; (5) route to Privacy/Legal if PII was exfiltrated (GDPR Art. 33 clock start); (6) rollback the prompt template to the last known-good version if a specific template change correlates to the injection surface. Evidence: ML alert record, affected prompt/completion log export, deployer-duty record update.

Agent rogue-action containment (ATLAS TA0014 Impact): Trigger: AGH detection fire on a production agent; tool-scope-violation detection on a destructive or customer-affecting tool; HITL-gate-bypass detection with confirmed customer-affecting tool execution. Containment: (1) execute kill-switch for the affected agent session (test kill-switch function; document invocation time and scope); (2) revoke the offending tool from the agent's tool allowlist (tool-revoke action) pending investigation; (3) assess full session tool-call log for actions taken prior to detection; (4) reverse any reversible actions (record deletions, API calls with undo paths) under human review; (5) notify the affected customer or internal user as appropriate; (6) engage Privacy/Legal if customer account data was affected. Evidence: full agent session tool-call log export, kill-switch event record, tool-revoke configuration record, deployer-duty record update.

Training-data-leakage containment: Trigger: training-data-leakage canary detection (canary string emitted in completion); membership-inference test from ST indicating high recall probability on regulated data. Containment: (1) roll back the affected model version to the previous production version in the model-serving registry; (2) execute eval-harness replay against the rolled-back version to confirm the leakage is absent; (3) identify the training run that introduced the vulnerable fine-tune (training-job event log); (4) assess whether the leaked data included personal data (GDPR Art. 33 trigger assessment); (5) quarantine the training dataset pending data-provenance review by the SR REM owner; (6) update the training-data provenance record in the REM. Evidence: canary detection alert record, model rollback event record, eval-replay results, training-job event export.

Silent model-family-swap rollback: Trigger: IR drift finding that a model-serving service is delivering completions from a different model family than the approved model version (silent swap by the inference provider). Containment: (1) pin the API call to the approved model version using the model-version parameter; (2) re-run the eval harness against the current production completions to assess output-integrity regression; (3) if output-integrity regression is confirmed, rollback to pinned-version traffic and notify the provider; (4) update the SR REM row for model-version pinning to reflect the confirmed failure mode; (5) trigger a re-review of the IR implementation record. Evidence: IR finding record, model-version parameter confirmation, eval-replay results.

RAG retrieval-source poisoning containment: Trigger: ML detection of injection-defense decision event with flagged retrieved content sourced from a specific document or corpus; ST retrieval-poisoning test finding. Containment: (1) disable the affected retrieval source (document-level or corpus-level) from the RAG query allowlist; (2) initiate re-indexing of the corpus without the poisoned source; (3) assess which users received completions influenced by the poisoned retrieval; (4) notify affected users if the completion materially affected a decision with legal or significant personal effect (GDPR Art. 22 consideration). Evidence: retrieval event log export for the affected period, injection-defense decision log, source-disable configuration record.

Shadow-AI emergence containment: Trigger: ML shadow-AI emergence detection (new outbound flow from a service not in the SM-Software inventory to an LLM provider domain). Containment: (1) block egress from the identified service to the LLM provider domain via the firewall / SASE egress policy; (2) identify the service and owning team from the SM-Software inventory (or open a new inventory record if not present); (3) route the artifact through the SM intake process (amnesty path if appropriate); (4) assess whether any customer or regulated data transited the LLM provider endpoint; (5) if data transited, assess GDPR Art. 33 and EU AI Act Art. 73 obligations. Evidence: shadow-AI detection alert record, egress-block configuration record, inventory record creation, data-transit assessment.

C) Track regulatory SLAs and run post-incident reviews

Regulatory SLA tracker, live, named obligation, with automated escalation on approach:

• GDPR Art. 33, 72-hour supervisory-authority notification window after the controller becomes aware of a personal data breach; clock starts on the first internal alert that constitutes awareness (ML detection, IR finding, external notification). Named owner: Privacy/Legal. If a GDPR Art. 33 clock starts from an AI/HAI software incident, the IM-Software backlog record is flagged; a daily-at-minimum status update is required until the notification is filed or the clock expires.
• EU AI Act Art. 73, serious incident involving a high-risk AI system (Annex III) or an AI system posing an unacceptable risk; reporting timeline per the implementing act (at L1, track and escalate to Privacy/Legal immediately on any Annex III-classified artifact incident). Named owner: Privacy/Legal + executive sponsor.
• HIPAA breach notification, 60-day discovery-to-notification ceiling for covered entities and business associates; individual notification without unreasonable delay; HHS Secretary notification within 60 days. Named owner: Privacy/Legal. Flag any AI/HAI software incident involving PHI immediately.
• NYDFS Part 500, 72-hour notification to the Superintendent for material cybersecurity events affecting covered entities. Named owner: CISO / Privacy/Legal.
• PCI-DSS, cardholder data breach notification requirements; named owner per the org's PCI compliance program.
• FINRA reporting, AI-in-financial-product incidents with material-customer-impact threshold; named owner per the org's FINRA compliance program.

Every Critical or blocker incident receives a post-incident review within 14 days of containment: - What happened: root cause, how the incident initiated, what controls failed or were absent. - What caught it: which ML detection, IM source, or external report surfaced it first; was this the expected detection path or a gap? - What did not catch it: which controls should have detected or prevented this but did not. - Update outputs (all four must be populated for Critical incidents): - SA-Software: pattern-update request if the incident exploited an architectural gap. - SR-Software: requirements-pack update request if the incident exploited a missing or vague requirement. - EG-Software: training-content update request if the incident indicates a literacy gap in the engineering population. - ML-Software: detection-update request (new detection, tuned query, or evidence that an existing detection's query can be sharpened).

Post-incident review outputs are tracked as IM-Software issues of their own (type: improvement); they age against the same process metric cadence as other issues.

Outcome Metrics (L1)

Process Metrics (leading)

• Backlog triage cadence honored, daily Critical/High triage; weekly Medium; monthly aging.
• Playbook runbook rehearsal, at least one tabletop per quarter exercising an AI-specific incident scenario (rotate through the six playbook classes).
• Regulatory SLA tracker reviewed weekly; named owner confirms clock start dates and status.
• Aging pockets, number of issues aging past SLA tracked; trending down.

Effectiveness Metrics (business value)

• Repeat-class incident rate, an incident class occurring twice in 12 months that did not produce a SA/SR/EG/ML update after the first occurrence is a process failure; repeat rate on same class trending down.
• Deployer-duty evidence chain, on an EU AI Act Art. 26 inquiry or GDPR audit, the incident records show the logging, containment, and notification chain for the affected artifact; evidence assembled within ≤5 business days.
• Mean-time-to-contain across Critical and High-severity AI/HAI incidents trending down over quarters.

Success Criteria

• Single AI/HAI software issue backlog established with standardized metadata; triage rubric with AI-specific severity definitions published.
• Six AI-specific incident playbook entries published (prompt-injection, agent rogue-action, training-data-leakage, silent model-family-swap, RAG poisoning, shadow-AI emergence), each with named roles, containment steps, evidence-capture instructions, and SLA targets.
• Regulatory SLA tracker live covering GDPR Art. 33, EU AI Act Art. 73, HIPAA, NYDFS Part 500, and sector-specific obligations; 100% adherence in the last 90 days.
• Post-incident review loop wired to SA, SR, EG, and ML; every Critical/blocker incident produces a review within 14 days with named update outputs.
• Program-sponsor dashboard showing backlog aging, SLA adherence, and post-incident learning outputs refreshed monthly.

Maturity Level 2

Objective: Calibrate incident response depth per SM-Software L2 risk tier; establish dedicated on-call rotation and escalation paths for Critical-tier artifacts; and automate cross-domain signal flow so that Software domain incidents affecting Vendors or Infrastructure generate coordinated response

At this level, incident response differentiates by tier. Critical-tier artifacts have a dedicated on-call rotation, pre-staged executive escalation paths, and a 24/7 coverage model. High-tier artifacts have scoped response with defined escalation. Medium and Low follow the standard queue. Post-incident reviews auto-feed SA, SR, EG, and ML queues via integration rather than manual handoff. When an AI/HAI software incident affects cross-domain scope (a Software artifact whose vendor inference provider is implicated, or an Infrastructure-hosted model whose agent compromised a Processes workflow), coordinated cross-domain response is activated.

Dependencies

• IM-Software L1 (required): unified backlog, AI-specific incident playbook, and regulatory SLA tracker must be operational before per-tier calibration adds meaningful depth.
• SM-Software L2 (required): risk-tier rubric and tier-treatment matrix drive response intensity; without SM L2 tiers, per-tier incident response has no substrate.
• ML-Software L2 (required): richer detections (anomaly-based, cross-artifact correlated) feed severity classification with higher-fidelity signals; tier-calibrated logging depth makes evidence collection faster and more complete at L2.
• Supports / unblocks: ML-Software L2 detection tuning loop (IM L2 post-incident reviews, now with tier-context, produce more targeted detection updates); SA-Software L2 pattern evolution (incidents from Critical-tier artifacts with IaC-encoded patterns drive conformance-test updates).

Desired Outcomes

• Response intensity matches tier, Critical-tier incidents do not wait in the general queue; they activate a named response team, a dedicated on-call path, and the full containment playbook within the published SLA.
• Post-incident review outputs auto-flow to SA, SR, EG, and ML practice backlogs via a defined integration (JIRA webhook, ticket automation, or equivalent) rather than manual handoff, no update gets lost in a spreadsheet.
• Cross-domain coordination is explicit: a Software-domain incident that implicates the Vendors domain (an inference provider's model swap that caused an output-integrity regression) or the Infrastructure domain (a model-serving misconfiguration that exposed the training pipeline) activates a named cross-domain coordination protocol with defined IC and status-board integration.
• Tier-movement in the SM-Software inventory auto-triggers IM policy changes: when an artifact is re-tiered to Critical, the on-call path, playbook variant, and SLA targets are automatically updated in the IM backlog configuration.

Activities

A) Tier-calibrated incident playbook and on-call

Extend L1 playbook entries with tier-specific activation criteria and on-call coverage:

• Critical tier: full IM activation, CISO or delegate + Privacy/Legal + engineering deployer-duty owner + executive sponsor notification; ≤1 hour acknowledgement; ≤4 hours containment-action initiated; 24/7 on-call coverage with a named AI/HAI software incident responder in each on-call rotation; pre-staged communication templates (internal, customer-facing, regulatory) loaded and reviewed quarterly.
• High tier: scoped response team, AppSec lead + Privacy/Legal (if regulated data involved) + deployer-duty owner; ≤4 hours acknowledgement; ≤24 hours containment-action initiated; business-hours on-call with after-hours escalation path defined.
• Medium tier: standard response; ≤1 business day acknowledgement; queue-based triage.
• Low tier: tracked in queue; aggregated weekly handling.

Critical-tier on-call rotation documented: named individuals per week, coverage handoff protocol, on-call briefing that includes the current Critical-tier artifact list and their active detection set.

B) Post-incident review auto-flow integration

• Wire IM-Software's post-incident review outputs to downstream practice backlogs via a defined integration:
• SA-Software pattern-update request → SA-Software architecture-backlog ticket (auto-created with IM incident reference linked).
• SR-Software requirements-pack update request → SR-Software pack-backlog ticket (auto-created with requirements-pack version and failing requirement row linked).
• EG-Software training-content update request → EG-Software training-backlog ticket (auto-created with affected population segment and incident summary linked).
• ML-Software detection-update request → ML-Software detection-registry update ticket (auto-created with detection name, current query, and proposed change linked).
• SLA for downstream updates: Critical-tier post-incident review outputs must be accepted or rejected by the downstream practice owner within 14 days. Accepted updates are treated as High-severity issues in the receiving practice's backlog.
• Post-incident review quality reviewed quarterly by the program sponsor, are the update outputs substantive (concrete change to a pattern, pack, curriculum, or detection) or nominal (a note saying "consider reviewing")?

C) Cross-domain coordination protocol

Publish a cross-domain coordination protocol that activates when a Software-domain AI/HAI incident implicates another domain:

• Software → Vendors: a production artifact's completions are impaired by a model-family swap at the inference provider (silent model swap); activates the Vendors-domain IM playbook entry for vendor material change alongside the Software-domain rollback play. Named cross-domain IC from the Software side.
• Software → Infrastructure: a model-serving service misconfiguration exposes the training pipeline or the model registry to unauthorized access; activates Infrastructure-domain EH and IM alongside Software-domain containment. Named Infrastructure-domain IM contact on file.
• Software → Processes: a production agent makes unauthorized writes to a business-process workflow (customer records, financial records, case management system); activates Processes-domain business-continuity coordinator alongside Software-domain agent-rogue-action play. Named Processes-domain contact on file.

Cross-domain incident activations: shared status board, one unified IC (from the primary impacted domain), coordinated remediation tracking, and a joint post-incident review spanning all affected domains.

Outcome Metrics (L2)

Process Metrics (leading)

• Critical-tier playbook review cadence, quarterly, tested in a tabletop covering the tier's specific artifact list.
• On-call rotation health, no uncovered periods; hand-off briefing completed per rotation; on-call briefing includes updated Critical-tier artifact list.
• Post-incident review quality score, sponsor reviews a sample quarterly; nominal updates flagged for improvement.
• Cross-domain coordination contacts verified quarterly, named contacts are current, communication channels tested.

Effectiveness Metrics (business value)

• Dwell time on Critical-tier incidents (time from first ML detection to containment action complete) trending down as L2 matures.
• Downstream practice update acceptance rate, % of Critical-tier post-incident updates accepted and resolved by the downstream practice; measures whether the feedback loop actually improves the program.
• Cross-domain coordination saves time vs. uncoordinated parallel response, measured as MTTU and MTTC on multi-domain incidents.

Success Criteria

• Critical-tier MTTA ≤1 hour; MTTC ≤4 hours; 24/7 on-call coverage with a documented rotation.
• Post-incident review auto-flow integration live; 100% of Critical-tier review outputs auto-routed to SA/SR/EG/ML backlogs; ≥90% of downstream practice owners responding within 14 days.
• Cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents.
• Tier-movement in SM-Software inventory auto-triggers IM configuration update (on-call path, playbook variant, SLA targets).

Maturity Level 3

Objective: Contribute incident patterns and playbook templates to ISACs, MITRE ATLAS, and AVID; automate runbook decisioning for low-severity, high-confidence detections; and benchmark MTTR against industry peers

At this level, IM-Software is a contributor to the AI-assurance incident-response ecosystem. Anonymized incident classification schemes, AI-specific severity anchors, and playbook templates for the six primary AI/HAI incident classes are contributed to sector ISACs, CSA AI Safety Initiative, AVID, and OpenSSF AI. Pre-authorized automated containment actions execute for low-severity, high-confidence detections without human triage delay, killing a scope-limited agent, blocking an egress flow, or disabling a retrieval source, within seconds of detection. Mean-time-to-resolve benchmarks are established from ISAC and peer data, and the program's MTTR is measured against those benchmarks quarterly.

Dependencies

• IM-Software L2 (required): tiered playbook, post-incident review auto-flow, cross-domain coordination must be operational and producing clean incident-pattern data before contributions to external bodies are substantive.
• PC-Software L3 (required): continuous compliance attestation substrate supports automated evidence capture for pre-authorized containment actions; legal authority for automated actions flows from the policy and compliance program.
• ML-Software L3 (required): detection-as-code and high-confidence anomaly detection signals provide the automation trigger quality needed for pre-authorized runbook execution.

Desired Outcomes

• Industry-standard incident classification and response playbooks for AI/HAI software incidents are contributed and maintained, sector ISACs, AVID, and standards bodies cite the org's artifacts.
• Pre-authorized automated containment actions execute for a defined set of low-severity, high-confidence incident types, reducing MTTR for these classes to seconds from hours.
• MTTR benchmarks are established from ISAC and peer data; the program's performance against benchmarks is reported to the sponsor quarterly and drives investment decisions.
• Contributions to MITRE ATLAS TA0014 Impact tactic documentation reflect the org's first-party incident experience with AI/HAI software containment plays.

Activities

A) Industry-coordinated incident sharing and contribution

• Participate in sector ISAC AI incident-sharing programs (FS-ISAC AI working group, H-ISAC, IT-ISAC, sector-specific):
• Consume ISAC AI incident feeds; integrate relevant advisories into the IM-Software external-advisory source.
• Contribute anonymized incident classification (incident type, ATLAS tactic tag, HAI-TTP tag, containment play used, MTTR achieved) on a per-incident-class basis; target ≥4 ISAC contributions per year.
• Contribute to AI incident taxonomy standards:
• CSA AI Safety Initiative, AI incident severity-anchor definitions, playbook template schemas.
• AVID (AI Vulnerability Database), submit AI/HAI software vulnerability entries for novel incident classes discovered in production; target ≥2 AVID entries per year.
• OpenSSF AI, contribute runbook schema for pre-authorized containment actions.
• Contribute to MITRE ATLAS TA0014 Impact documentation, submit incident-derived technique observations or mitigation entries for Impact-tactic techniques (AML.T0048 through AML.T0053 range and successors); target ≥1 ATLAS contribution per year for IM-primary tactics.

B) Pre-authorized automated runbook decisioning

Define and publish a pre-authorization policy for automated containment actions, the set of actions that can execute without human approval when a detection fires at a defined confidence threshold:

• Pre-authorized actions (examples; published list vetted by Privacy/Legal and executive sponsor):
• Kill-switch execution for a Low-tier or Medium-tier agent artifact when an AGH detection fires above 95% confidence threshold.
• Egress-block for shadow-AI emergence on a non-Critical-tier service (first-time detection of new LLM provider domain from an unregistered service).
• Retrieval-source disable for a RAG pipeline when an injection-defense detection fires with a specific flagged source document ID (scoped to documented retrieval sources that have triggered before).
• Tool-revoke for a pre-defined "auto-revocable" tool category on an agent when a tool-scope-violation detection fires.
• Pre-authorized actions for Critical-tier artifacts require human confirmation within 15 minutes; the action fires after that window if no human confirmation arrives (timer-based fallback), with executive notification at fire time.
• All pre-authorized actions produce: a full audit log entry in the IM backlog, a human-review ticket auto-created at the time of execution, and a notification to the artifact's deployer-duty owner.
• Pre-authorization policy reviewed quarterly by Privacy/Legal and the executive sponsor; any automated action that produces an unexpected outcome triggers a review of the pre-authorization threshold.

C) MTTR benchmarking

• Establish MTTR benchmarks from:
• ISAC AI incident data exchanges.
• BSIMM-style observational data on AI/HAI incident response at comparable organizations.
• MITRE ATLAS practitioner community data.
• Peer roundtables (CISO and AI-safety practitioner communities).
• Publish a quarterly MTTR benchmark brief to the program sponsor:
• MTTR per incident class vs. benchmark (prompt-injection, agent rogue-action, training-data-leakage, shadow-AI emergence, RAG poisoning, model-swap rollback).
• MTTR per tier (Critical, High, Medium) vs. benchmark.
• Delta trend (improving, stable, degrading) vs. benchmark.
• Investment driver: where MTTR is above benchmark, root-cause mapped to a specific practice gap (missing detection, unclear playbook, on-call latency) with a budget-linked improvement proposal.

Outcome Metrics (L3)

Process Metrics (leading)

• ISAC participation cadence, sector ISAC feeds consumed and contributions submitted at least quarterly; ISAC AI exercises attended annually.
• Contribution pipeline health, ≥2 taxonomy/playbook/ATLAS items in-flight (draft, in-review, submitted) at any time.
• Pre-authorization policy review cadence, quarterly; any automated action producing an unexpected outcome triggers an out-of-cycle review.
• Benchmark data source refresh, MTTR benchmark inputs updated at least semi-annually; stale benchmarks flagged.

Effectiveness Metrics (business value)

• MTTR for pre-authorized containment classes drops to seconds from hours, the most significant MTTR-reduction lever available without adding headcount.
• Mean-time-to-contain on Critical-tier incidents continuing to compress as ISAC-shared intelligence accelerates root-cause identification and playbook refinement.
• External recognition, citations or adoption of contributed AI incident taxonomy artifacts by ISACs, CSA, AVID, ATLAS, or sector standards bodies.
• Budget efficiency, MTTR benchmark brief demonstrates that pre-authorized automation and ISAC-sourced playbook refinements deliver MTTR improvement without proportional headcount growth.

Success Criteria

• ≥4 ISAC AI incident contributions per year; ≥2 AVID entries per year; ≥1 ATLAS TA0014 Impact contribution per year; all contributions anonymized, legally vetted, and maintained.
• ≥3 pre-authorized automated containment actions live, vetted by Privacy/Legal and the executive sponsor, producing 100% audit records + human-review tickets on execution.
• Quarterly MTTR benchmark brief published to sponsor; Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes; deltas above benchmark linked to investment proposals.
• Pre-authorization policy reviewed quarterly; no unauthorized automated action executed; all unexpected automation outcomes reviewed within 5 business days.

Key Success Indicators

Level 1: - Single AI/HAI software issue backlog operational with standardized metadata (source, affected artifact, severity rubric, owner, SLA, regulatory flag, evidence link) capturing ≥95% of AI/HAI software issues from all source practices. - Six AI-specific incident playbook entries published (prompt-injection, agent rogue-action, training-data-leakage, silent model-swap, RAG poisoning, shadow-AI emergence) with named roles, containment plays, evidence-capture instructions, and SLA targets, each exercised in at least one tabletop in the last 12 months. - Regulatory SLA tracker live covering GDPR Art. 33 (72h), EU AI Act Art. 73, HIPAA (60d), NYDFS Part 500 (72h), and sector-specific obligations; 100% adherence in the last 90 days. - Post-incident review loop wired to SA, SR, EG, and ML, every Critical/blocker incident produces a review within 14 days with named update outputs for each downstream practice. - Program-sponsor dashboard refreshed monthly showing backlog aging, SLA adherence, and post-incident learning outputs.

Level 2: - Critical-tier MTTA ≤1 hour; MTTC ≤4 hours; 24/7 on-call coverage with a documented rotation that includes a current Critical-tier artifact briefing. - Post-incident review auto-flow integration live; 100% of Critical-tier review outputs auto-routed to SA/SR/EG/ML backlogs; ≥90% of downstream practice owners responding within 14 days. - Cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents; named cross-domain contacts verified quarterly. - Tier-movement in SM-Software inventory auto-triggers IM configuration updates within 14 days (Critical re-tier) or 30 days (other tiers).

Level 3: - ≥4 ISAC contributions per year; ≥2 AVID entries per year; ≥1 ATLAS TA0014 contribution per year, all maintained and tracked for external adoption. - ≥3 pre-authorized automated containment actions live, vetted, producing 100% audit records, with quarterly policy review and zero unauthorized executions. - Quarterly MTTR benchmark brief published; Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes; deltas linked to investment proposals.

Common Pitfalls

Level 1: - ❌ "Single backlog" created but source practices continue filing into separate queues, ST failures stay in the CI dashboard, ML alerts route to a Slack channel, and TA residual risks live in a spreadsheet; the backlog achieves only ~40% of issues and the ≥95% coverage target is never achieved. - ❌ Triage rubric severity anchors are generic (probability × impact without AI-specific axes), an agent executing a rogue tool invocation with real-world damage is triaged Medium because CVSS-analog scoring does not capture kill-switch failure or account modification scope. - ❌ Playbook entries published but roles not pre-assigned, on the first live agent rogue-action, the team spends the first 30 minutes figuring out who invokes the kill-switch, not invoking it. - ❌ GDPR Art. 33 72-hour clock tracked informally, when a prompt-injection incident involving PII lands on a Friday evening, the clock starts but no named owner confirms the start time; the SLA slips before anyone documents the awareness event. - ❌ Post-incident reviews completed but outputs filed in a document that no downstream practice owner reads, SA, SR, EG, and ML do not update; the same incident class recurs with the same root cause six months later. - ❌ Vendor material-change incidents (inference provider model-family swap, subprocessor addition by the model vendor) are not recognized as IM-Software playbook triggers, the Software-domain IR re-review never happens and the output-integrity regression reaches customers.

Level 2: - ❌ Critical-tier activation criteria are vague, incidents that qualify for full-team + executive activation stay in the standard queue until the deployer-duty owner escalates; the SLA that required ≤1-hour acknowledgement is already missed by the time the right people engage. - ❌ Post-incident review auto-flow integration wired but downstream practice backlogs never treat the auto-created tickets as actionable, the SR team closes the ticket as "acknowledged" without updating the requirements pack; the feedback loop is nominally present but produces no change. - ❌ Cross-domain coordination protocol exists on paper but no IC is pre-designated, the first cross-domain incident where a Software artifact's vendor inference provider is implicated produces ownership confusion; the Vendors-domain IM and Software-domain IM both wait for the other to take the IC role. - ❌ 24/7 on-call coverage implemented but the on-call briefing is stale, the rotation shifts include a Critical-tier artifact list that was accurate 90 days ago; new Critical-tier artifacts are not in the briefing; on-call responders do not know the kill-switch path for recently tiered artifacts.

Level 3: - ❌ ISAC participation limited to consuming feeds, contributions are absent; the org is labeled a free-rider; influence over AI incident taxonomy standards diminishes and the ISAC feed quality degrades without reciprocal intelligence. - ❌ Pre-authorized automated containment fires on a Critical-tier artifact because the confidence threshold was set too loosely, a false positive executes a kill-switch on a production agent handling customer sessions; the pre-authorization policy had no Critical-tier exception check. - ❌ MTTR benchmark brief cites benchmarks from organizations with fundamentally different AI/HAI portfolio scale or risk profiles, "we are at benchmark" is true but the benchmark set was chosen to flatter rather than stretch. - ❌ AVID entries submitted once and never updated, novel incident classes evolve; the org's AVID entry reflects a vulnerability from 18 months ago that has since been mitigated; the community builds on stale data. - ❌ Automated containment produces audit records that are technically complete but lack the narrative context needed for a post-incident root-cause review, humans reviewing automated-action logs cannot reconstruct what the detection saw and why the threshold triggered.

Practice Maturity Questions

Level 1: 1. Is there a single AI/HAI software issue backlog with standardized metadata (source, affected artifact linked to SM inventory, severity rubric anchored to AI-specific axes, active exfiltration / agent damage / kill-switch failure / regulated-data breach for Critical; confirmed control failure with potential impact for High, etc., owner, SLA, regulatory flag, evidence link) capturing ≥95% of issues from all source practices (TA, SR, DR, IR, ST, ML, external)? 2. Is the AI/HAI software incident playbook published with ≥6 named AI-specific incident classes (prompt-injection, agent rogue-action, training-data-leakage, silent model-swap, RAG poisoning, shadow-AI emergence), each with pre-assigned roles, containment plays (kill-switch, model rollback, tool-revoke, retrieval-source disable, egress-block), evidence-capture steps, and SLA targets, and has each class been exercised in at least one tabletop in the last 12 months? 3. Is the regulatory SLA tracker live covering GDPR Art. 33 (72h), EU AI Act Art. 73, HIPAA (60d), NYDFS Part 500 (72h), and sector-specific obligations, with 100% adherence in the last 90 days, and does every Critical/blocker incident produce a post-incident review within 14 days with named update outputs flowing to SA, SR, EG, and ML?

Level 2: 1. Is a tier-calibrated incident playbook operational with Critical-tier MTTA ≤1 hour and MTTC ≤4 hours, 24/7 on-call coverage with a documented rotation including a current Critical-tier artifact briefing, and tier-movement in the SM-Software inventory automatically triggering IM configuration updates (on-call path, playbook variant, SLA targets) within 14 days (Critical re-tier)? 2. Is a post-incident review auto-flow integration live routing Critical-tier review outputs to SA/SR/EG/ML practice backlogs, with ≥90% of downstream practice owners responding within 14 days and the sponsor reviewing output quality quarterly to distinguish substantive changes from nominal acknowledgements? 3. Is a cross-domain coordination protocol published and used for 100% of multi-domain AI/HAI software incidents, with named cross-domain contacts for Vendors, Infrastructure, and Processes domains verified quarterly, a single IC from the primary impacted domain, and joint post-incident reviews spanning all affected domains?

Level 3: 1. Does the program contribute ≥4 anonymized AI incident-classification entries per year to sector ISACs, ≥2 entries per year to AVID, and ≥1 contribution per year to MITRE ATLAS TA0014 Impact tactic documentation, with all contributions maintained current, legally vetted, and tracked for external adoption? 2. Are ≥3 pre-authorized automated containment actions live (kill-switch, egress-block, retrieval-source disable, or tool-revoke classes), vetted by Privacy/Legal and the executive sponsor, producing 100% audit records plus human-review tickets on execution, with the pre-authorization policy reviewed quarterly and any unexpected outcome triggering an out-of-cycle review? 3. Is a quarterly MTTR benchmark brief published to the sponsor comparing the program's MTTR per incident class and per tier against ISAC-sourced and peer-sourced benchmarks, with Critical-tier MTTR at or below benchmark for ≥3 of 6 incident classes and deltas above benchmark linked to specific practice gaps and investment proposals?

Document Version: HAIAMM v3.0 Practice: Issue Management (IM) Domain: Software Last Updated: 2026-05-13 Author: Verifhai