Issue Management (IM) - Software Domain

HAIAMM Assessment Questionnaire v3.5

Generated from cells/IM-Software-L*.cell.json. Do not edit by hand. Canonical model: HAIAMM-v3.5-Framing.md · Handbook: HAIAMM-v3.5-Handbook.md.

---

Practice: Issue Management (IM) Domain: Software Business Function: Operations Purpose: Run one backlog for software AI issues with an AI-aware severity taxonomy and published service levels, so AI findings age and escalate like everything else. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

---

Instructions

• Answer from current, implemented practice, not plans or aspirations.
• Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked).
• Levels are derived, never self-assigned. Record evidence and metric values; the level falls out of the scoring below.
• Silence is not a pass. A metric you cannot produce is left blank and counts as not met. Do not estimate.
• Answer progressively. Complete all Level 1 questions before Level 2.

Evidence grade

Record how each answer is known, not only what is claimed (v3.5 framing §6). A level claim is graded at its weakest field.

Grade	Meaning
E0	Asserted by a practitioner, unevidenced
E1	Asserted, with a link to a named artifact
E2	Emitted by a system as telemetry
E3	Machine-attested and independently checkable

---

Scoring Methodology

Tier	Score	Criteria
Fully Mature	1.0	Evidence complete + ≥3 outcome metrics meet targets
Implemented	0.67	Evidence complete + 2 outcome metrics meet targets
Partial	0.33	Evidence partially complete + <2 outcome metrics meet targets
Not Implemented	0.0	No substantive evidence of practice

Practice maturity level achieved = the highest level where all 3 questions score ≥ 0.67.

---

Maturity Level 1, Foundational

Objective: Run one backlog for software AI issues with an AI-aware severity taxonomy and published service levels, so AI findings age and escalate like everything else.

Prerequisite cells: SM-Software-L1, PC-Software-L1, TA-Software-L1

Question L1.1: Extend the severity taxonomy to AI failure modes

Q1.1: Does the severity taxonomy extend the existing scheme with AI-specific classes including prompt injection reaching an action and denial-of-wallet, with published criteria keyed to data exposed, reversibility, population affected and whether the asset acts or advises, plus worked examples per class?

Evidence Required:

• [ ] Extend the existing severity taxonomy rather than writing a new one, adding the AI-specific classes that a classical scheme cannot rate: prompt injection reaching an action, an agent acting outside its declared tool scope, training-data leakage through a shipped model, output-integrity regression after a model change, and denial-of-wallet through unbounded generation or retry
• [ ] For each class, state what makes it Critical, High, Medium or Low, using the dimensions that actually vary, being the data exposed, the reversibility of the effect, the population affected and whether the asset acts or advises
• [ ] Include denial-of-wallet as a named class with its own criteria, because it degrades no availability signal and a scheme keyed to availability rates it as nothing while the budget drains
• [ ] Publish worked examples per class, since a taxonomy without examples is applied inconsistently and its inconsistency is invisible in the aggregate

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L1.2: Run one backlog with wired-in sources and service levels

Q1.2: Are all AI issues managed in the shared security backlog with at least 95% carrying severity, owner and service level at creation, and do at least 80% arrive from a wired source rather than manual entry? For Software, the assessor should be shown an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Route every AI finding into the shared security backlog rather than a separate AI register
• [ ] Wire the sources so findings arrive automatically: implementation review, security testing, monitoring detections, adversarial exercises, vendor advisories and self-reports
• [ ] Assign a severity, an owner and a service level at creation rather than at triage, because an issue that waits for triage to acquire a service level is an issue whose clock has not started
• [ ] Publish service levels by severity and report adherence rather than backlog size, since backlog size is a function of discovery rate as much as of remediation and improves when discovery stops

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L1.3: Document containment for AI-specific incidents

Q1.3: Are containment actions documented for every AI incident class, covering revoking the agent's tool credentials and the others, with named invocation authority and technical availability verified rather than assumed?

Evidence Required:

• [ ] Write containment actions for each AI-specific incident class before they are needed, covering revoking the agent's tool credentials, disabling the affected feature flag, reverting to the previously validated model version, and invoking the kill switch
• [ ] AI containment differs from classical containment in a way that catches teams out: the fastest effective action is usually to remove the system's ability to act rather than to take the system offline, because an AI asset that is merely unavailable stops delivering value while an asset whose tool access is revoked stops causing harm and may keep delivering
• [ ] Name who is authorised to invoke each action at what hour, and verify the technical means exists and works
• [ ] An action documented but never exercised is discovered to be unavailable at the worst moment

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 1 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
AI issues managed in the shared security backlog rather than a separate register	___	___	100%	☐
Issues carrying a severity, owner and service level at creation	___	___	≥95% at creation rather than at triage	☐
Issue sources wired to create records automatically	___	___	≥80% of issues arrive from a wired source rather than manual entry	☐
AI-specific severity classes with published criteria and worked examples	___	___	100% of named classes	☐
Containment actions documented and technically verified as available	___	___	100% of documented actions verified at least once	☐

Where each number is read from:

• AI issues managed in the shared security backlog rather than a separate register — Issue tracker records tagged by domain
• Issues carrying a severity, owner and service level at creation — Issue tracker creation records
• Issue sources wired to create records automatically — Issue creation source field
• AI-specific severity classes with published criteria and worked examples — Severity taxonomy documentation
• Containment actions documented and technically verified as available — Containment verification records

Level 1 Consumption Metrics

Absolute spend is a ledger input, never a metric. Every row is divided by a unit of business work. If AI spend is not allocated to an owning team, record this level as blocked on allocation rather than scoring it down (v3.5 framing §12).

Metric	Denominator	Current	Economic failure it detects	Met?
Denial-of-wallet incidents detected, with time from onset to containment	per denial-of-wallet incident	___	budget drain running long because no availability signal degrades, which makes detection time the entire loss and is invisible to conventional incident triggers	☐
Consumption anomaly cost accrued before containment	per completed task	___	containment measured in elapsed time rather than in accrued cost, which understates the urgency of exactly the incident class where cost is the harm	☐
AI incidents whose containment action was a consumption limit rather than a service withdrawal	per contained AI incident	___	defaulting to taking the system offline when revoking its ability to act or spend would have contained the harm while preserving the service	☐

Level 1 Oversight Metrics

"A human approves it" is not a control unless you measure whether they refuse. A metric that cannot fail is not a metric.

Metric	Current	Oversight failure it detects	Met?
Issues closed without a recorded remediation or acceptance	___	closure as backlog hygiene rather than resolution, where an issue is marked done because it is old rather than because anything changed	☐
Severity downgrades applied after creation, with the downgrading authority recorded	___	severity drift under service-level pressure, where reclassification is used to make a clock more comfortable rather than to correct an assessment	☐

Level 1 30-second check (E0)

• ☐ Yes ☐ No — Do AI security issues go into the same backlog as everything else?
• ☐ Yes ☐ No — Does your severity scale have a rating for an AI that drains your budget?
• ☐ Yes ☐ No — Do you know how to stop an AI agent acting without taking it offline?

---

Maturity Level 2, Comprehensive

Objective: Calibrate software AI issue service levels to asset tier, run rehearsed AI incident response, and close the loop from root cause to the practice that owns it.

Prerequisite cells: IM-Software-L1, SM-Software-L2, ML-Software-L1

Question L2.1: Calibrate service levels and authority to tier

Q2.1: Do service levels vary by asset tier and severity with adherence reported per tier rather than in an aggregate that can be met while every Critical-tier breach is missed? Evidence this in the Software domain against an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Publish remediation service levels by severity and asset tier, tighter at Critical, and report adherence per tier rather than in aggregate, because an aggregate figure can be met while every Critical-tier breach is missed
• [ ] Make deferral and risk-acceptance authority rise with tier and severity, so a deferred Critical finding requires executive acceptance with an expiry and quarterly review
• [ ] Report deferrals to the sponsor alongside closures, since a practice reporting only what it closed describes a backlog that does not exist
• [ ] Track findings deferred and later implicated in an incident as a distinct measure of whether deferral judgment is sound rather than merely authorised
• [ ] Tier-proportionate response in this domain is shaped by reversibility: an agent that took an irreversible action needs containment in minutes, while an advisory feature producing poor output does not

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L2.2: Rehearse AI incident response and instrument the notification clocks

Q2.2: Have AI incident playbooks been exercised within 12 months including out of hours, measured by the gaps found, and are regulatory notification clocks instrumented in advance with owners and evidence requirements identified? For Software, the assessor should be shown an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Write and exercise incident playbooks for each AI-specific class, covering detection signals, containment using revoking the agent's tool credentials, disabling the affected feature flag, reverting to the previously validated model version, and invoking the kill switch, evidence preservation, communication and recovery
• [ ] Exercise at least semi-annually with the people who would actually respond, including out of hours, and measure the exercise by the gaps it finds rather than by its completion
• [ ] Instrument the regulatory notification clocks explicitly: identify which incident classes trigger EU AI Act serious-incident reporting or GDPR breach notification, what starts the clock, who decides, and what evidence the notification requires
• [ ] A team that discovers a 72-hour obligation 40 hours into an incident has lost most of the window to a question it could have answered in advance
• [ ] Playbook exercises in this domain rehearse revoking tool credentials and reverting a model version under time pressure, both of which teams believe are fast until they try them out of hours

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L2.3: Route root causes to the owning practice

Q2.3: Do at least 90% of incidents reaching root cause open a work item in the owning practice rather than closing where they were found, and is the share whose root cause was already known and untreated tracked openly? In Software, ask to see an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Require every incident reaching a root cause to name the practice that owns the fix and to open a work item there, rather than closing the incident where it was found
• [ ] An incident caused by a missing architectural control belongs to secure architecture, one caused by an untested threat belongs to security testing, one caused by a knowledge gap belongs to education
• [ ] Closing it in the incident record alone guarantees recurrence, and recurrence is the single clearest signal that a response practice is functioning as a repair shop rather than as a feedback mechanism
• [ ] Track the share of incidents whose root cause was already known and untreated, which separates a discovery problem from an action problem and is usually uncomfortable reading
• [ ] Root causes in this domain most often belong to secure architecture and implementation review rather than to the responding team

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 2 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
Issues remediated within the tier and severity service level	___	___	≥90% per tier, reported per tier rather than in aggregate	☐
AI incident playbooks exercised in the last 12 months	___	___	100% of AI-specific classes, including an out-of-hours exercise	☐
Incident classes with an instrumented regulatory notification clock	___	___	100% of classes that can trigger a notification obligation	☐
Incidents reaching root cause with a work item opened in the owning practice	___	___	≥90%	☐
Deferred Critical findings carrying executive acceptance with expiry	___	___	100%, with none past expiry	☐
Incidents whose root cause was already known and untreated	___	___	tracked and falling, reported openly	☐

Where each number is read from:

• Issues remediated within the tier and severity service level — Issue tracker with tier and service-level fields
• AI incident playbooks exercised in the last 12 months — Exercise register with dates and findings
• Incident classes with an instrumented regulatory notification clock — Notification decision tree with owners
• Incidents reaching root cause with a work item opened in the owning practice — Incident records joined to practice backlogs
• Deferred Critical findings carrying executive acceptance with expiry — Finding disposition records
• Incidents whose root cause was already known and untreated — Root-cause analysis joined to the threat library and prior findings

Level 2 Consumption Metrics

Absolute spend is a ledger input, never a metric. Every row is divided by a unit of business work. If AI spend is not allocated to an owning team, record this level as blocked on allocation rather than scoring it down (v3.5 framing §12).

Metric	Denominator	Current	Economic failure it detects	Met?
Denial-of-wallet incidents contained within their tier service level	per denial-of-wallet incident	___	a response clock calibrated to availability incidents applied to an incident class where cost accrues continuously and no availability signal ever degrades	☐
Cost accrued between detection and containment, by tier	per completed task	___	containment urgency judged by elapsed time rather than by accruing cost, which systematically under-prioritises the incident class where the two diverge most	☐
Emergency consumption limits applied during incidents and subsequently reverted	per applied emergency limit	___	emergency limits left in place indefinitely and degrading legitimate service, or reverted immediately and leaving the exposure open, neither of which is visible without tracking	☐

Level 2 Oversight Metrics

"A human approves it" is not a control unless you measure whether they refuse. A metric that cannot fail is not a metric.

Metric	Current	Oversight failure it detects	Met?
Incidents closed without a root cause recorded, by severity	___	closure without diagnosis, which guarantees recurrence and is most common under load at exactly the times recurrence is least affordable	☐
Risk acceptances at Critical tier, with the accepting authority and expiry recorded	___	acceptance authority drifting downward under pressure, so the decision to live with a Critical risk is taken by whoever is available	☐
Exercise findings closed without a change to the playbook or the environment	___	rehearsal treated as a compliance activity rather than as a source of correction, which is visible only by tracking what the exercise changed	☐

Level 2 30-second check (E0)

• ☐ Yes ☐ No — Have you rehearsed an AI incident with the people who would respond?
• ☐ Yes ☐ No — Do you know which AI incidents start a regulatory notification clock?
• ☐ Yes ☐ No — When an AI incident is fixed, does another practice get a work item?

---

Maturity Level 3, Industry-Leading

Objective: Automate software AI issue triage and containment where it is safe, measure response by outcome rather than activity, and contribute incident learning back to the field.

Prerequisite cells: IM-Software-L2, ML-Software-L2, EH-Software-L3

Question L3.1: Automate triage and bounded first-line containment

Q3.1: Are at least 85% of issues auto-triaged with severity, owner and clock assigned, and are automatic containment classes enumerated explicitly with everything else alerting and every action logged with evidence and a reversal path? Evidence this in the Software domain against an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Automate severity assignment, ownership routing and duplicate detection from the detection signal, so an issue arrives with its clock started and its owner assigned
• [ ] Enumerate explicitly the containment actions that may fire automatically, being those where the correct response is unambiguous and reversible, such as applying an emergency consumption limit, revoking a specific tool credential or blocking a destination
• [ ] Log every automated action with what was done, on what evidence, and how to reverse it
• [ ] Publish the human-judgment classes explicitly, because automated containment is safest when its boundary is stated rather than assumed, and the assumption is always that it covers more than it does
• [ ] Automatic containment in this domain is well bounded: revoking a tool credential, disabling a feature flag and reverting to a previously validated model version are all unambiguous and reversible

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L3.2: Measure response by outcome rather than activity

Q3.2: Is recurrence rate by root-cause class the practice's headline outcome measure reported to the sponsor, rather than throughput and backlog size which are dominated by discovery rate and improve when discovery degrades? For Software, the assessor should be shown an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Shift the primary measures from throughput to outcome: recurrence rate by root-cause class, harm avoided estimated from the consumption ledger and the affected population, and time from detection to containment rather than to closure
• [ ] Ticket throughput and backlog size remain as health indicators but are explicitly not the practice's outcome measures, because both are dominated by discovery rate and both improve when discovery degrades
• [ ] Report recurrence to the sponsor as the headline, since a response practice that closes quickly and recurs constantly is failing at the thing it exists for while every throughput metric looks excellent
• [ ] Recurrence in this domain clusters around tool scope and prompt-template changes, which points remediation at secure architecture and implementation review rather than at the responding team

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L3.3: Contribute anonymised incident learning back

Q3.3: Does the organisation contribute at least four accepted anonymised incident-learning artifacts per year with disclosure coordinated, prioritising the incidents that were most surprising internally? In Software, ask to see an incident record showing tool-credential revocation as the containment action.

Evidence Required:

• [ ] Contribute at least four substantive artifacts per year: anonymised incident case studies, detection and containment patterns, playbooks others can adopt, and technique observations submitted to MITRE ATLAS where they fit
• [ ] The public corpus of real AI security incidents is thin, disproportionately so relative to the deployment base, which means most organisations learn only from their own incidents, which is the slowest and most expensive way to learn anything
• [ ] Contributing changes that for everyone including the contributor
• [ ] Anonymise carefully, review legally, coordinate disclosure with any affected vendor or provider, and count only accepted artifacts
• [ ] Prefer contributing the incidents that were most surprising internally, since those carry the most information for others
• [ ] Software incident write-ups contribute well because agent and RAG failure modes are common across organisations and the public corpus is thin

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 3 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
Issues arriving with severity, owner and clock assigned automatically	___	___	≥85% auto-triaged	☐
Enumerated containment classes executed automatically within the detection window	___	___	≥70% of eligible events, with everything else alerting	☐
Automated containment actions logged with evidence and reversal path	___	___	100%	☐
Recurrence rate by root-cause class	___	___	tracked and falling year over year, reported to the sponsor as the headline measure	☐
Accepted external incident-learning contributions per year	___	___	≥4 accepted, anonymised, legally reviewed and disclosure-coordinated	☐

Where each number is read from:

• Issues arriving with severity, owner and clock assigned automatically — Issue creation telemetry
• Enumerated containment classes executed automatically within the detection window — Automated containment log
• Automated containment actions logged with evidence and reversal path — Containment audit log
• Recurrence rate by root-cause class — Root-cause analysis aggregated by class over time
• Accepted external incident-learning contributions per year — Contribution log with acceptance references

Level 3 Consumption Metrics

Absolute spend is a ledger input, never a metric. Every row is divided by a unit of business work. If AI spend is not allocated to an owning team, record this level as blocked on allocation rather than scoring it down (v3.5 framing §12).

Metric	Denominator	Current	Economic failure it detects	Met?
Automated emergency consumption limits applied within the detection window	per denial-of-wallet detection	___	cost accruing through a human triage delay on an incident class where the loss is continuous and the correct response is unambiguous	☐
Cost accrued between detection and automated containment	per completed task	___	an automation path that fires but not fast enough to matter, which looks successful in coverage terms and fails in loss terms	☐
Harm avoided from contained consumption incidents, computed from burn rate and time to contain	per contained consumption incident	___	a response practice unable to state what its speed is worth, which loses the argument for the investment that made the speed possible	☐

Level 3 Oversight Metrics

"A human approves it" is not a control unless you measure whether they refuse. A metric that cannot fail is not a metric.

Metric	Current	Oversight failure it detects	Met?
Automated containment actions reversed by a human within one hour	___	automation acting on a misread signal, which indicates the enumerated class was wrongly scoped and is only visible if reversals are tracked separately from actions	☐
Incidents in human-judgment classes acknowledged but not actioned within the tier window	___	alert fatigue concentrating precisely on the classes automation deliberately left to people because they need judgment	☐
Recurrence accepted without a corresponding work item in the owning practice	___	recurrence normalised as a cost of doing business, which is how a response practice becomes a repair shop	☐

Level 3 30-second check (E0)

• ☐ Yes ☐ No — Does an AI security alert get an owner without a human assigning one?
• ☐ Yes ☐ No — Do you measure whether the same AI issue keeps coming back?
• ☐ Yes ☐ No — Have you published an anonymised AI incident others can learn from?

---

Summary Scorecard

Level	Q1	Q2	Q3	Level score	Weakest evidence grade	Achieved?
L1	___	___	___	___	___	☐
L2	___	___	___	___	___	☐
L3	___	___	___	___	___	☐

Issue Management (IM) maturity in the Software domain: ____________

---

Document Version: HAIAMM v3.5 (generated)