Strategy & Metrics (SM) - Data Domain

HAIAMM Assessment Questionnaire v3.5

Generated from cells/SM-Data-L*.cell.json. Do not edit by hand. Canonical model: HAIAMM-v3.5-Framing.md · Handbook: HAIAMM-v3.5-Handbook.md.

---

Practice: Strategy & Metrics (SM) Domain: Data Business Function: Governance Purpose: Stand up AI data governance, inventory every corpus AI touches, and baseline the metrics that prove regulated data is not leaving governed stores for AI services. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

---

Instructions

• Answer from current, implemented practice, not plans or aspirations.
• Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked).
• Levels are derived, never self-assigned. Record evidence and metric values; the level falls out of the scoring below.
• Silence is not a pass. A metric you cannot produce is left blank and counts as not met. Do not estimate.
• Answer progressively. Complete all Level 1 questions before Level 2.

Evidence grade

Record how each answer is known, not only what is claimed (v3.5 framing §6). A level claim is graded at its weakest field.

Grade	Meaning
E0	Asserted by a practitioner, unevidenced
E1	Asserted, with a link to a named artifact
E2	Emitted by a system as telemetry
E3	Machine-attested and independently checkable

---

Scoring Methodology

Tier	Score	Criteria
Fully Mature	1.0	Evidence complete + ≥3 outcome metrics meet targets
Implemented	0.67	Evidence complete + 2 outcome metrics meet targets
Partial	0.33	Evidence partially complete + <2 outcome metrics meet targets
Not Implemented	0.0	No substantive evidence of practice

Practice maturity level achieved = the highest level where all 3 questions score ≥ 0.67.

---

Maturity Level 1, Foundational

Objective: Stand up AI data governance, inventory every corpus AI touches, and baseline the metrics that prove regulated data is not leaving governed stores for AI services.

Question L1.1: Charter AI data governance and name the stewards

Q1.1: Is there a published AI data governance charter with a named steward per corpus, three priority policies covering AI data use, acceptable use and the intake gate, and explicit decision rights over which data classes may reach which model deployments?

Evidence Required:

• [ ] Publish an AI data governance charter within the first 30 days, sponsored jointly by the CISO and the Chief Data Officer or equivalent, and co-signed by Privacy counsel
• [ ] The charter names the seven in-scope archetypes, being training corpus, fine-tuning dataset, inference input stream, retrieval store, embedding store, prompt and completion log corpus, and evaluation or test set
• [ ] It assigns a named data steward per corpus rather than per system, because corpora outlive the systems that created them
• [ ] It establishes three priority policies, being AI Data Use, Data Acceptable Use for AI, and the data intake and sanction gate
• [ ] And it defines decision rights over which data classes may reach which model deployments, who approves an exception and who owns the revocation path when a flow must stop
• [ ] The working group includes Data Governance, Privacy and Legal counsel, AI/ML engineering, Compliance and the platform team that operates the stores

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L1.2: Inventory the corpora and instrument AI-egress detection

Q1.2: Does one AI data inventory cover all seven archetypes including prompt and completion log corpora, recording classification, lineage, retention, legal basis, cross-border flow and subject-access exposure, at 85% coverage or better against discovered corpora?

Evidence Required:

• [ ] Build one AI data inventory recording, per corpus, the archetype, steward, classification against the enterprise scheme, volume and growth rate, lineage from originating source, the model deployments that consume it, whether it is used at training or inference or both, retention period and legal basis, cross-border flow and its transfer mechanism, and subject-access exposure
• [ ] Seed it from the data catalogue, pipeline telemetry, training-run registries, cloud storage inventories and access-control groups
• [ ] Then instrument the domain's shadow-AI mechanism: data-loss prevention rules on egress to model-provider endpoints, tuned to the classified data patterns that matter rather than to everything, and canary records seeded into governed stores so that a canary appearing in a model output or an external log is unambiguous evidence of an ungoverned flow
• [ ] Treat prompt and completion logs as a first-class corpus in the inventory
• [ ] They accumulate regulated data by default, they are frequently replicated to observability vendors, and they are the corpus most often missed at the first inventory pass

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L1.3: Baseline the data scoreboard and open the retrieval ledger

Q1.3: Are DLP on AI-endpoint egress and canary tagging both operating as the domain's shadow-AI mechanism, with regulated-data flows lacking a sanctioned path standing at zero and every canary trigger investigated within five business days?

Evidence Required:

• [ ] Instrument and report quarterly: inventory coverage against discovered corpora, the share of corpora carrying a classification and a named steward, regulated-data flows to AI services that lack a sanctioned path, canary-trigger events, and the share of corpora with a defined retention period
• [ ] Open the consumption side of the ledger for this domain, recording retrieval cost per query, embedding re-index cost per indexed document, and training-run cost per run, each attributed to the corpus and the owning team
• [ ] Retention cost belongs in the ledger explicitly, because it is the mechanism by which log corpora get silently truncated to save money, taking with them the evidence that monitoring and deployer-duty obligations depend on
• [ ] Record the denominator per archetype rather than reporting absolute storage or query spend, and where corpus-level cost allocation does not exist, record the ledger as blocked on allocation and raise the tagging requirement with FinOps

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 1 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
AI data inventory coverage against discovered corpora	___	___	≥85% within 12 months	☐
Corpora carrying a classification and a named steward	___	___	100% of regulated and confidential corpora	☐
Regulated-data flows to AI services without a sanctioned path	___	___	0 for regulated classes, and falling for confidential	☐
Canary-trigger events indicating an ungoverned corpus flow	___	___	0 sustained, with every trigger investigated within 5 business days	☐
Corpora with a defined retention period and legal basis	___	___	≥90%, and 100% for prompt and completion log corpora	☐

Where each number is read from:

• AI data inventory coverage against discovered corpora — Data catalogue reconciled against pipeline and storage telemetry
• Corpora carrying a classification and a named steward — AI data inventory steward and classification fields
• Regulated-data flows to AI services without a sanctioned path — DLP rules on AI-endpoint egress
• Canary-trigger events indicating an ungoverned corpus flow — Canary detection log
• Corpora with a defined retention period and legal basis — AI data inventory retention field

Level 1 Consumption Metrics

Absolute spend is a ledger input, never a metric. Every row is divided by a unit of business work. If AI spend is not allocated to an owning team, record this level as blocked on allocation rather than scoring it down (v3.5 framing §12).

Metric	Denominator	Current	Economic failure it detects	Met?
Retrieval cost per query, by retrieval store	per retrieval query	___	retrieval amplification, where a single request fans out across a store, and systematic extraction reconnaissance, which is expensive and shows in the cost profile before it shows in the access logs	☐
Embedding re-index cost per indexed document	per indexed document	___	unbounded re-indexing loops and over-collection, where corpora grow faster than their governance, driving cost that nobody attributes to a steward	☐
Log-retention cost per retained prompt-completion record	per retained record	___	retention silently shortened for cost reasons, removing the corpus that monitoring detection and EU AI Act deployer-duty evidence both depend on	☐

Level 1 30-second check (E0)

• ☐ Yes ☐ No — Do you know which datasets your AI systems were trained on?
• ☐ Yes ☐ No — Are your prompt and completion logs treated as a governed data corpus?
• ☐ Yes ☐ No — Would you detect regulated data being sent to an AI service today?

---

Maturity Level 2, Comprehensive

Objective: Tier the corpora by sensitivity and exposure, calibrate governance intensity per tier, and bound the consumption of high-tier retrieval and training workloads.

Prerequisite cells: SM-Data-L1, PC-Data-L1, TA-Data-L1

Question L2.1: Publish the corpus risk-tier rubric

Q2.1: Is every corpus assigned a tier from a published rubric covering classification, training-versus-inference use, decision-affecting use, cross-border flow, subject-access exposure and lineage quality, re-derived automatically when classification or exposure changes?

Evidence Required:

• [ ] Define four tiers derived deterministically from six auditable dimensions
• [ ] Data classification is the floor, where regulated personal, health, payment or customer-confidential content places a corpus at Critical or High
• [ ] Training-versus-inference use elevates, because a corpus that shapes model weights carries a permanence that an inference input stream does not, and cannot be retracted by deleting a record
• [ ] Decision-affecting use elevates to Critical where the corpus feeds a system making decisions with legal or significant effect under GDPR Art. 22 or falling within EU AI Act Annex III. Cross-border flow elevates where the transfer basis is anything other than adequacy
• [ ] Subject-access exposure elevates where records are attributable to identifiable individuals and therefore carry erasure and portability obligations that must be executable against a model, not only against a store
• [ ] Lineage quality lowers confidence and therefore elevates tier where provenance is incomplete, because a corpus of unknown origin cannot be defended
• [ ] Publish the rubric with its derivation rule, permit overrides only with recorded rationale, and re-derive tiers automatically when a classification or exposure field changes

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L2.2: Calibrate governance intensity and budgets per tier

Q2.2: Is there a published tier-treatment matrix covering access control, lineage depth, retention and deletion propagation, encryption and residency, monitoring detections and testing obligations, with at least 95% of Critical-tier corpora having received full-scope treatment in the last 12 months?

Evidence Required:

• [ ] Publish a tier-treatment matrix covering, per tier: access-control model from broad read to per-request authorization with purpose binding
• [ ] Lineage depth from source attribution to full transformation provenance
• [ ] Retention discipline and deletion-propagation obligations including whether erasure must propagate to embeddings and fine-tuned weights
• [ ] Encryption and residency requirements
• [ ] Monitoring detections including extraction-pattern detection on retrieval stores
• [ ] Testing obligations including poisoning-resistance and extraction-resistance batteries
• [ ] And re-review triggers on material change
• [ ] Add the consumption row: Critical- and High-tier retrieval stores and training workloads carry declared budgets with a defined behaviour at the cap, because systematic extraction against a retrieval store and an unbounded re-index loop both present first as cost
• [ ] Treat a retention reduction, a sampling change or a cache extension on a Critical-tier corpus as a material change triggering re-review, since each alters the evidence base that monitoring and regulatory obligations rest on

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L2.3: Report per tier and govern corpus tier movement

Q2.3: Do erasure requests propagate to embeddings and derived artifacts within the statutory window for all Critical- and High-tier corpora, and do Critical-tier retrieval and training workloads operate inside declared budgets with a tested behaviour at the cap?

Evidence Required:

• [ ] Report the scoreboard per tier and per archetype quarterly to the sponsor
• [ ] Report inventory state, classification and steward coverage, unsanctioned-flow counts, canary triggers, retention conformance and per-tier service-level adherence, with every Critical-tier corpus as its own row
• [ ] Report the Assured Spend Ratio for retrieval and training consumption per tier, targeting full attribution at Critical
• [ ] Maintain a tier-movement log capturing every corpus that changed tier, with the triggering dimension recorded, and review it quarterly
• [ ] Corpus tier movement is more common than software tier movement because content drifts continuously, so a store that starts with public documentation and gradually accumulates customer tickets must surface as an upgrade rather than persist under its founding classification
• [ ] Reconcile the steward roster against HR records each quarter so that departures produce a reassignment rather than an orphan

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 2 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
Corpora carrying a current tier assignment	___	___	100% of inventoried corpora, with tier age under 6 months	☐
Critical-tier corpora receiving full-scope treatment in the last 12 months	___	___	≥95%	☐
Erasure requests propagated to embeddings and derived artifacts within the statutory window	___	___	100% within 30 days for Critical and High tier	☐
High- and Critical-tier retrieval stores with extraction-pattern detection active	___	___	≥95%	☐
Assured Spend Ratio for retrieval and training consumption at Critical tier	___	___	100% of Critical-tier consumption attributed to a corpus and steward	☐

Where each number is read from:

• Corpora carrying a current tier assignment — AI data inventory tier field
• Critical-tier corpora receiving full-scope treatment in the last 12 months — Cross-practice records joined to corpus tier
• Erasure requests propagated to embeddings and derived artifacts within the statutory window — DSAR workflow log joined to corpus lineage
• High- and Critical-tier retrieval stores with extraction-pattern detection active — Monitoring detection registry joined to corpus tier
• Assured Spend Ratio for retrieval and training consumption at Critical tier — Consumption ledger joined to corpus inventory

Level 2 Consumption Metrics

Absolute spend is a ledger input, never a metric. Every row is divided by a unit of business work. If AI spend is not allocated to an owning team, record this level as blocked on allocation rather than scoring it down (v3.5 framing §12).

Metric	Denominator	Current	Economic failure it detects	Met?
Retrieval cost per query at Critical tier against the store's rolling median	per retrieval query	___	systematic extraction reconnaissance against a high-value store, which is costly by nature and therefore visible economically before the access pattern alone looks anomalous	☐
Training and fine-tuning compute cost per completed run, by corpus tier	per completed training run	___	unbounded or repeatedly failing training runs consuming budget, and unsanctioned runs against high-tier corpora appearing on the bill before they appear in the registry	☐
Retention cost per retained record at Critical tier, tracked against retention conformance	per retained record	___	cost pressure driving retention below the longest applicable regulatory requirement, which trades a visible invoice line for an invisible evidence gap	☐

Level 2 30-second check (E0)

• ☐ Yes ☐ No — Does every dataset your AI uses carry a risk tier?
• ☐ Yes ☐ No — If someone asked you to delete their data, would it leave the embeddings?
• ☐ Yes ☐ No — Is there a budget limit on queries against your most sensitive retrieval store?

---

Maturity Level 3, Industry-Leading

Objective: Drive corpus tiering and lineage from live pipeline signals, benchmark data-governance unit economics externally, and contribute data-attack intelligence back to the ecosystem.

Prerequisite cells: SM-Data-L2, ML-Data-L2, EG-Data-L2

Question L3.1: Automate lineage, classification and tier from pipeline events

Q3.1: Are corpus lineage, classification and tier derived from pipeline events at 90% or better against a published data-quality service-level objective covering correctness within 48 hours of material content change, with human curation reserved for unestablishable provenance and conflicting signals?

Evidence Required:

• [ ] Derive the data picture from the pipelines themselves rather than from periodic curation
• [ ] Emit lineage events from every extract, transform, embed, train and evaluate step, so a corpus record is assembled from what the pipeline actually did rather than from what a steward believed it would do
• [ ] Run automated classification over corpus samples on a schedule and on ingest, so content drift triggers re-classification and therefore re-tiering without waiting for a human to notice that a documentation store has begun accumulating customer tickets
• [ ] Wire tier changes to open the downstream obligations automatically across access control, monitoring, testing and retention
• [ ] Publish a data-quality service-level objective covering lineage completeness, classification freshness and tier correctness within 48 hours of a material content change
• [ ] Reserve human curation for three cases: corpora whose provenance cannot be established automatically, conflicting classification signals, and genuinely novel archetypes that the existing seven do not describe

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L3.2: Benchmark data-governance unit economics and detection efficacy

Q3.2: Does the semi-annual benchmarking brief compare at least five peer-comparable metrics including retrieval cost per query and training cost per run, with a named peer cohort whose selection is defended and where the program trails recorded as explicitly as where it leads?

Evidence Required:

• [ ] Publish a semi-annual brief comparing the program against named peers on at least five metrics: corpus inventory coverage, erasure propagation time, extraction-detection precision and recall, retrieval cost per query by store archetype, and training cost per completed run
• [ ] Unit economics matter here more than in any other domain, because retrieval and embedding costs are the ones most often optimised without a security review
• [ ] A store whose cost per query is far below the peer median may be efficient, or it may be skipping the authorization and logging that peers pay for
• [ ] Draw comparison data from FinOps Foundation AI-scope working groups for the economics, and from MITRE ATLAS practitioner exchanges, sector information-sharing bodies and CSA AI Safety Initiative groups for detection efficacy
• [ ] Name the peer cohort and defend its selection
• [ ] Record where the program trails as explicitly as where it leads, and route the deltas into the next investment cycle

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L3.3: Contribute observed data-attack intelligence and compute the domain return

Q3.3: Does the program contribute at least four externally accepted data-attack intelligence artifacts per year from its own telemetry, and is the domain return computed from ledger arithmetic covering reclaimed waste and contained extraction rather than asserted?

Evidence Required:

• [ ] Contribute at least four substantive artifacts per year drawn from the organization's own telemetry: poisoning and extraction techniques observed against first-party corpora submitted to MITRE ATLAS, retrieval-store and embedding-store attack patterns contributed to OWASP LLM working groups, data-governance reference patterns published through OpenSSF AI, responsibly disclosed findings submitted to the AI Vulnerability Database, and NIST AI RMF Playbook data-chapter contributions
• [ ] Anonymize and legally vet every submission, and count only artifacts that external bodies accept, because talks and press releases do not move the ecosystem
• [ ] Alongside this, compute the domain's return from the consumption ledger rather than asserting it: reclaimed re-indexing waste, avoided retention overspend where retention was right-sized to the longest applicable obligation rather than to habit, extraction attempts contained valued at burn rate multiplied by detection time, and storage reclaimed by retiring corpora that no model consumes
• [ ] Reconcile the result with Finance annually

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 3 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
Corpus lineage completeness derived from pipeline events rather than curated by hand	___	___	≥90% event-derived	☐
Classification and tier correctness latency after a material content change	___	___	≤48 hours at the 99th percentile	☐
Extraction-detection precision on retrieval stores	___	___	≥80% precision with recall tracked and published	☐
Peer-comparable metrics in the semi-annual brief	___	___	≥5 metrics with a named and defended peer cohort	☐
Data-attack intelligence artifacts accepted externally per year	___	___	≥4 accepted, excluding talks and press	☐

Where each number is read from:

• Corpus lineage completeness derived from pipeline events rather than curated by hand — Lineage event pipeline telemetry
• Classification and tier correctness latency after a material content change — Classification pipeline event log
• Extraction-detection precision on retrieval stores — Detection telemetry with analyst disposition labels
• Peer-comparable metrics in the semi-annual brief — Published benchmarking brief
• Data-attack intelligence artifacts accepted externally per year — Contribution log with external acceptance references

Level 3 Consumption Metrics

Absolute spend is a ledger input, never a metric. Every row is divided by a unit of business work. If AI spend is not allocated to an owning team, record this level as blocked on allocation rather than scoring it down (v3.5 framing §12).

Metric	Denominator	Current	Economic failure it detects	Met?
Retrieval cost per query benchmarked against the external peer cohort median	per retrieval query	___	a cost profile far below peers that reflects skipped authorization, logging or filtering rather than genuine efficiency, which internal trend lines cannot distinguish	☐
Reclaimed re-indexing and retention waste	per indexed document retained	___	embedding stores and log corpora growing without a consuming model, which is pure carrying cost and also pure unnecessary exposure	☐
Extraction-attempt containment value: burn rate multiplied by time to detect	per contained extraction attempt	___	extraction reconnaissance running long enough to be commercially material, which is the component of return that only the ledger can quantify	☐

Level 3 30-second check (E0)

• ☐ Yes ☐ No — Does your data lineage come from the pipeline or from a person?
• ☐ Yes ☐ No — Do you know how your retrieval cost per query compares with peers?
• ☐ Yes ☐ No — Have you contributed a data-attack technique back to the community this year?

---

Summary Scorecard

Level	Q1	Q2	Q3	Level score	Weakest evidence grade	Achieved?
L1	___	___	___	___	___	☐
L2	___	___	___	___	___	☐
L3	___	___	___	___	___	☐

Strategy & Metrics (SM) maturity in the Data domain: ____________

---

Document Version: HAIAMM v3.5 (generated)