# Implementation Review (IR) - Data Domain
## HAIAMM Assessment Questionnaire v3.0

**Practice:** Implementation Review (IR)
**Domain:** Data
**Purpose:** Assess organizational maturity in verifying that the actually-deployed data flows feeding AI/HAI systems match the design approved at DR-Data, and stay there as pipelines, classification schemes, and consumer AI artifacts evolve.
**Scoring Model:** Evidence + Outcome Metrics (see Scoring Methodology below)

---

## Instructions

- Answer each question honestly based on **current, implemented practices** (not plans or aspirations)
- Each question has two components: **Evidence** (what you did) and **Outcome Metrics** (how well it worked)
- **Scoring uses 4 tiers:** Fully Mature (1.0), Implemented (0.67), Partial (0.33), Not Implemented (0.0)
- **Answer progressively**, Complete all Level 1 questions before Level 2
- **Level progression**, Achieve ALL questions at lower level before advancing
- **Baseline first**, Record current metric values before setting targets

---

## Scoring Methodology

| Score | Label | Criteria |
|-------|-------|----------|
| 1.0 | Fully Mature | Evidence complete + ≥3 outcome metrics meet targets |
| 0.67 | Implemented | Evidence complete + 2 outcome metrics meet targets |
| 0.33 | Partial | Evidence partially complete + <2 metrics meet targets |
| 0.0 | Not Implemented | No evidence of practice |

**Level Score** = Average of the three question scores at that level
**Overall Score** = Weighted average across all levels achieved

---

## Maturity Level 1
### Objective: Run per-archetype implementation reviews at go-live, annually, and on material change, verifying deployed data flows match the SA-Data pattern, configuration matches the DR-Data decision, and SR-Data REM evidence is current

At this level, the gap between approved data-flow design and running pipeline is systematically checked at the moments it matters most. Every review produces findings with severity tags, named owners, and SLA-bound resolution dates.

---

### Question 1: Per-Archetype Data IR Checklist

**Q1.1:** Is there a published, per-archetype IR checklist, one per SM-Data archetype (training corpus, inference input stream, retrieval store, prompt/completion log corpus, embedding store, fine-tuning dataset, evaluation/test set), covering classification-label propagation, lineage-as-designed, consent-basis currency, retention enforcement, encryption-key vault binding, access-control verification, and DSAR-query accuracy test?

**Evidence Required:**
- [ ] Published checklist per SM-Data archetype on file and linked from the SM-Data inventory (training corpus, inference input stream, retrieval store, prompt/completion log corpus, embedding store, fine-tuning dataset, evaluation/test set)
- [ ] Inference-input-stream checklist includes a PII-redaction canary test: a canary PII record injected into the test path and the payload reaching the LLM API verified to have the PII redacted; test execution log on file
- [ ] Classification-label propagation verified by pulling a sample of records at each pipeline stage and confirming labels match the approved taxonomy and are not silently stripped by transformation steps
- [ ] Retention-enforcement verification present: deletion job or retention policy execution log with last successful run timestamp; not just a policy declaration
- [ ] DSAR-query accuracy tested: a test DSAR query for a canary subject record executed and verified to return expected data from this archetype's DSAR surface; test result on file
- [ ] No-train flag verified via vendor admin API for flows with outbound LLM calls (OpenAI / Anthropic / Bedrock / Vertex admin APIs), not from DPA text or one-time admin-console screenshots; probe result on file with IR record

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % AI/HAI data flows with a go-live IR record | ___% | ___% | 100% | ☐ | SM-Data inventory × IR records |
| % active AI/HAI data flows with a current-year IR record | ___% | ___% | ≥90% | ☐ | SM-Data inventory × IR records |
| Critical / blocker findings open at go-live | ___ | ___ | 0 | ☐ | Findings backlog |
| Median closure time for High findings | ___ days | ___ days | ≤7 days | ☐ | Findings backlog |

**Metric Collection Guidance:**
- **Go-live IR coverage**: count data flows in SM-Data inventory with a linked IR record dated at or before production cutover divided by total flows entering production. Formula: `flows_with_golive_IR / total_flows_in_production × 100`
- **Current-year IR coverage**: count flows with an IR record dated within the last 12 months divided by total active flows. Source: SM-Data inventory last-IR-date field
- **Blocker findings at go-live**: count Critical/blocker findings that were open on the go-live date. Source: IM-Data findings history
- **High finding closure time**: median calendar days from finding-opened to evidence-linked finding-closed for High-severity findings. Source: IM-Data backlog timestamps

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No per-archetype checklist published)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

### Question 2: Review Triggers and No-Train Vendor Probing

**Q1.2:** Do 100% of new AI/HAI data flows going to production in the last 90 days carry a go-live IR record, with material-change triggers wired to SM-Data inventory events, and are no-train flags verified via vendor admin API for ≥80% of flows with outbound LLM calls?

**Evidence Required:**
- [ ] Go-live IR records on file for all data flows entering production in the last 90 days; catalog and pipeline metadata confirming the IR was completed before production cutover
- [ ] Material-change trigger wired to SM-Data inventory: new data sources, classification changes, cross-border routing changes, retention policy changes, new consumer AI artifacts, and vendor LLM provider changes generate a review-due alert within 5 business days
- [ ] Vendor admin API probe records on file for flows with outbound LLM calls: OpenAI `data_controls.training_data_sharing` = `false`; Anthropic no-train terms; Bedrock Service Control Policy; Vertex AI / Gemini Organization Policy, at go-live, at annual review, and on material change
- [ ] Lineage-as-designed verified: data-catalog lineage graph (Atlan / Collibra / DataHub / Unity Catalog) confirmed to match the approved flow design; no new sources or consumers added since the last DR without a corresponding IR
- [ ] Consent-basis / lawful-basis evidence current: GDPR Art. 6 or Art. 9 lawful-basis record cited in the SR-Data REM confirmed still active; for consent-based processing, a sample of consent records verified
- [ ] Annual review calendar populated from the SM-Data inventory with flows nearing review-due date visible at least 30 days in advance

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % AI/HAI data flows with a go-live IR record | ___% | ___% | 100% | ☐ | SM-Data inventory × IR records |
| % active AI/HAI data flows with a current-year IR record | ___% | ___% | ≥90% | ☐ | SM-Data inventory × IR records |
| % material changes to production data flows that trigger an IR before the change ships | ___% | ___% | 100% | ☐ | SM-Data change events × IR records |
| % flows with outbound LLM calls where no-train confirmed via vendor admin API (not DPA text alone) | ___% | ___% | ≥80% | ☐ | Vendor API probe log |

**Metric Collection Guidance:**
- **Material-change IR trigger rate**: cross-reference SM-Data inventory material-change events against IR records created within 5 business days of each event
- **No-train API coverage**: count flows with outbound LLM calls where a vendor admin API probe result is on file (not DPA text only) divided by total flows with outbound LLM calls. Source: vendor API probe log
- **Lineage accuracy**: % of flows where the live catalog lineage graph matches the DR-approved flow design at the time of IR. Source: catalog diff records × DR flow design
- **Current-year IR coverage**: count flows with an IR record dated within the last 365 days. Source: SM-Data inventory last-IR-date

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No material-change trigger or no-train API probing)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

### Question 3: Findings Tracking and SR-Data REM Loop

**Q1.3:** Are findings severity-tagged and tracked in IM-Data with named owners and SLA-bound closure dates, and does every IR finding that reveals stale or inaccurate REM evidence trigger an SR-Data REM row update before the finding is closed?

**Evidence Required:**
- [ ] Findings backlog in IM-Data showing all IR findings with severity tag (Critical / High / Medium / Low), named owner (not "the data team"), SLA-bound closure date, and linked after-fix evidence artifact
- [ ] SR-Data REM update loop active: IR findings that reveal stale or inaccurate REM evidence have a linked REM row update on file before the finding is closed
- [ ] Pipeline drift sources verified at L1: pipeline metadata (Airflow DAG versions, dbt model versions, Fivetran connector versions) reviewed since the last IR and compared against the DR-Data-approved pipeline specification; deviations documented as findings
- [ ] Encryption-key vault binding verified: key references in deployed pipeline artifact point to the declared KMS vault, not to environment variables, code, or config files; confirmed via secrets scan in the deployed artifact
- [ ] Severity calibration consistent: Critical findings include classification-label gap on a training corpus, PII-redaction step bypassed on an inference input stream, and DSAR-query returning unexpected subjects, not treated as Medium
- [ ] Findings-aging dashboard reviewed at least monthly by the program sponsor: meeting record or dashboard screenshot on file

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % AI/HAI data flows with a go-live IR record | ___% | ___% | 100% | ☐ | SM-Data inventory × IR records |
| % active AI/HAI data flows with a current-year IR record | ___% | ___% | ≥90% | ☐ | SM-Data inventory × IR records |
| Critical / blocker findings open at go-live | ___ | ___ | 0 | ☐ | Findings backlog |
| Median closure time for High findings | ___ days | ___ days | ≤7 days | ☐ | Findings backlog |

**Metric Collection Guidance:**
- **REM update loop rate**: count IR findings where the closure is linked to an SR-Data REM row update vs. total IR findings that identified stale or inaccurate REM evidence. Source: IM-Data × SR-Data REM cross-reference
- **Named-owner coverage**: count findings with a named individual as owner divided by total open findings. Source: IM-Data
- **SLA adherence by severity**: % of High findings closed within 7 days; % of Critical findings closed before go-live. Source: IM-Data timestamps
- **Findings-aging review cadence**: confirm monthly review via calendar record or dashboard export. Source: program sponsor review log

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No findings backlog or severity tagging in place)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

## Maturity Level 2
### Objective: Detect data-flow drift continuously for Critical and High-tier flows via catalog webhooks, pipeline-metadata events, lineage-graph signals, classification-scan deltas, and vendor admin API recurrent probes; calibrate IR cadence per SM-Data tier

At this level, implementation review becomes a continuous signal. Catalog-change webhooks, pipeline-metadata events, lineage-graph signals, classification-label-scan deltas, and vendor admin API probes are all wired to automated detection.

---

### Question 4: Continuous Data-Flow Drift Detection

**Q2.1:** Are ≥90% of Critical-tier AI/HAI data flows under continuous drift detection, via data-catalog change webhooks, pipeline-metadata events, lineage-graph signals, classification-label-scan deltas, vendor admin API recurrent probes, and cross-border-flow routing monitoring, with median detection latency ≤7 days?

**Evidence Required:**
- [ ] Catalog drift detection active: Atlan / Collibra / DataHub / Unity Catalog change webhooks configured for Critical and High-tier flows; classification-label changes, ownership changes, policy-tag changes, and new downstream consumers trigger automated diff against DR-Data-approved baseline with findings auto-generated on material deviations
- [ ] Pipeline drift detection active: Airflow DAG changes, dbt model changes, Fivetran connector version changes, new source additions compared against the DR-Data-approved pipeline specification; deviations flagged
- [ ] Lineage drift detection active: new upstream sources or downstream consumers appearing in the lineage graph since the last IR automatically open findings; any lineage edge not present in the approved flow design generates a finding
- [ ] Classification drift detection active: Macie / BigID / Purview scan deltas vs. baseline run on a schedule; newly discovered PII or regulated data classes in an archetype not approved for that class open Critical or High findings
- [ ] Cross-border flow drift detection: routing metadata compared against the approved cross-border transfer map; new jurisdictions or changed transfer mechanisms open IR findings within the detection SLA
- [ ] Drift-detection pipeline health monitored: % Critical flows producing a fresh signal in the last 7 days; on-call alert configured for >48 hours of feed silence; alert test record on file

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier data flows under continuous drift detection (catalog, pipeline, lineage, classification-scan, vendor admin API) | ___% | ___% | ≥90% | ☐ | Drift-detection telemetry |
| Median drift detection latency, Critical-tier | ___ days | ___ days | ≤7 days | ☐ | IR telemetry |
| % Critical/High-tier flows with outbound LLM calls where no-train confirmed via vendor admin API recurrent probe | ___% | ___% | ≥80% | ☐ | Vendor API probe log |
| Tier-cadence adherence (% of flows reviewed on their published cadence) | ___% | ___% | ≥95% | ☐ | IR schedule × SM-Data inventory |

**Metric Collection Guidance:**
- **Continuous drift detection coverage**: count Critical-tier flows with all six signal sources active (catalog, pipeline, lineage, classification-scan, vendor admin API, cross-border) divided by total Critical-tier flows. Source: drift-detection pipeline configuration registry
- **Drift detection latency**: median time from change-event timestamp to IR-finding-opened timestamp for Critical-tier drift findings. Source: IR telemetry × IM-Data
- **Vendor admin API recurrent probe coverage**: count Critical/High-tier flows with outbound LLM calls where a vendor admin API probe result is dated within the current probe cycle divided by total Critical/High-tier flows with outbound LLM calls. Source: vendor API probe log
- **Tier-cadence adherence**: count flows reviewed within their published cadence window divided by total flows. Source: IR schedule × SM-Data inventory last-IR-date

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No automated drift detection in place)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

### Question 5: Vendor Admin API Recurrent Probing for No-Train and Retention

**Q2.2:** Are no-train flags and retention settings verified via vendor admin APIs on a monthly (Critical) and quarterly (High) probing cadence, not from DPA text alone, covering ≥80% of Critical/High-tier flows with outbound LLM calls?

**Evidence Required:**
- [ ] Vendor admin API probe records on file for Critical/High-tier flows with outbound LLM calls: OpenAI Org Settings API (`data_controls.training_data_sharing` = `false`); Anthropic Organization admin settings; Bedrock Service Control Policy and invocation logging config; Vertex AI / Gemini Organization Policy constraints
- [ ] Probing cadence log showing Critical-tier probes monthly and High-tier probes quarterly, not one-time screenshots; all probe runs date-stamped with method (API or fallback UI)
- [ ] Delta findings on file: any setting change detected (no-train toggle reset, retention period changed, logging config altered) generated an IR finding with severity matching the data-class impact of the change
- [ ] Vendor API probing calendar maintained: missed probes tracked as process-metric failures with root cause and remediation on file
- [ ] Classification-scan deltas compared to the DR-Data-approved baseline: Macie / BigID / Purview scan output compared against the approved classification scope; newly discovered classes generate findings
- [ ] IR findings from probe deltas routed to IM-Data automatically with severity and owner pre-populated

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier data flows under continuous drift detection (catalog, pipeline, lineage, classification-scan, vendor admin API) | ___% | ___% | ≥90% | ☐ | Drift-detection telemetry |
| Median drift detection latency, Critical-tier | ___ days | ___ days | ≤7 days | ☐ | IR telemetry |
| % Critical/High-tier flows with outbound LLM calls where no-train confirmed via vendor admin API recurrent probe | ___% | ___% | ≥80% | ☐ | Vendor API probe log |
| % Critical/High-tier flows contributing to DSAR surface with DSAR-query accuracy test on record (current IR cycle) | ___% | ___% | 100% | ☐ | IR records |

**Metric Collection Guidance:**
- **Vendor admin API recurrent probe coverage**: count Critical/High-tier flows with a vendor admin API probe result dated within the current probe cycle divided by total flows with outbound LLM calls. Source: vendor API probe log
- **Delta finding rate**: count probe cycles where a setting change was detected and a finding was generated vs. total probe cycles. Source: probe log × IM-Data
- **Probe cadence compliance**: count monthly Critical-tier probes completed on schedule vs. planned. Source: probing calendar
- **Classification-scan delta coverage**: % of Critical/High-tier flows with a classification-scan delta report compared to the DR-Data-approved baseline in the current IR cycle. Source: scan tooling output

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No recurrent vendor admin API probing)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

### Question 6: DSAR-Query Accuracy Verification and Tier-Calibrated Cadence

**Q2.3:** Are 100% of Critical/High-tier data flows contributing to the DSAR surface covered by DSAR-query accuracy tests in the current IR cycle, and is tier-cadence adherence ≥95% with Critical-tier findings aged per the SM-Data L2 tier-treatment matrix SLAs?

**Evidence Required:**
- [ ] DSAR-query accuracy test records on file for Critical/High-tier flows contributing to the DSAR surface: canary subject record inserted, subject-access query executed via the DSAR fulfillment system, canary record returned with correct attributes, no co-mingling with other subjects, and canary record correctly excluded after a deletion request is processed
- [ ] Cadence for DSAR tests documented and executed: Critical-tier quarterly, High-tier semi-annual; test records date-stamped with method and reviewer
- [ ] Tier-cadence enforcement visible in SM-Data inventory: Critical-tier flows flagged when no IR in the last 180 days; escalation to program sponsor automated and test record on file
- [ ] IR backlog tier-aware: Critical-tier findings reported separately; no Critical-tier finding waiting behind Low-tier queue items
- [ ] DSAR-test failures documented as High findings (DSAR-surface exposure risk) with named owner and SLA-bound closure date
- [ ] Tier-treatment matrix SLA compliance tracked: Critical-tier findings closed per SM-Data L2 SLA (High ≤7 days, Critical blocker before go-live); SLA breach rate reported to program sponsor

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier data flows under continuous drift detection (catalog, pipeline, lineage, classification-scan, vendor admin API) | ___% | ___% | ≥90% | ☐ | Drift-detection telemetry |
| Median drift detection latency, Critical-tier | ___ days | ___ days | ≤7 days | ☐ | IR telemetry |
| % Critical/High-tier flows contributing to DSAR surface with DSAR-query accuracy test on record (current IR cycle) | ___% | ___% | 100% | ☐ | IR records |
| Tier-cadence adherence (% of flows reviewed on their published cadence) | ___% | ___% | ≥95% | ☐ | IR schedule × SM-Data inventory |

**Metric Collection Guidance:**
- **DSAR-query accuracy test coverage**: count Critical/High-tier flows contributing to the DSAR surface with a DSAR-query accuracy test record in the current IR cycle divided by total Critical/High-tier flows contributing to the DSAR surface. Source: IR records
- **DSAR test pass rate**: % of DSAR-query accuracy tests where canary record was included correctly and excluded correctly after deletion. Source: test records
- **Tier-cadence adherence**: % of flows reviewed within their published cadence window. Source: SM-Data inventory last-IR-date × cadence policy
- **Tier-SLA breach rate**: count Critical-tier findings exceeding the SM-Data L2 tier-treatment matrix SLA divided by total Critical-tier findings. Source: IM-Data timestamps

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No DSAR-query accuracy testing or tier-calibrated cadence)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

## Maturity Level 3
### Objective: Daily attestation per Critical-tier data flow, classification labels current, retention SLA met, encryption-key-rotation healthy, no-train probe results green, with drift auto-opening IM-Data tickets and configuration baseline schemas contributed to OpenSSF AI Data, DAMA, and OWASP SAMM AI

At this level, configuration for Critical-tier data flows is attested continuously across four dimensions. Every Critical data flow produces a daily attestation signal. Drift auto-opens IM-Data tickets.

---

### Question 7: Daily Attestation Signal for Data Flows

**Q3.1:** Are ≥90% of Critical-tier AI/HAI data flows producing a daily attestation signal across all four dimensions (classification-label currency, retention SLA, encryption-key-rotation health, no-train probe results green), with deviations auto-opening IM-Data tickets within 1 hour?

**Evidence Required:**
- [ ] Daily attestation pipeline operational covering four dimensions per Critical-tier flow: (1) classification labels current, automated catalog scan confirms labels match approved taxonomy and have not been silently modified in the last 24 hours; (2) retention SLA met, deletion-job execution log confirms job ran within expected window and no records past retention date remain; (3) encryption-key-rotation healthy, KMS confirms key is within declared rotation schedule and has not been exported outside vault; (4) no-train probe results green, vendor admin API probe result from the most recent probe cycle (≤30 days) confirms no-train setting active
- [ ] Auto-open IM-Data ticket on drift: ticket created within 1 hour of detection; carries drift dimension, specific control that failed tolerance, and link to DR-Data decision record
- [ ] Attestation-pipeline health monitored: % Critical flows with fresh signal in the last 24 hours; on-call paged for any Critical flow silent for >24 hours; pager history on file
- [ ] Zero stale-evidence violations confirmed for Critical-tier REMs: no evidence citations outside their defined freshness window
- [ ] Attestation artifacts regulator-consumable: machine-readable format confirmed suitable for GDPR Art. 35 DPIA evidence, EU AI Act Art. 10 data-governance records, and ISO/IEC 42001 AIMS operational records via test export
- [ ] Configuration tolerance definitions documented per-control: tolerance rules versioned and linked from SM-Data inventory; tolerances tuned to avoid alert fatigue from minor pipeline dependency version bumps

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier data flows producing a daily attestation signal (all 4 dimensions) | ___% | ___% | ≥90% | ☐ | Attestation telemetry |
| % attestation findings auto-opening IM-Data tickets within 1 hour of detection | ___% | ___% | ≥95% | ☐ | IM-Data integration telemetry |
| Evidence freshness violations (stale evidence in active REMs) | ___ | ___ | 0 for Critical; trending toward 0 for High | ☐ | Attestation telemetry |
| IR reviewer-hours per Critical data flow per year | ___ hrs | ___ hrs | trending down QoQ | ☐ | Reviewer time tracking |

**Metric Collection Guidance:**
- **Daily attestation coverage**: count Critical-tier flows with a signed attestation artifact dated within the last 24 hours covering all four dimensions divided by total Critical-tier flows. Source: attestation pipeline telemetry
- **Auto-open ticket latency**: median time from drift-detection event to IM-Data ticket created. Source: attestation pipeline timestamp × IM-Data created-at timestamp
- **Stale-evidence violations**: count Critical-tier REM rows with evidence citations outside their freshness window. Source: attestation pipeline evidence-freshness dimension output
- **IR reviewer-hours**: total hours logged by reviewers for Critical-tier data IR activities per year. Source: reviewer time tracking

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No daily attestation signal in place)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

### Question 8: External Data-Flow Schema Contribution

**Q3.2:** Has the program published per-archetype data-flow configuration baseline schemas to OpenSSF AI Data, DAMA International, OWASP SAMM AI, or CSA AI Safety Initiative, with documented external adoption and internal practice aligned to the published versions?

**Evidence Required:**
- [ ] Per-archetype data-flow configuration baseline schemas published to at least one external body: OpenSSF AI Data working group, DAMA International AI Data Governance, OWASP SAMM AI extensions, or CSA AI Safety Initiative AI Controls Matrix
- [ ] Internal practice aligned to published external versions: documented mapping between internal IR checklist and published schema; internal-only deviations proposed as upstream changes with a PR or issue link
- [ ] External adoption tracked: citations, forks, direct acknowledgment from peer organizations, or inclusion in external tooling or assessment frameworks, evidence on file
- [ ] Schema publication pipeline active: at least one schema in-draft, in-review, or published at any time; publication status tracker maintained
- [ ] Adoption trend documented: at least one adoption data point on file since initial publication
- [ ] Internal schema version pinned to published external version: divergence from the external version documented as a proposed upstream change

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier data flows producing a daily attestation signal (all 4 dimensions) | ___% | ___% | ≥90% | ☐ | Attestation telemetry |
| % attestation findings auto-opening IM-Data tickets within 1 hour of detection | ___% | ___% | ≥95% | ☐ | IM-Data integration telemetry |
| External adoption of published configuration baseline schemas | 0 | tracked | trending up | ☐ | External telemetry |
| IR reviewer-hours per Critical data flow per year | ___ hrs | ___ hrs | trending down QoQ | ☐ | Reviewer time tracking |

**Metric Collection Guidance:**
- **External adoption**: count citations, forks, and direct acknowledgments per quarter since publication. Source: GitHub stars/forks, external publication references, peer organization acknowledgments
- **Schema alignment rate**: % of internal IR checklist items directly traceable to a published external schema item. Source: internal-to-external schema mapping document
- **Publication pipeline health**: number of schemas in active publication stages. Source: publication tracker

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No schemas published externally)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

### Question 9: Automated Drift-to-IM Escalation and Post-Incident Feedback Loop

**Q3.3:** Is the post-incident IR feedback loop operational, IM-Data post-incident reviews include a mandatory IR-record re-examination step, and ≥1 attestation rule update is produced per material incident?

**Evidence Required:**
- [ ] All IR findings (daily attestation and periodic reviews) flow into IM-Data automatically with severity and SLA pre-populated from the SM-Data L2 tier-treatment matrix; no manual entry required for attestation-generated findings
- [ ] IM-Data SLA clock automated: overdue Critical findings escalate to the program sponsor at 50% and 100% of the SLA window automatically; escalation automation tested quarterly with synthetic finding, test record on file
- [ ] Post-incident IR feedback loop documented in the IM-Data incident-review template: a mandatory IR-record re-examination step confirmed in the last material incident review record
- [ ] Attestation rule updates produced from incident learning: at least one attestation rule update linked from a completed incident review in the last 12 months
- [ ] Post-incident IR step completion tracked: % of material incidents in the last 12 months with a completed post-incident review including the IR-record re-examination step
- [ ] Attestation-pipeline health log on file: ≥90% of Critical flows with fresh signal in the last 24 hours; pager history on file

**Outcome Metrics:**

| Metric | Baseline | Current | Target | Met? | Notes |
|--------|----------|---------|--------|------|-------|
| % Critical-tier data flows producing a daily attestation signal (all 4 dimensions) | ___% | ___% | ≥90% | ☐ | Attestation telemetry |
| % attestation findings auto-opening IM-Data tickets within 1 hour of detection | ___% | ___% | ≥95% | ☐ | IM-Data integration telemetry |
| Evidence freshness violations (stale evidence in active REMs) | ___ | ___ | 0 for Critical; trending toward 0 for High | ☐ | Attestation telemetry |
| IR reviewer-hours per Critical data flow per year | ___ hrs | ___ hrs | trending down QoQ | ☐ | Reviewer time tracking |

**Metric Collection Guidance:**
- **Auto-open ticket rate**: count attestation-generated findings that created an IM-Data ticket within 1 hour divided by total attestation-generated findings. Source: attestation pipeline × IM-Data integration telemetry
- **Escalation automation test pass rate**: % of quarterly escalation tests that confirmed correct automated escalation. Source: escalation test records
- **Post-incident IR step completion**: % of material incident post-reviews with the IR-record re-examination step completed. Source: IM-Data incident review records
- **Attestation rule update rate**: count attestation rule updates linked from post-incident reviews in the last 12 months. Source: attestation pipeline rule-change log × IM-Data incident reviews

**Answer:**
- ☐ **Fully Mature** (Evidence complete + ≥3 metrics meet targets)
- ☐ **Implemented** (Evidence complete + 2 metrics meet targets)
- ☐ **Partial** (Evidence partially complete + <2 metrics meet targets)
- ☐ **Not Implemented** (No automated escalation or post-incident feedback loop)

**Evidence Location:** _________________________________

**Validation Date:** _________________________________

**Notes:** ___________________________________________

---

## Summary Scorecard

| Level | Question | Score | Weight |
|-------|----------|-------|--------|
| L1 | Q1: Per-Archetype Data IR Checklist | ___ | 33% |
| L1 | Q2: Review Triggers and No-Train Vendor Probing | ___ | 33% |
| L1 | Q3: Findings Tracking and SR-Data REM Loop | ___ | 33% |
| **L1 Total** | | ___ | |
| L2 | Q4: Continuous Data-Flow Drift Detection | ___ | 33% |
| L2 | Q5: Vendor Admin API Recurrent Probing | ___ | 33% |
| L2 | Q6: DSAR-Query Accuracy and Tier-Calibrated Cadence | ___ | 33% |
| **L2 Total** | | ___ | |
| L3 | Q7: Daily Attestation Signal for Data Flows | ___ | 33% |
| L3 | Q8: External Data-Flow Schema Contribution | ___ | 33% |
| L3 | Q9: Automated Escalation and Post-Incident Feedback | ___ | 33% |
| **L3 Total** | | ___ | |
| **Overall IR-Data Score** | | ___ | |

**Current Maturity Level:** ___

**Key Findings:**

**Priority Improvements:**

---

**Document Version:** HAIAMM v3.0
**Practice:** Implementation Review (IR)
**Domain:** Data
**Questionnaire Date:** 2026-05-15
**Author:** Verifhai