Design Review (DR)
Processes Domain - HAIAMM v3.0
Practice Overview
Objective: Operate the design checkpoint between intake approval and build-out for every new AI-embedded business workflow, confirming the proposed design follows the applicable SA-Processes reference pattern, covers the SR-Processes requirements pack, satisfies applicable compliance obligations, and documents residual risks before the workflow goes live.
Description: DR-Processes is the single moment where workflow architecture (SA-Processes), requirements (SR-Processes), and threats (TA-Processes) meet a specific planned AI-embedded business workflow. At L1 the review is deliberate but lean: a per-archetype design checklist, a named reviewer, and a written decision (approve / approve-with-conditions / send-back) stored against the SM-Processes inventory record. The review runs before the workflow goes live, catching design flaws when correction costs hours, not months of production rollback. A two-lane model routes Low / Medium-tier workflows to an async fast-lane (≤2 BD) and High / Critical-tier, Annex III, Art. 22, or sector-regulated workflows to a full-lane architect review (≤5 BD) with Privacy and Legal participation. Loop-back signals ensure the review process improves SA-Processes patterns and SR-Processes packs over time rather than accumulating silent governance debt.
Context: Without a design checkpoint, AI-embedded business workflows go live without verified HITL gates, without Art. 50 disclosure UX designed, without decision logging scoped, without appeal or contestation paths surfaced. The SA reference pattern and SR requirements pack exist, but delivery teams skip them under pressure, deviate without recording rationale, or simply launch before the archetype pattern is consulted. DR-Processes enforces the handoff between "design approved" and "workflow goes live," making deviations visible and deliberate. EU AI Act Art. 9 risk management requires documented pre-deployment decisions for high-risk AI systems; Art. 22 requires that affected persons have the right to human review and explanation; the DR decision record is the documented pre-deployment decision that FRIA and DPIA evidence references. Cross-domain handoff is explicit: when a workflow embeds AI features built in-house, the AI features receive DR-Software; the workflow orchestrating those features receives DR-Processes.
Maturity Level 1
Objective: Run a per-archetype design checkpoint for every new AI-embedded workflow before go-live, producing a written decision traceable to the SA-Processes pattern, SR-Processes requirements pack, and TA-Processes threat snapshot
At this level, design review is a consistent gate, not an after-the-fact audit when a workflow produces contested outcomes. Every AI-embedded workflow above the triage threshold receives a review before go-live, and every review produces a written decision linked to the SM-Processes inventory record.
Dependencies
- SA-Processes L1 (required): the reference patterns for each workflow archetype are what the checklist measures the proposed design against; without patterns there is no review baseline.
- SR-Processes L1 (required): the requirements pack (base plus archetype deltas) defines the acceptance bar; the REM is the primary evidence input to the review.
- TA-Processes L1 (required): the per-archetype threat snapshot names what the design must defend against; the reviewer walks the snapshot's top threats against the proposed workflow design.
- EG-Processes L1 (required): reviewers must understand AI-embedded workflow archetypes, HITL design patterns, and applicable compliance obligations (Art. 22 / Art. 50 / Annex III) before they can produce a credible DR decision.
- Supports / unblocks: IR-Processes L1 (implementation reviews check the running workflow against the approved design), ST-Processes L1 (tests target the approved workflow controls), IM-Processes L1 (incident triage references design assumptions).
Desired Outcomes
- Every AI-embedded workflow above the triage threshold is reviewed before go-live; no workflow touches affected persons without a DR decision on file.
- DR decisions are written, versioned, and stored against the SM-Processes inventory record, not tribal knowledge held by the reviewing architect or business owner.
- Deviations from SA-Processes reference patterns are approved or rejected explicitly with a named reviewer, a rationale, and a residual-risk acceptance.
- Annex III and Art. 22 triggers are identified at design time, not discovered post-deployment, and linked to the DR decision record with FRIA status noted.
- Review is timeboxed: teams know whether to expect a 2 BD async check or a 5 BD full-lane session based on the SM-Processes tier and deviation status.
- Recurring deviation themes feed back into SA-Processes (pattern updates) and SR-Processes (pack updates), the review accumulates organizational learning, not only per-ticket decisions.
Activities
A) Publish the per-archetype AI-Embedded Workflow Design Checklist
One checklist per SM-Processes archetype, derived from the applicable SA-Processes reference pattern and keyed to the SR-Processes base pack and archetype delta. Each item is a yes/no with an evidence pointer. The seven checklists share a common spine and carry archetype-specific additions.
Common spine across all seven checklists: - Pattern adherence, using the SA-Processes reference pattern or documented deviation with rationale. - HITL placement and depth, human-in-the-loop gate placement declared; reviewer role, trigger condition, timeout, and fallback all specified; reviewer-capacity model confirms the declared SLA is achievable without rubber-stamping. - Art. 50 disclosure design, if the workflow produces AI-generated content or makes AI-assisted decisions visible to external persons, the disclosure UX is designed and present; disclosure requirement linked to PC compliance map. - Decision logging, which decisions are logged, at what field granularity, by which system, with what retention period; traceability to affected person is confirmed. - Override audit trail, every human override of an AI recommendation is captured with reviewer identity, rationale, and timestamp; the audit trail is queryable. - Output-integrity SLA, acceptable error rate, staleness window, and degraded-mode behavior for the AI component declared. - Reviewer-capacity gating, the workflow design cannot advance if the declared HITL SLA is arithmetically impossible given the staffed reviewer headcount. - Affected-persons rights surface, the channel through which an affected person can contest an AI-assisted outcome is identified and responsive; contestation path tested in design. - DPIA / FRIA status, DPIA triggered if personal data processed at scale under GDPR Art. 35; FRIA required if workflow qualifies as Annex III high-risk AI; status linked from DR record. - Fallback / kill-switch, documented fallback or manual-process path for AI component outage; kill-switch design specified and test plan defined.
Archetype-specific additions:
-
Decision pipeline: Annex III screen completed (is the decision within a listed high-risk AI use-case category?); FRIA gate status recorded; Art. 22 lawful basis documented (automated decision-making with legal or similarly significant effect requires Art. 22 exception or human-in-the-loop sufficient to remove "solely automated" character); appeal and explanation path designed (how does an affected person challenge the decision and receive an explanation?); class-shift monitoring baseline defined (decision rate by protected-characteristic proxy tracked from day one).
-
Customer-facing flow: Art. 50 disclosure UX present and visible on every AI-touched customer interaction (not buried in terms); brand-safety filter placement declared; escalation path from AI output to human agent designed with SLA specified; output-integrity SLA for customer-visible content declared.
-
Human-AI collaboration chain: review-UI design surfaces AI rationale, confidence level, and at least one counterfactual to the human reviewer, not only the recommendation; reviewer-capacity model confirmed; injection-defense confirmed (review UI does not execute embedded content from AI output).
-
Back-office augmentation: tool scope bounded (the AI component's tool access is the minimum necessary for the stated task); classification-aware routing confirmed (regulated or sensitive output triggers a review gate rather than direct downstream consumption); output-review gate designed.
-
Approval / review workflow: class-shift monitor designed (approval rate by tier and by protected-characteristic proxy tracked from day one); tier-routing logic reviewed (which cases require human review and which are auto-approved is explicit and documented).
-
Content-generation workflow: copyright and brand-voice filter placement declared; output-review gate required for material outputs (content reaching external audiences, legal documents, patient-facing materials); downstream-input-validation design confirms generated content cannot inject downstream systems.
-
Knowledge-management workflow: provenance requirement declared (every retrieved chunk carries source, classification, and trust label); classification-aware retrieval confirmed (retrieval scope bounded by requester's access class); injection-defense scope declared (retrieved content treated as untrusted in the prompt structure); role-based retrieval access model designed.
B) Triage and route reviews by risk tier and deviation status
The two-lane model is driven by the SM-Processes tier assignment and the deviation flag:
- Fast-lane (Low / Medium tier, on-pattern, no Annex III, no Art. 22, no sector-regulated): async checklist review by the designated reviewer; target SLA ≤2 business days. Output: one structured decision record, approve / approve-with-conditions (explicit list) / send-back (reasons stated), stored against the SM-Processes inventory record.
- Full-lane (High / Critical tier OR Annex III trigger OR Art. 22 solely-automated-decision risk OR sector-regulated OR any pattern deviation): architect review with Privacy and Legal walking the SA-Processes reference pattern section-by-section with the business owner; target SLA ≤5 business days. Output: written decision record with the residual-risk list reviewed by a named architect; Privacy and Legal sign-off recorded for Art. 22 and Annex III workflows.
Triage rules at L1 (before SM-Processes L2 tiers are established): decision pipelines and customer-facing flows processing personal data default to full-lane. All others default to fast-lane with override to full-lane available on reviewer judgment.
Decision record contents (both lanes): decision (approve / approve-with-conditions / send-back); checklist completed with evidence pointers; deviations listed with rationale; residual risks listed with named owner and expiry; reviewer name and date; Privacy and Legal acknowledgment for Art. 22 / Annex III workflows; links to SM-Processes inventory record, TA threat snapshot, SR REM, and FRIA / DPIA status.
C) Close the loop with SA-Processes, SR-Processes, and IM-Processes
Design review is a learning surface for the program:
- SA pattern update trigger: three deviations in the same direction for the same archetype auto-queue a pattern-update review with SA-Processes ownership. Recurring deviations signal a miscalibrated pattern, not a non-compliant team.
- SR pack update trigger: an SR requirement repeatedly waived with a compensating control auto-queues an SR pack-revision review. If every decision-pipeline team waives the same base requirement, the pack needs recalibration.
- IM incident feedback loop: every IM-Processes incident re-examines the DR decision record that approved the affected workflow. Was the issue visible at design time? Which checklist item would have caught it? The answer updates the checklist and feeds the next archetype review cycle.
Outcome Metrics (L1)
| Metric | Baseline | L1 Target | Source |
|---|---|---|---|
| % AI-embedded workflows going live with a completed DR decision record | measure | ≥95% | SM inventory × DR records |
| % DR decision records referencing the applicable SA pattern and SR REM | measure | 100% | DR records |
| % Annex III / Art. 22 workflows with FRIA / DPIA status documented in the DR record | measure | 100% | DR records × compliance tracker |
| Median review turnaround, fast-lane | measure | ≤2 business days | Review SLA telemetry |
| Median review turnaround, full-lane | measure | ≤5 business days | Review SLA telemetry |
| Open approve-with-conditions items aging >60 days | measure | 0 | Action-item backlog |
Process Metrics (leading)
- Reviewer population staffed and trained (EG-Processes L1 completion confirmed); named lead reviewer per archetype.
- Fast-lane vs. full-lane ratio monitored, a drift toward all-fast-lane may indicate under-review; toward all-full-lane may indicate over-routing.
- Pattern-deviation rate tracked by archetype, feeds the SA pattern-update trigger.
- Checklists aligned to current SA patterns and SR pack; updated within 30 days of any SA or SR change.
Effectiveness Metrics (business value)
- Issues caught at design vs. caught at IR or in incident, design-stage catch rate trends up as the program matures.
- Art. 22 / Annex III compliance findings caught pre-deployment (vs. post-deployment audit findings), design gate effectiveness for regulatory exposure.
- SA / SR update volume driven by DR feedback, a healthy program generates pattern and pack improvements, not only per-ticket decisions.
Success Criteria
- Per-archetype AI-Embedded Workflow Design Checklists published, versioned, and traceable to the applicable SA reference pattern, SR requirements pack, and TA threat snapshot, all seven archetypes covered.
- Two-lane review model operational with published SLAs (≤2 BD fast-lane, ≤5 BD full-lane) and named lead reviewers per archetype.
- ≥95% of AI-embedded workflows going live in the last 90 days carry a completed DR decision record; 100% of Annex III / Art. 22 workflows include FRIA / DPIA status.
- SA pattern-update and SR pack-update triggers wired; every IM-Processes incident re-examines the DR record that approved the affected workflow.
- Named reviewer population trained (EG-Processes L1) and active; Privacy and Legal participation confirmed for full-lane reviews.
Maturity Level 2
Objective: Upgrade Critical-tier reviews to scenario-based walkthroughs using TA-Processes per-workflow models, conduct FRIA workshops for Annex III workflows, detect design drift for High and Critical workflows on a published cadence, and run cross-domain coordination with DR-Software for workflows wrapping first-party AI features
At this level, design review for Critical-tier workflows moves from checklist conformance to scenario-based conversations driven by TA-Processes per-workflow deep threat models. FRIA workshops replace checklist sign-offs for Annex III workflows. Design drift, the live workflow diverging from the approved design, is detected on a published cadence and automatically re-routed to DR. Where a Critical-tier workflow wraps a first-party AI feature, DR-Processes coordinates a joint review with DR-Software on the explicit handoff boundary.
Dependencies
- DR-Processes L1 (required): per-archetype checklists, two-lane model, and loop-back triggers.
- TA-Processes L2 (required): per-workflow deep threat models are the source material for Critical-tier scenario walkthroughs.
- SA-Processes L2 (required): tier-conditional pattern overlays are what L2 reviews verify the workflow design against.
- SM-Processes L2 (required): the risk-tier rubric (Critical / High / Medium / Low) drives which workflows receive scenario-based reviews and the per-tier drift-detection cadence.
- Supports / unblocks: IR-Processes L2 (drift detected here re-opens IR), ST-Processes L2 (scenario-based reviews feed the workflow test suite).
Desired Outcomes
- Every Critical-tier DR covers 3–5 specific threat or compliance scenarios from the TA-Processes library, with the decision tied explicitly to how the proposed workflow design handles each scenario.
- Annex III workflows receive a structured FRIA workshop before go-live, not a checkbox against a template.
- Design drift is detected quarterly for Critical-tier and annually for High-tier; material drift automatically re-opens the DR record.
- Where a Critical-tier workflow wraps a first-party AI feature, DR-Processes and DR-Software produce joint review records with an explicit handoff boundary and shared residual-risk ownership.
- Fewer IR-stage surprises, drift caught at the design review or drift-detection phase rather than at implementation review or incident.
Activities
A) Scenario-based reviews for Critical and High-tier workflows
For every Critical-tier workflow, the full-lane checklist walkthrough is replaced by a scenario walkthrough:
- Source 3–5 specific threat or compliance scenarios from the TA-Processes per-workflow deep threat model, from anonymized IM-Processes incidents of the same archetype, and from applicable regulatory scenarios (Annex III high-risk category, Art. 22 automated-decision challenge, Art. 50 disclosure failure, sector-specific). Scenarios must be specific to this workflow's HITL placement, data classes, affected-person population, and AI component, not generic archetype scenarios.
- Walk each scenario: "If this workflow produces an incorrect AI recommendation and the reviewer rubber-stamps it, what is the outcome for the affected person? Which design control prevents or detects this?" The DR decision record maps each scenario to a design control or an accepted residual risk with a named owner and expiry.
- Scenario sources: TA-Processes per-workflow deep threat model; anonymized IM-Processes incidents from the same archetype; MITRE ATLAS technique candidates relevant to the AI components in the workflow (EA/AGH/TM/RA as applicable); OWASP LLM / Agentic Top 10 entries relevant to the embedded AI feature's archetype.
- For High-tier workflows: standard full-lane review augmented with at least one scenario from the TA archetype library; not a full scenario walkthrough.
B) FRIA workshops for Annex III workflows and cross-domain joint reviews
FRIA workshops: For any workflow that clears the Annex III screen, the DR-Processes full-lane review is replaced or extended with a structured Fundamental Rights Impact Assessment workshop:
- Workshop attendees: named architect, Privacy, Legal, business owner, and a representative of the affected-person population (or their proxy) where feasible.
- Workshop agenda: (1) map affected-person populations and impacts; (2) enumerate fundamental rights at stake (Art. 22 right to explanation, Art. 8 data protection, Art. 47 right to an effective remedy, sector-specific rights); (3) assess likelihood and severity of rights impact across the workflow's decision paths; (4) design or confirm mitigations; (5) document residual rights exposure with named owner and review cadence.
- Workshop output: completed FRIA artifact linked from the DR decision record; FRIA status updated in the SM-Processes inventory; feeds back to TA-Processes threat library and SR-Processes pack.
Cross-domain joint reviews: When a Critical-tier workflow wraps a first-party AI feature (an approval workflow calling an internally built scoring model, a customer-facing flow surfacing an internally built LLM output):
- DR-Processes reviewer and DR-Software reviewer attend the same session; the handoff boundary is explicitly documented in both DR records (which controls are the workflow's responsibility vs. the AI feature's responsibility).
- Residual risks spanning both are noted in both records with shared ownership and a single named resolution owner.
- Where the AI feature has no DR-Software record, DR-Processes flags the gap and holds the workflow's Sanctioned status until DR-Software completes.
C) Design-drift detection
Compare the live production workflow against its approved DR design at the published cadence:
- Critical-tier: quarterly drift check. Sources checked: workflow-tool config repos (Camunda / Temporal / Argo / ServiceNow BPM model versions); product-flow analytics (where the AI step actually fires, compared to the approved design); HITL queue configuration (reviewer routing, timeout settings, escalation rules); override-audit-log schema (fields captured vs. fields declared in the DR); Art. 50 disclosure presence in the deployed UI (screenshot audit vs. approved disclosure design).
- High-tier: annual drift check using the same sources.
- Material drift (HITL gate removed or bypassed, AI component swapped, new affected-person population added, Art. 50 disclosure removed, decision-logging scope reduced, class-shift monitor disabled) automatically re-opens the DR record and routes back through the appropriate lane.
- Drift check produces a written artifact: the diff between approved design and live workflow, each delta classified as material / non-material, material deltas tracked to DR re-review or accepted residual.
Outcome Metrics (L2)
| Metric | Baseline | L2 Target | Source |
|---|---|---|---|
| % Critical-tier DR records using scenario-based walkthrough | measure | 100% | DR records |
| % Annex III workflows with a completed FRIA workshop before go-live | measure | 100% | DR records × Annex III register |
| % Critical/High-tier workflows with drift check on published cadence | measure | ≥95% | Drift-check schedule × SM inventory |
| % material drift findings re-routed to DR | measure | 100% | Drift-detection queue |
| % Critical-tier workflows wrapping first-party AI features with a joint DR-Processes / DR-Software record | measure | 100% | DR records × software integration tracker |
| IR-stage design surprises (findings at IR with no corresponding DR condition) | measure | trending down | IR records |
Process Metrics (leading)
- Scenario library from TA-Processes refreshed quarterly; scenario content aligned to current TA-Processes per-workflow models and latest Annex III guidance.
- FRIA workshop calendar maintained; Annex III screen runs as part of triage before DR routing.
- Drift-detection tooling health monitored, staleness alert if a Critical workflow has no drift check in the last 90 days.
- Cross-domain coordination channel with DR-Software established; joint-review calendar maintained.
Effectiveness Metrics (business value)
- Annex III / Art. 22 regulatory findings caught at design vs. post-deployment audit, FRIA workshop effectiveness measured.
- Scenario-driven reviews produce more specific approve-with-conditions lists; conditions are more actionable than checklist items.
- Joint DR-Processes / DR-Software reviews reduce handoff gaps for Critical-tier workflows wrapping internally built AI features.
Success Criteria
- 100% of Critical-tier DR reviews conducted as scenario-based walkthroughs with the decision tied to how the design handles each scenario.
- 100% of Annex III workflows with a completed FRIA workshop before go-live; FRIA artifacts linked from DR records.
- Design-drift detection operating for Critical (quarterly) and High (annual) workflows; 100% of material drifts re-routed to DR.
- Joint DR-Processes / DR-Software review records on file for 100% of Critical-tier workflows wrapping first-party AI features.
- IR-stage design surprises measurably fewer than at L1 over consecutive quarters.
Maturity Level 3
Objective: Operate continuous design attestation from workflow-execution telemetry, automate drift-triggered DR exception tickets, and contribute review rubrics and scenario templates to OECD AI, ISO/IEC 42005, and sector standards bodies
At this level, Critical-tier workflows attest continuously rather than being reviewed periodically. Workflow-execution telemetry (HITL queue throughput, decision-distribution signals, Art. 50 disclosure presence rates, override-audit completeness) produces a daily attestation signal confirming the workflow's design posture. Pattern drift opens a DR-exception ticket automatically. Review rubrics, scenario templates, and FRIA workshop frameworks are contributed to OECD AI, ISO/IEC 42005, and applicable sector bodies.
Dependencies
- DR-Processes L2 (required): scenario reviews, FRIA workshops, drift detection, and joint-review process must be established before automation is trustworthy.
- SA-Processes L3 (required): externalized patterns supply the attestation frame; automated SA-pattern-compliance checks verify against the published pattern.
- ML-Processes L2+ (required): workflow-execution monitoring signals (HITL throughput, decision distribution, disclosure presence rates) feed the continuous attestation pipeline.
- IM-Processes L2+ (required): incidents auto-trigger DR re-examination; the IM → DR feedback loop must be operational before L3 automation is meaningful.
Desired Outcomes
- Critical-tier workflows' design posture is readable from a daily attestation signal, reviewers handle exceptions and novel workflow designs, not routine checks.
- Pattern evolution is driven quarterly by external signals (OECD AI guidance, ISO/IEC 42005 updates, sector regulatory guidance) and internal signals (IM-Processes incidents, ML-Processes telemetry) with a traceable change log.
- Review rubrics, scenario templates, and FRIA workshop frameworks are published externally and adopted by peer organizations; the program contributes to the AI-assurance workflow-governance ecosystem.
- DR review backlog shrinks to exception and novel-workflow work.
Activities
A) Continuous design attestation via workflow-execution telemetry
Each Critical-tier workflow produces a daily attestation signal covering:
- HITL gate health: HITL queue telemetry confirms reviewer SLA is being met (not saturated, not rubber-stamping, override-rationale field populated at declared rate); alerts when queue throughput implies SLA miss or when override-rationale completion drops below threshold.
- Decision-logging completeness: decision-logging pipeline confirms required fields are flowing at expected volume; stale or silent logs open a finding.
- Art. 50 disclosure presence: product-flow analytics confirm the disclosure element fires on every AI-touched interaction (not only on sampled sessions); A/B-test variant drift alerts if disclosure disappears from a traffic slice.
- Override audit freshness: override-audit log confirms entries are being written with required fields (identity, rationale, timestamp) at the rate implied by HITL queue throughput; silence or schema drift opens a finding.
- Fallback / kill-switch readiness: synthetic test of the fallback or kill-switch path runs on the published cadence; failure to complete opens a finding.
Deviations automatically open a DR-exception ticket in IM-Processes; the ticket is triaged within 3 business days. Attestation artifacts are machine-readable and regulator-consumable, EU AI Act Art. 9 risk-management evidence, Art. 26 deployer-duty documentation, and ISO/IEC 42001 AIMS operational records are produced by the attestation pipeline without manual assembly.
B) Contribute review rubrics and scenario templates to industry
Publish under Apache 2.0 or equivalent through OECD AI Policy Observatory, ISO/IEC 42005 working groups, applicable sector bodies (financial-services AI supervisory guidance, healthcare AI governance bodies), or OWASP SAMM AI extensions:
- Per-archetype AI-Embedded Workflow Design Review Rubric (tier-assignment criteria, checklist items with evidence pointers, scenario-selection guidance, FRIA trigger indicators).
- FRIA workshop framework (agenda template, rights-impact mapping tool, mitigation-design guide, residual-exposure documentation format).
- Scenario template library (scenario format, per-archetype examples including Art. 22 challenge scenarios, debrief rubric for reviewer calibration exercises).
- Pattern-evolution framework (how external signals, OECD AI guidance, ISO/IEC 42005 updates, sector regulatory guidance, IM-Processes incidents, feed DR checklist and scenario updates on a quarterly cadence).
Internal rubrics and templates remain aligned to the published external versions; internal deviations are proposed as upstream changes, not silently forked.
C) Pattern evolution driven by external and internal signals
Quarterly pattern-evolution review: external signals (OECD AI guidance on high-risk AI systems and HITL requirements; ISO/IEC 42005 AI impact assessment standard updates; sector regulatory AI guidance; Annex III expansion or amendment updates) plus internal signals (IM-Processes incident patterns by archetype, ML-Processes telemetry anomalies, ST-Processes red-team findings) feed structured checklist and scenario library updates.
Updates change-logged with signal provenance; downstream DR records for in-flight reviews notified of pattern changes affecting their archetype. Where a new regulatory update or IM incident reveals a checklist gap, the gap is propagated to SA-Processes and SR-Processes as well, the traceability chain from threat or compliance obligation to requirement to design review is maintained.
Outcome Metrics (L3)
| Metric | Baseline | L3 Target | Source |
|---|---|---|---|
| % Critical-tier workflows producing a daily attestation signal | measure | ≥90% | Attestation telemetry |
| Mean DR-exception ticket age from open to triage | measure | ≤3 business days | DR-exception queue |
| Industry contributions per year (rubrics, FRIA frameworks, scenario templates) | 0 | ≥2 | Contribution log |
| Review backlog age, non-exception items | measure | ≤7 days | Review queue telemetry |
| Quarterly pattern-evolution reviews conducted | measure | 4 / year | Pattern-update log |
Process Metrics (leading)
- Attestation-signal health, % Critical workflows producing a fresh attestation signal in the last 24 hours; staleness alert if any Critical workflow silent for >48 hours.
- External-signal ingestion cadence, OECD AI, ISO/IEC 42005, and sector feeds processed monthly into the pattern-update queue.
- Contribution pipeline, at least one artifact in draft, in-review, or published at any time.
- Exception-queue freshness, DR-exception tickets triaged within 3 business days of opening.
Effectiveness Metrics (business value)
- Reviewer-hours per workflow trending down quarter-over-quarter as continuous attestation absorbs routine design-check work.
- External adoption of published rubrics, FRIA frameworks, and scenario templates, citations from peer organizations, sector bodies, or regulators signal industry recognition.
- Critical-incident MTTR shortened because workflow design posture is continuously visible; incident responders do not need to reconstruct the approved design from scratch during an incident.
Success Criteria
- Daily attestation operating for ≥90% of Critical-tier workflows; DR-exception tickets opened on deviation and triaged within 3 business days.
- ≥2 externally contributed review artifacts per year (per-archetype rubrics, FRIA workshop frameworks, scenario templates) with documented adoption.
- Review backlog for non-exception work inside ≤7 days; attestation has absorbed the pre-L3 routine review volume.
- Quarterly pattern-evolution cadence traceable to external (OECD AI, ISO/IEC 42005, sector guidance, Annex III updates) and internal (IM-Processes, ML-Processes, ST-Processes) signals with a versioned change log.
Key Success Indicators
Level 1: - Per-archetype AI-Embedded Workflow Design Checklists published and versioned for all seven archetypes (decision pipeline, customer-facing flow, human-AI collaboration chain, back-office augmentation, approval/review workflow, content-generation workflow, knowledge-management workflow), each traceable to the applicable SA pattern, SR requirements pack, and TA threat snapshot. - Two-lane review model operational (fast-lane ≤2 BD, full-lane ≤5 BD) with named lead reviewers per archetype trained on EG-Processes L1; Privacy and Legal participation confirmed for full-lane reviews. - ≥95% of AI-embedded workflows going live in the last 90 days carry a completed DR decision record; 100% of Annex III / Art. 22 workflows include FRIA / DPIA status with named owner. - SA pattern-update and SR pack-update triggers wired; every IM-Processes incident re-examines the DR record that approved the affected workflow.
Level 2: - 100% of Critical-tier DR reviews conducted as scenario-based walkthroughs with 3–5 scenarios sourced from TA-Processes per-workflow deep models and anonymized IM-Processes incidents; the DR decision is tied to how the design handles each scenario. - 100% of Annex III workflows with a completed FRIA workshop before go-live; FRIA artifacts linked from DR records and SM-Processes inventory. - Design-drift detection operating quarterly for Critical and annually for High; 100% of material drifts re-routed to DR. - Joint DR-Processes / DR-Software records on file for 100% of Critical-tier workflows wrapping first-party AI features.
Level 3: - ≥90% of Critical-tier workflows producing a daily attestation signal across HITL gate health, decision-logging completeness, Art. 50 disclosure presence, override-audit freshness, and fallback readiness; deviations auto-open DR-exception tickets triaged within 3 business days. - ≥2 externally contributed review artifacts per year published to OECD AI, ISO/IEC 42005, or sector bodies with documented adoption; internal practice aligned to published versions. - Quarterly pattern-evolution loop traceable to OECD AI, ISO/IEC 42005, sector regulatory guidance, Annex III updates, IM-Processes incidents, ML-Processes telemetry, and ST-Processes findings; versioned change log maintained.
Common Pitfalls
Level 1: - ❌ Design review runs after the workflow has already gone live, the checkpoint loses leverage because affected persons have already received AI-assisted decisions; the review becomes a retrospective audit, not a gate. - ❌ HITL gate is declared in the checklist but reviewer capacity is never modeled, the declared SLA is arithmetically impossible; reviewers rubber-stamp every AI recommendation to keep up with volume. - ❌ Art. 50 disclosure is assumed to be handled by a downstream team, the disclosure design is absent from the DR record; no one verified the UX shows it on every AI-touched interaction. - ❌ Annex III screen is skipped because "this doesn't feel like high-risk AI", a decision pipeline affecting employment, credit, or education outcomes goes live without a FRIA; regulatory exposure is undocumented. - ❌ Approve-with-conditions is issued but conditions (HITL gate, override audit trail, contestation path) have no named owner and no expiry date, conditions sit unresolved at go-live. - ❌ DR decision records are not linked to the SM-Processes inventory record, the program cannot answer "was this workflow reviewed?" without a manual search. - ❌ Cross-domain handoff with DR-Software is never established, a workflow wrapping a first-party LLM feature has a DR-Processes record that assumes the AI feature is correctly built, with no DR-Software record confirming that.
Level 2: - ❌ "Scenario-based" review is the same checklist read aloud in a meeting, the scenario-to-design-control mapping is never performed; no one asks "what happens to the affected person if the AI recommendation is wrong and the reviewer doesn't catch it?" - ❌ FRIA workshop is a template filled in by the Legal team without the business owner or affected-person proxy, rights impacts are theorized rather than grounded in the specific workflow; mitigations are generic. - ❌ Design-drift detection runs on a schedule but findings dead-end in a spreadsheet, no DR-exception ticket is opened; the approved design remains fiction while the live workflow has diverged. - ❌ Joint DR-Processes / DR-Software reviews never happen because the coordination channel with DR-Software was never established, the handoff boundary between the workflow and the embedded AI feature is undocumented. - ❌ Per-tier drift-detection cadence exists on paper but the workflow-config sources (BPM tool change events, product-flow analytics) were never wired.
Level 3: - ❌ Attestation signals show green across all Critical workflows but the underlying checks cover only decision-logging volume, HITL gate health, override-rationale completion, and Art. 50 disclosure presence are not checked; attestation is cosmetic. - ❌ Externally published rubrics diverge from internal practice, the published FRIA framework reflects how the org assessed workflows 18 months ago; peer adopters find inconsistencies when comparing the framework to actual DR records. - ❌ Exception queue overwhelms reviewers because attestation thresholds are too sensitive, every reviewer-queue fluctuation opens a DR-exception ticket; reviewers suppress the signal source to stop the noise rather than tune the sensitivity threshold. - ❌ Industry contributions are conference talks and blog posts describing the program, no technical artifacts (rubrics, FRIA frameworks, scenario templates) land in OECD AI / ISO/IEC 42005 / sector bodies with documented adoption.
Practice Maturity Questions
Level 1: 1. Is there a published, versioned per-archetype AI-Embedded Workflow Design Checklist for all seven archetypes, each covering the common spine (HITL placement and depth, Art. 50 disclosure design, decision logging, override audit trail, output-integrity SLA, reviewer-capacity gating, affected-persons rights surface, DPIA/FRIA status, fallback/kill-switch) plus archetype-specific items, and traceable to the applicable SA pattern, SR requirements pack, and TA threat snapshot? 2. Do ≥95% of AI-embedded workflows going live in the last 90 days carry a completed DR decision record, with a two-lane routing model (fast-lane ≤2 BD, full-lane ≤5 BD), named lead reviewers per archetype trained on EG-Processes L1, Privacy and Legal participation for full-lane reviews, and 100% of Annex III / Art. 22 workflows including FRIA / DPIA status with named owner? 3. Are recurring pattern deviations and repeatedly-waived SR requirements automatically queuing SA pattern-update and SR pack-update reviews, and does every IM-Processes incident trigger a re-examination of the DR record that approved the affected workflow?
Level 2: 1. Are 100% of Critical-tier DR reviews conducted as scenario-based walkthroughs, with 3–5 specific threat or compliance scenarios sourced from TA-Processes per-workflow deep models and anonymized IM-Processes incidents, with the DR decision tied explicitly to how the proposed workflow design handles each scenario rather than checklist conformance alone? 2. Do 100% of Annex III workflows receive a completed FRIA workshop before go-live, with the workshop output linked from the DR decision record and the SM-Processes inventory, and is design-drift detection running quarterly for Critical-tier and annually for High-tier, with 100% of material drifts re-routed to DR? 3. Are joint DR-Processes / DR-Software review records on file for 100% of Critical-tier workflows wrapping first-party AI features, with an explicit handoff boundary and shared residual-risk ownership documented in both DR records?
Level 3: 1. Are ≥90% of Critical-tier AI-embedded workflows producing a daily attestation signal, covering HITL gate health, decision-logging completeness, Art. 50 disclosure presence, override-audit freshness, and fallback readiness, with deviations auto-opening DR-exception tickets triaged within 3 business days? 2. Has the program contributed ≥2 substantive review artifacts per year (per-archetype rubrics, FRIA workshop frameworks, scenario templates) to OECD AI, ISO/IEC 42005, or applicable sector bodies, with documented adoption and internal practice aligned to the published versions? 3. Is there a quarterly pattern-evolution review driven by external signals (OECD AI guidance, ISO/IEC 42005 updates, Annex III changes, sector regulatory AI guidance) and internal signals (IM-Processes incidents, ML-Processes telemetry, ST-Processes findings), with a versioned change log and notification to in-flight DR reviews affected by pattern changes?
Document Version: HAIAMM v3.0 Practice: Design Review (DR) Domain: Processes Last Updated: 2026-05-14 Author: Verifhai
☑ Interactive Self-Assessment
Answer each question based on your current, implemented practices only. Progress saves automatically in your browser.