Monitoring & Logging (ML)

Data Domain - HAIAMM v3.5

Generated from the canonical cell records in cells/. Do not edit by hand; edit cells/ML-Data-L*.cell.json and re-run tools/render.py.

---

Practice Overview

Business Function: Operations

Objective: Establish a logging baseline per data archetype and a detection set covering the top threats, with retention set by the longest applicable obligation.

Description: A program at this level can see what its AI is doing. Each archetype has a logging baseline covering corpus access with requester identity and purpose, retrieval queries and the documents returned, training and fine-tuning run records with their input corpora, embedding generation and re-index events, erasure operations and their propagation, and canary access events, and a detection set targeting the threats the threat library rates highest. Retention is set by the longest applicable regulatory obligation rather than by storage cost, because the retention decision is usually made on cost grounds by people who do not know the obligation and is discovered to be wrong during an investigation.

---

Maturity Level 1, Foundational

Objective: Establish a logging baseline per data archetype and a detection set covering the top threats, with retention set by the longest applicable obligation.

A program at this level can see what its AI is doing. Each archetype has a logging baseline covering corpus access with requester identity and purpose, retrieval queries and the documents returned, training and fine-tuning run records with their input corpora, embedding generation and re-index events, erasure operations and their propagation, and canary access events, and a detection set targeting the threats the threat library rates highest. Retention is set by the longest applicable regulatory obligation rather than by storage cost, because the retention decision is usually made on cost grounds by people who do not know the obligation and is discovered to be wrong during an investigation.

Dependencies

• Required: SM-Data-L1, PC-Data-L1, TA-Data-L1
• Alignment (not blocking): Existing logging and detection platform, so AI telemetry lands in the established pipeline rather than a separate one
• Supports / unblocks: IM-Data-L1; IR-Data-L2; ST-Data-L2; EH-Data-L2; Required by: IM-Data-L2, ML-Data-L2

Desired Outcomes

• Every data archetype has a logging baseline that is emitting rather than configured.
• A detection set covers the threats the library rates highest, with coverage measured rather than assumed.
• Retention is set by the longest applicable obligation rather than by storage cost.
• Logs carry enough identity and context to answer who, what and which model without a second investigation.
• Detections route to the shared issue backlog rather than to a dashboard nobody owns.

Activities

A) Publish and verify the logging baseline per archetype

Define a logging baseline per archetype covering corpus access with requester identity and purpose, retrieval queries and the documents returned, training and fine-tuning run records with their input corpora, embedding generation and re-index events, erasure operations and their propagation, and canary access events. Verify that each stream is actually emitting rather than merely configured, by asserting the presence of recent events per stream per asset, because a logging configuration that exists and emits nothing passes every configuration check and fails the one investigation that needs it. Include the identity and context fields that make a log answerable: who initiated, what asset, which model and version, which session. A log that records that something happened without recording which model version did it cannot answer the question that follows any output-integrity incident. Publish the baseline alongside the hardening baseline so telemetry travels with configuration.

B) Build the detection set from the threat library

Build detections for the threats the TA library rates highest, covering in this domain a canary appearing outside its governed store, retrieval volume or breadth departing from a requester's baseline, an unregistered corpus entering a training run, erasure that did not propagate within its window, and retrieval cost per query departing from the store median. Map each detection to the threat identifiers it addresses and publish the threats with no detection, because coverage is the claim that matters and detection count is not a proxy for it. Route every detection to the shared issue backlog with a severity from the IM taxonomy rather than to a dashboard, since a detection whose only destination is a dashboard has no owner and no clock. Include at least one consumption-anomaly detection from the outset, expressed against a denominated metric, because it is the cheapest detection available and the only one that covers denial-of-wallet.

C) Set retention by obligation and make logs usable

Set retention per log class by the longest applicable regulatory, contractual or investigative requirement, and record which obligation drives each period so the decision is reviewable. Retention is routinely set on storage cost by people who do not hold the obligation, and the error surfaces during an investigation or a regulator request when it can no longer be corrected. Apply redaction at write rather than at read so that sensitive content is not retained in the clear pending a query-time control. Verify that logs are actually queryable at the volumes an investigation would need, because a retention period that cannot be searched within a working day is not meaningfully retained.

Outcome Metrics (L1)

Metric	Baseline	L1 Target	Source
Data archetypes with a logging baseline defined and verified as emitting	first verification pass	≥85% of archetypes within 9 months	Log stream presence assertions per asset
Top-rated threats from the library covered by at least one detection	first coverage mapping	≥70%, with uncovered threats published as gaps	Threat-to-detection coverage map
Log classes with a retention period traced to a named obligation	first retention review	100%	Retention configuration with obligation references
Detections routed to the shared issue backlog with an IM severity	first routing pass	100% of active detections	Detection routing configuration
Log streams verified as queryable within a working day at investigation volumes	first query exercise	100% of Critical-tier asset log classes	Query exercise records

Process Metrics (leading)

• Log stream emission asserted continuously, with alerting when a stream goes silent.
• Detection coverage map refreshed whenever the threat library changes.
• Retention configuration reviewed whenever an in-scope obligation changes.
• Query exercises run at least semi-annually at realistic investigation volumes.

Effectiveness Metrics (business value)

• Time to answer a basic investigative question about an AI asset falls, measured from request to answer.
• Incidents where the necessary log did not exist fall toward zero.
• Detections that produced an actioned issue rise as a share of total detections.

Oversight Metrics (the HAI control, measured)

Metric	Oversight failure it detects	Source
Detections acknowledged but closed without an investigation note	alert closure as queue management rather than as analysis, which is invisible in any alert-volume metric and is the normal response to a noisy detection set	Detection disposition records
Detections muted or suppressed for more than one week, with the suppressing authority recorded	silent suppression, where a noisy detection is muted to clear a queue and never re-enabled, so coverage falls with no decision taken	Detection suppression log

Consumption Metrics (denominated, never absolute spend)

Metric	Denominator	Economic failure it detects	Source
Consumption-anomaly detections active against a denominated metric	per archetype detection set	no coverage at all for denial-of-wallet, which produces no availability signal and is therefore invisible to every other detection in the set	Detection registry filtered to consumption anomalies
Detection threshold expressed against consumption per unit of work rather than absolute spend	per retrieval query	thresholds set on absolute spend, which fire on ordinary growth and are then widened until they never fire at all	Detection rule definitions
Logging and retention cost per retained record, by log class	per retained record	retention silently shortened for cost, removing the evidence that monitoring, investigation and deployer-duty obligations all depend on	Storage cost allocation joined to the retention configuration

Level of Effort (incremental over the level below)

• Stand-up: 170 hours · Ongoing: 110 hours/quarter
• Security Analyst / SOC — primary, 34 days/year
• DevOps / Platform Engineer — regular, 16 days/year
• AppSec / SecEng — regular, 10 days/year
• Legal / Privacy Counsel — low, 4 days/year
• Material new tooling: AI telemetry ingestion into the existing detection platform

Success Criteria

• A logging baseline is defined per archetype and verified as emitting for at least 85% of data archetypes.
• At least 70% of top-rated library threats are covered by a detection, with uncovered threats published as gaps.
• Every log class has a retention period traced to a named obligation rather than to storage cost.
• Every active detection routes to the shared issue backlog with an IM severity rather than to a dashboard.
• At least one consumption-anomaly detection operates against a denominated metric from the outset.

---

Maturity Level 2, Comprehensive

Objective: Calibrate data detection and logging depth to asset tier, tune for alert quality, and establish behavioural baselines so departure is detectable.

At this level monitoring depth follows tier and detections are tuned rather than merely deployed. Behavioural baselines per asset make departure detectable, which is the only practical way to catch the AI failure modes that look like normal operation, since an agent doing the wrong thing competently produces traffic indistinguishable from an agent doing the right thing. Alert quality is measured with precision and recall reported together, because precision alone rewards a detector that fires rarely and catches nothing.

Dependencies

• Required: ML-Data-L1, SM-Data-L2, TA-Data-L2, IM-Data-L2
• Alignment (not blocking): Detection engineering practice, so AI detections are tuned on the same rails as the rest of the estate
• Supports / unblocks: IR-Data-L2; ST-Data-L2; EH-Data-L2; IM-Data-L3; Required by: EG-Data-L3, EH-Data-L3, IR-Data-L3, ML-Data-L3, PC-Data-L3, SA-Data-L3, SM-Data-L3, ST-Data-L3, TA-Data-L3

Desired Outcomes

• Logging and detection depth follow asset tier, and the difference is published.
• Behavioural baselines exist per Critical-tier asset, so departure from normal is detectable.
• Alert quality is measured with precision and recall reported together rather than precision alone.
• Detection coverage is measured against the threat library rather than counted.
• A silent log stream or a stopped detection is detected in hours rather than at the next review.

Activities

A) Calibrate logging and detection depth to tier

Publish what each tier receives: Critical gets full logging, the complete detection set tuned for the asset, behavioural baselining and the shortest alerting path; Low gets baseline logging and the shadow-AI detections. Derive the applicable set from the inventory tier automatically. Publish the retention and sampling position per tier explicitly, because sampling is where cost pressure lands first and a Critical-tier asset whose telemetry is sampled has monitoring that is statistical at exactly the tier where specific answers are needed. Treat a sampling or retention reduction at Critical tier as a material change requiring approval rather than an operational tuning decision. Behavioural baselining in this domain covers retrieval breadth and volume per requester, because extraction looks like enthusiastic legitimate use until compared against a baseline.

B) Establish behavioural baselines per Critical-tier asset

Build a behavioural baseline per Critical-tier asset covering its normal operating envelope: request and tool-call volumes, the distribution of actions taken, typical session shapes, identity and access patterns, and consumption per unit of work. Alert on departure rather than only on signature, because the AI failure modes that matter most produce traffic that is individually well-formed and collectively abnormal, and a signature-based set cannot see an agent competently doing something nobody asked for. Refresh baselines on a defined cadence and after any material change to the asset, and record baseline shifts explicitly, since an unexplained baseline shift is itself the signal and is lost if baselines are refreshed silently. Detection-as-code in this domain is fragile to retrieval and embedding schema changes, which silently break a rule that continues to report healthy.

C) Tune for alert quality and report precision with recall

Measure detection quality using analyst disposition labels, and report precision and recall together. Precision alone rewards a detector that fires twice a year and misses everything, which is exactly the detector a tuning programme under alert-fatigue pressure will produce. Set a false-positive budget per detection and tune or retire detections that exceed it, because analysts who learn that a detection is usually wrong stop reading it and the coverage is then notional. Assert continuously that each detection is still evaluating and each stream still emitting, alerting within hours, since a stopped detection and a clean environment are indistinguishable in every dashboard. Data detection content contributes well as extraction and canary-trigger rules, which are expensive to develop and widely applicable.

Outcome Metrics (L2)

Metric	Baseline	L2 Target	Source
Assets receiving the logging and detection depth derived from their tier	first tier-derivation pass	100% of Critical and High tier assets	Monitoring configuration joined to inventory tier
Critical-tier assets with a current behavioural baseline	first baselining round	≥90%, refreshed on cadence and after material change	Baseline registry with refresh dates
Detection precision and recall both reported per detection	first tuning cycle	100% of active detections report both	Detection quality reporting with analyst disposition labels
Detections exceeding their false-positive budget, tuned or retired	first tuning cycle	100% actioned within the quarter	Detection tuning records
Time to detect a silent log stream or stopped detection	first health monitoring cycle	≤4 hours	Telemetry health monitoring alerts
Sampling or retention reductions at Critical tier approved as material changes	first quarter of change telemetry	100%	Change records filtered to telemetry configuration

Process Metrics (leading)

• Monitoring configuration re-derived automatically when an asset's tier changes.
• Behavioural baselines refreshed on cadence and after material change, with shifts recorded.
• Detection tuning cycle run quarterly using the previous quarter's disposition labels.
• Telemetry health asserted continuously with alerting within hours.

Effectiveness Metrics (business value)

• Analyst time per actioned detection falls as tuning improves precision without sacrificing recall.
• Incidents detected by monitoring rather than reported externally rise as a share of total.
• Mean time to detect falls, reported per threat class rather than in aggregate.
• Monitoring effort concentrates on Critical and High tiers relative to L1.

Oversight Metrics (the HAI control, measured)

Metric	Oversight failure it detects	Source
Alert disposition distribution per analyst, compared across analysts on comparable queues	disposition drift, where the same alert would be escalated by one analyst and closed by another, making detection outcome depend on shift rather than on evidence	Detection disposition records segmented by analyst
Alerts per analyst per shift against the published ceiling	analyst overload, where triage quality falls as volume rises while closure rates continue to look healthy	Alert assignment telemetry
Critical-tier detections closed as false positive and later implicated in an incident	dismissal under load of exactly the alerts that mattered, visible only by tracing closures forward	Detection closures joined to incident root causes

Consumption Metrics (denominated, never absolute spend)

Metric	Denominator	Economic failure it detects	Source
Consumption-anomaly detections operating against a per-asset behavioural baseline	per retrieval query	consumption thresholds set globally rather than per asset, which either miss a small asset's drain entirely or drown a large asset's normal operation in alerts	Detection rule definitions joined to the baseline registry
Time from a consumption anomaly beginning to its detection	per consumption anomaly	slow detection on the one incident class where the loss accrues continuously and nothing else in the monitoring set will notice	Consumption ledger joined to detection timestamps
Telemetry cost per retained record at Critical tier, tracked against sampling decisions	per retained record	sampling introduced at Critical tier for cost reasons, which makes monitoring statistical at exactly the tier where specific answers are required	Telemetry cost allocation joined to sampling configuration

Level of Effort (incremental over the level below)

• Stand-up: 190 hours · Ongoing: 180 hours/quarter
• Security Analyst / SOC — primary, 40 days/year
• DevOps / Platform Engineer — regular, 18 days/year
• AppSec / SecEng — regular, 12 days/year
• AI/ML Engineer — low, 8 days/year
• Material new tooling: Behavioural baselining across AI telemetry, Detection quality measurement with disposition labelling

Success Criteria

• Logging and detection depth are published per tier and derived automatically from the inventory.
• At least 90% of Critical-tier assets have a current behavioural baseline, refreshed on cadence and after material change, with shifts recorded.
• Every active detection reports precision and recall together, with a false-positive budget enforced by tuning or retirement.
• A silent log stream or stopped detection is detected within 4 hours.
• Sampling or retention reductions at Critical tier are approved as material changes rather than treated as operational tuning.

---

Maturity Level 3, Industry-Leading

Objective: Engineer data detections as versioned, tested code, close the loop from incident to detection continuously, and contribute detection content back to the field.

At this level detections are engineered rather than configured. Each is versioned, tested against known-positive and known-negative cases, and deployed through a pipeline, so a detection that has silently stopped matching is caught by its own test rather than by an incident. The loop from incident to detection runs continuously. Detection content is contributed back, which matters in AI security because shared detection content is scarce and the failure modes are common across organisations, so one organisation's detection genuinely improves another's floor.

Dependencies

• Required: ML-Data-L2, TA-Data-L3, IM-Data-L3
• Alignment (not blocking): Detection engineering tooling and pipeline ownership
• Supports / unblocks: Every other Data practice at L3, which depends on the observability this cell sustains

Desired Outcomes

• Detections are versioned, tested and deployed through a pipeline rather than configured in a console.
• Every detection carries known-positive and known-negative test cases that run continuously.
• Incidents and adversarial findings generate detections within days rather than at the next review.
• Detection content is contributed back in deployable form.
• Coverage against the threat library is measured continuously and reported with its gaps.

Activities

A) Engineer detections as versioned, tested code

Move detection content into version control with the same discipline as application code: each detection carries an owner, a version, a documented intent, and test cases that assert it fires on known-positive events and does not fire on known-negative ones. Run the tests continuously against replayed or synthetic events, because the most common silent failure in monitoring is a detection that still exists, still evaluates and no longer matches anything due to a log-format change upstream, and nothing in a conventional monitoring stack reports that. Deploy through a pipeline with review, so a detection change is as traceable as a code change and can be reverted as easily. Behavioural baselining in this domain covers retrieval breadth and volume per requester, because extraction looks like enthusiastic legitimate use until compared against a baseline. Detection tests in this domain replay retrieval and canary-access events, and the known-negative cases matter most because legitimate heavy retrieval is common.

B) Close the incident-to-detection loop continuously

Require every incident and every adversarial finding to be assessed for detection coverage within days: did a detection fire, should one have, and if so why did it not. Where no detection existed, author one and deploy it within 10 business days, with its test cases. Where one existed and did not fire, treat that as a detection defect with an owner rather than as an incident detail. Track the share of incidents detected internally before external report as the practice's headline measure, since it is the one number that expresses whether monitoring is working, and report it alongside coverage rather than instead of it. Detection-as-code in this domain is fragile to retrieval and embedding schema changes, which silently break a rule that continues to report healthy.

C) Contribute detection content back

Contribute at least four substantive artifacts per year in deployable form: detection rules, behavioural baselining approaches, log schemas for AI telemetry and the test cases that verify a detection works. Shared detection content is scarce in AI security while the failure modes are largely common across organisations, which makes this among the highest-leverage contributions available, since a peer can deploy a rule and cannot deploy a description of one. Work through OWASP projects, MITRE ATLAS, sector information-sharing bodies and detection-engineering communities. Anonymise, review legally, coordinate disclosure where content reveals a provider weakness, and prioritise contributing the detections that most often produce actioned findings internally. Data detection content contributes well as extraction and canary-trigger rules, which are expensive to develop and widely applicable.

Outcome Metrics (L3)

Metric	Baseline	L3 Target	Source
Detections held in version control with an owner, intent and test cases	L2 detection set	≥85% of active detections	Detection repository
Detections whose tests run continuously against replayed or synthetic events	first test rollout	≥85%, with failures alerting within the day	Detection test pipeline results
Incidents and adversarial findings assessed for detection coverage within 5 business days	first quarter of loop telemetry	100%	Incident and finding records with coverage assessment
New detections authored and deployed within 10 business days of an uncovered incident	first quarter of loop telemetry	≥90%	Detection repository change log joined to incident records
Incidents detected internally before external report	first quarter of outcome measurement	≥85% and rising, reported as the headline measure	Incident records with detection source
Accepted external detection-content contributions per year	0	≥4 accepted in deployable form	Contribution log with acceptance references

Process Metrics (leading)

• Detection tests run continuously, with a failing test alerting within the day.
• Incident-to-detection assessment completed within 5 business days for every incident.
• Threat-library coverage recomputed automatically whenever detections or the library change.
• Contribution pipeline carries at least two deployable artifacts in draft or review.

Effectiveness Metrics (business value)

• Internal detection share rises, which is the clearest statement that monitoring works.
• Mean time to detect falls per threat class.
• Detections silently broken by upstream log changes fall toward zero as tests catch them.
• External adoption of contributed detection content by peer organisations.

Oversight Metrics (the HAI control, measured)

Metric	Oversight failure it detects	Source
Detection changes deployed without review, by severity of the detection	detection content changed directly in the console, which is untraceable, unrevertable and the most common way coverage is lost without a decision	Detection deployment pipeline records
Detections whose tests have been failing for more than one week without action	test failure normalised, which restores exactly the silent-failure mode the tests were introduced to remove	Detection test pipeline history
Incidents detected externally rather than internally, with a coverage assessment completed	external detection accepted without asking why internal detection missed it, which forfeits the single most informative signal monitoring produces	Incident records with detection source and coverage assessment

Consumption Metrics (denominated, never absolute spend)

Metric	Denominator	Economic failure it detects	Source
Consumption-anomaly detections held as tested code with known-positive cases	per consumption detection	a consumption detection that silently stopped matching after a billing or telemetry schema change, leaving denial-of-wallet uncovered while the detection appears active	Detection repository filtered to consumption rules
Time from consumption anomaly onset to internal detection	per retrieval query	detection latency on the incident class where loss accrues continuously, which is the only class where detection time and loss are the same quantity	Consumption ledger joined to detection timestamps
Telemetry and detection platform cost per monitored asset	per monitored asset	monitoring cost growing faster than the estate, which eventually triggers a sampling or retention cut that removes coverage at the tiers that need it most	Platform cost allocation joined to the asset inventory

Level of Effort (incremental over the level below)

• Stand-up: 230 hours · Ongoing: 170 hours/quarter
• Security Analyst / SOC — primary, 42 days/year
• DevOps / Platform Engineer — regular, 22 days/year
• AppSec / SecEng — regular, 12 days/year
• AI/ML Engineer — low, 6 days/year
• Material new tooling: Detection-as-code pipeline with continuous test execution

Success Criteria

• At least 85% of active detections are held in version control with an owner, documented intent and test cases.
• Detection tests run continuously against replayed or synthetic events, with failures alerting within the day.
• Every incident and adversarial finding is assessed for detection coverage within 5 business days, and uncovered cases produce a deployed detection within 10.
• Internal detection share is reported as the practice's headline measure alongside coverage.
• At least four accepted external contributions per year in deployable form.

---

Key Success Indicators

Level 1:

• A logging baseline is defined per archetype and verified as emitting for at least 85% of data archetypes.
• At least 70% of top-rated library threats are covered by a detection, with uncovered threats published as gaps.
• Every log class has a retention period traced to a named obligation rather than to storage cost.
• Every active detection routes to the shared issue backlog with an IM severity rather than to a dashboard.
• At least one consumption-anomaly detection operates against a denominated metric from the outset.

Level 2:

• Logging and detection depth are published per tier and derived automatically from the inventory.
• At least 90% of Critical-tier assets have a current behavioural baseline, refreshed on cadence and after material change, with shifts recorded.
• Every active detection reports precision and recall together, with a false-positive budget enforced by tuning or retirement.
• A silent log stream or stopped detection is detected within 4 hours.
• Sampling or retention reductions at Critical tier are approved as material changes rather than treated as operational tuning.

Level 3:

• At least 85% of active detections are held in version control with an owner, documented intent and test cases.
• Detection tests run continuously against replayed or synthetic events, with failures alerting within the day.
• Every incident and adversarial finding is assessed for detection coverage within 5 business days, and uncovered cases produce a deployed detection within 10.
• Internal detection share is reported as the practice's headline measure alongside coverage.
• At least four accepted external contributions per year in deployable form.

---

Common Pitfalls

Level 1:

• ❌ Verifying that logging is configured rather than that it is emitting, which passes a stream that has been silent for months.
• ❌ Omitting model and version from the log context, which makes every output-integrity investigation start from scratch.
• ❌ Setting retention on storage cost, an error discovered during an investigation when it can no longer be corrected.
• ❌ Routing detections to a dashboard, where they have no owner and no clock.
• ❌ Building a detection set with no consumption anomaly, leaving the one attack class that produces no availability signal entirely uncovered.

Level 2:

• ❌ Reporting precision without recall, which rewards a detector that fires twice a year and catches nothing.
• ❌ Relying on signature detections alone, which cannot see an agent competently doing something nobody asked for.
• ❌ Refreshing behavioural baselines silently, which discards the baseline shift that was itself the signal.
• ❌ Allowing sampling at Critical tier as an operational tuning decision, making monitoring statistical where specific answers are needed.
• ❌ Assuming a quiet detection is a clean environment, when a stopped detection looks identical.

Level 3:

• ❌ Leaving detections in a console rather than version control, which makes changes untraceable and coverage loss undecided.
• ❌ Deploying detections without test cases, which leaves the silent-failure mode after an upstream log change entirely uncovered.
• ❌ Normalising failing detection tests, which restores exactly the problem the tests were introduced to remove.
• ❌ Accepting external detection of an incident without asking why internal detection missed it, forfeiting the most informative signal available.
• ❌ Contributing descriptions of detections rather than deployable rules, which peers read and cannot run.

---

Practice Maturity Questions

Level 1:

1. Is a logging baseline defined per data archetype covering identity, asset, model and version context, and verified as actually emitting rather than merely configured, for at least 85% of archetypes?
2. Do detections cover at least 70% of the top-rated threats from the library with uncovered threats published as gaps, and does every detection route to the shared issue backlog with an IM severity rather than to a dashboard? For Data, the assessor should be shown the corpus access log and the canary-trigger detection.
3. Is retention per log class traced to a named obligation rather than to storage cost, with redaction applied at write, and are log streams verified as queryable within a working day at investigation volumes? In Data, ask to see the corpus access log and the canary-trigger detection.

Level 2:

1. Are logging and detection depth published per tier and derived automatically from the inventory, with sampling and retention positions stated per tier and reductions at Critical tier treated as material changes? Evidence this in the Data domain against the corpus access log and the canary-trigger detection.
2. Do at least 90% of Critical-tier assets carry a current behavioural baseline alerting on departure rather than only on signature, with baseline shifts recorded explicitly rather than refreshed silently? For Data, the assessor should be shown the corpus access log and the canary-trigger detection.
3. Is detection quality measured with precision and recall reported together using analyst disposition labels, with a false-positive budget enforced, and is a stopped detection or silent stream detected within 4 hours? In Data, ask to see the corpus access log and the canary-trigger detection.

Level 3:

1. Are at least 85% of active detections held in version control with an owner, documented intent and test cases running continuously against replayed or synthetic events, so a detection broken by an upstream log change is caught by its own test? Evidence this in the Data domain against the corpus access log and the canary-trigger detection.
2. Is every incident and adversarial finding assessed for detection coverage within 5 business days, with uncovered cases producing a deployed detection and its tests within 10, and detections that existed but did not fire treated as defects with owners? For Data, the assessor should be shown the corpus access log and the canary-trigger detection.
3. Is internal detection share reported as the headline outcome measure alongside coverage, and does the organisation contribute at least four accepted detection artifacts per year in deployable rather than descriptive form? In Data, ask to see the corpus access log and the canary-trigger detection.

---

Lightweight Questions (E0 rung)

Answerable yes/no by one person in under 30 seconds, with nothing open.

Level 1:

• Are your AI systems actually emitting logs, or just configured to?
• Do you know why your AI logs are kept for the period they are kept?
• Is there a detection that would fire on unusual AI spend?

Level 2:

• Do you know what normal looks like for your highest-risk AI asset?
• Do you report how often your detections miss, or only how often they are right?
• Would you notice within a day if a detection stopped running?

Level 3:

• Are your detections in version control with tests?
• Would you know if a detection stopped matching because a log format changed?
• Have you published a detection rule others can deploy?

---

Tracking

Level 1 — Jira Epic, labels haiamm-ml-data-l1, haiamm-operations

• Done when: Logging baseline emission verified for at least 85% of archetypes with threat detection coverage at or above 70% and every log class carrying an obligation-traced retention period, sustained two quarters.
• Git artifacts: monitoring/data/, monitoring/detection-registry.json, monitoring/retention-config.json
• Git signal: Stream presence is asserted continuously and alerts when a stream goes silent, and threat-detection-coverage.json is regenerated by joining the detection registry to the threat library.
• CI check: log-stream-presence-check

Level 2 — Jira Epic, labels haiamm-ml-data-l2, haiamm-operations

• Done when: Critical-tier behavioural baseline coverage at or above 90% with every detection reporting precision and recall and telemetry health detection within 4 hours, sustained two consecutive quarters.
• Git artifacts: monitoring/baseline-registry.json, monitoring/detection-quality.json, monitoring/detection-rules/
• Git signal: baseline-registry.json records a refresh date and any shift per asset, and detection-quality.json is regenerated from analyst disposition labels rather than estimated.
• CI check: detection-quality-report

Level 3 — Jira Epic, labels haiamm-ml-data-l3, haiamm-operations

• Done when: Detections in version control with running tests at or above 85% and internal detection share at or above 85%, sustained across two consecutive quarters.
• Git artifacts: monitoring/detections/, monitoring/detection-tests/, governance/contribution-log.json
• Git signal: Detection rules are deployed only through the pipeline, so console changes are absent from the deployment record, and test results are emitted per run rather than per review.
• CI check: detection-as-code-tests

---

Mappings

Level	HAI TTP	ATLAS	OWASP LLM 2025	Compliance
L1	RA	AML.T0018, AML.T0024, AML.T0025, AML.T0029, AML.T0115	LLM02, LLM04, LLM07, LLM08, LLM10	EU AI Act Art. 12; EU AI Act Art. 26; GDPR Art. 32; ISO/IEC 27001 A.8.15; NIST AI RMF MEASURE 2.6
L2	RA	AML.T0018, AML.T0024, AML.T0025, AML.T0029, AML.T0115	LLM02, LLM04, LLM07, LLM08, LLM10	EU AI Act Art. 12; EU AI Act Art. 26; EU AI Act Art. 72; GDPR Art. 32; ISO/IEC 27001 A.8.16; NIST AI RMF MEASURE 2.6
L3	RA	AML.T0018, AML.T0024, AML.T0025, AML.T0029, AML.T0115	LLM02, LLM04, LLM07, LLM08, LLM10	EU AI Act Art. 12; EU AI Act Art. 72; GDPR Art. 32; ISO/IEC 27001 A.8.16; ISO/IEC 42001 9.1; NIST AI RMF MEASURE 2.6

---

Document Version: HAIAMM v3.5 (generated) / Practice: Monitoring & Logging (ML) / Domain: Data