Secure Architecture (SA) - Software Domain

HAIAMM Assessment Questionnaire v3.5

Generated from cells/SA-Software-L*.cell.json. Do not edit by hand. Canonical model: HAIAMM-v3.5-Framing.md · Handbook: HAIAMM-v3.5-Handbook.md.

---

Practice: Secure Architecture (SA) Domain: Software Business Function: Building Purpose: Publish reference architecture patterns and an antipattern catalogue for each software archetype, so the secure design is the default path rather than an achievement. Scoring Model: Evidence + Outcome Metrics (see Scoring Methodology below)

---

Instructions

• Answer from current, implemented practice, not plans or aspirations.
• Each question has two components: Evidence (what you did) and Outcome Metrics (how well it worked).
• Levels are derived, never self-assigned. Record evidence and metric values; the level falls out of the scoring below.
• Silence is not a pass. A metric you cannot produce is left blank and counts as not met. Do not estimate.
• Answer progressively. Complete all Level 1 questions before Level 2.

Evidence grade

Record how each answer is known, not only what is claimed (v3.5 framing §6). A level claim is graded at its weakest field.

Grade	Meaning
E0	Asserted by a practitioner, unevidenced
E1	Asserted, with a link to a named artifact
E2	Emitted by a system as telemetry
E3	Machine-attested and independently checkable

---

Scoring Methodology

Tier	Score	Criteria
Fully Mature	1.0	Evidence complete + ≥3 outcome metrics meet targets
Implemented	0.67	Evidence complete + 2 outcome metrics meet targets
Partial	0.33	Evidence partially complete + <2 outcome metrics meet targets
Not Implemented	0.0	No substantive evidence of practice

Practice maturity level achieved = the highest level where all 3 questions score ≥ 0.67.

---

Maturity Level 1, Foundational

Objective: Publish reference architecture patterns and an antipattern catalogue for each software archetype, so the secure design is the default path rather than an achievement.

Prerequisite cells: SM-Software-L1, TA-Software-L1, SR-Software-L1

Question L1.1: Publish reference patterns per archetype

Q1.1: Does a reference pattern exist for at least 85% of inventoried software archetypes, each stating components, boundaries, controls, the threats it mitigates by TA identifier, the requirements it satisfies and where it does not apply, validated with an engineer who has built that archetype?

Evidence Required:

• [ ] Author one reference pattern per archetype, covering the agent pattern with an explicit tool allowlist and per-tool argument schema validation, the RAG pattern separating instruction from retrieved content, the LLM-feature pattern with output handling for untrusted responses, and the fine-tuning pattern isolating training data from inference
• [ ] Each pattern states the components and their boundaries, the controls at each boundary, the threats it mitigates by identifier from the TA library, the requirements it satisfies from the SR pack, and the conditions under which it does not apply
• [ ] Keep each pattern to a length a team reads before designing rather than after, targeting three pages with a diagram
• [ ] Validate each with an engineer who has built that archetype, because a pattern authored entirely within security describes a system that is secure and unbuildable, and teams respond to that by ignoring it rather than by arguing
• [ ] Publish patterns in the repository where the work happens rather than in a separate architecture space

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L1.2: Publish the antipattern catalogue

Q1.2: Is there an antipattern catalogue of at least eight entries drawn from designs actually seen in review, each stating why the shape is attractive, what goes wrong, and which reference pattern replaces it? For Software, the assessor should be shown the agent reference pattern and the antipattern it replaces.

Evidence Required:

• [ ] Author an antipattern catalogue naming the insecure shapes that actually recur in this domain, being the agent with an implicit tool surface, the RAG pipeline that concatenates retrieved text into the instruction position, and the long-session agent with no memory bound or session boundary
• [ ] For each antipattern, state why it is attractive, since it is always attractive for a reason and a catalogue that does not acknowledge that reads as naive
• [ ] What specifically goes wrong
• [ ] And which reference pattern replaces it
• [ ] The antipattern half of the library is the more useful and the more commonly omitted, because teams recognise the thing they are building far more readily than they recognise the thing they should have built instead
• [ ] Draw the catalogue from real designs seen in review rather than from literature, and add to it whenever review encounters a new recurring shape

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L1.3: Establish the untrusted-content boundary and a design-time self-check

Q1.3: Does every archetype ingesting content the organisation does not author have a documented untrusted-content boundary, and is a design-time self-check published and completed by at least 70% of designs arriving at review? In Software, ask to see the agent reference pattern and the antipattern it replaces.

Evidence Required:

• [ ] Document explicitly where untrusted content enters each archetype and how it is kept separate from instruction, control and identity
• [ ] This is the single most consequential architectural decision in AI systems and the one most often left implicit, because the failure it prevents, content becoming instruction, does not look like a boundary violation in any conventional sense
• [ ] State the pattern for isolation, for provenance marking, and for what the system must never do with content it did not author
• [ ] Publish a short design-time self-check, at most ten questions, that a team can answer before requesting review, which both raises the quality of what arrives at review and tells the team what the review will ask, removing the asymmetry that makes design review feel arbitrary

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 1 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
Software archetypes with a published reference pattern	___	___	≥85% of inventoried archetypes within 9 months	☐
Patterns stating the threats they mitigate by TA identifier and the requirements they satisfy	___	___	100% of published patterns	☐
Antipatterns in the catalogue drawn from designs actually seen in review	___	___	≥8 antipatterns, each traced to an observed design	☐
Archetypes with a documented untrusted-content boundary	___	___	100% of archetypes that ingest content the organisation does not author	☐
Designs arriving at review having completed the design-time self-check	___	___	≥70%	☐

Where each number is read from:

• Software archetypes with a published reference pattern — Pattern library register joined to the asset inventory
• Patterns stating the threats they mitigate by TA identifier and the requirements they satisfy — Pattern library traceability audit
• Antipatterns in the catalogue drawn from designs actually seen in review — Antipattern catalogue with review references
• Archetypes with a documented untrusted-content boundary — Pattern library boundary section audit
• Designs arriving at review having completed the design-time self-check — Design review intake records

Level 1 Oversight Metrics

"A human approves it" is not a control unless you measure whether they refuse. A metric that cannot fail is not a metric.

Metric	Current	Oversight failure it detects	Met?
Share of designs where the reviewer recorded a required change before approval	___	rubber-stamping at design review, where approval is recorded without the review having altered anything, which is the default state of a review with no capacity to say no	☐
Designs approved with a human gate declared on a side-effecting tool call but no stated refusal authority	___	a gate that exists structurally while the person holding it cannot actually refuse, which satisfies the control on paper and not in fact	☐

Level 1 30-second check (E0)

• ☐ Yes ☐ No — Is there a reference architecture for each kind of AI software asset?
• ☐ Yes ☐ No — Can a team tell at design time whether their shape is sanctioned?
• ☐ Yes ☐ No — Is it documented where untrusted content enters your AI systems?

---

Maturity Level 2, Comprehensive

Objective: Calibrate the reference-pattern set by risk tier and make the autonomy controls in each pattern scale with the asset's capability tier.

Prerequisite cells: SA-Software-L1, SR-Software-L2, SM-Software-L2, PC-Software-L2

Question L2.1: Declare tier applicability across the pattern library

Q2.1: Is every pattern in the library published with a tier-applicability row derived from the SM-Software L2 risk rubric, a named accountable owner, and a review date, with no pattern currently past that date?

Evidence Required:

• [ ] For each pattern in the L1 library, publish a tier-applicability row: which of Critical, High, Medium and Low must adopt it, and what a compliant subset looks like for the tiers that do not
• [ ] Derive applicability from the SM-Software L2 tier rubric rather than from author preference, so two reviewers reach the same answer
• [ ] Where a pattern is mandatory only above a threshold, state the threshold in the pattern itself, not in a separate matrix that will drift
• [ ] Record for each pattern a named owner accountable for its currency, and a review date no more than twelve months out
• [ ] Teams building a Medium-tier or Low-tier asset must be able to read the library and know, without a meeting, which patterns are binding on them and which are optional
• [ ] Publish the tier-applicability rows alongside the patterns in the same repository so a single pull request changes both

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L2.2: Scale autonomy controls to the asset's capability tier

Q2.2: Do the patterns governing tool-using assets carry autonomy controls that escalate with capability tier -- approval gates on side-effecting actions, enforced blast-radius scope, and a declared autonomy-step budget -- and can you evidence conformance from configuration rather than self-report?

Evidence Required:

• [ ] Extend every pattern that governs a side-effecting or tool-using asset with an escalating autonomy-control block keyed to the asset's capability tier
• [ ] At minimum: any asset that takes an action with effects outside itself requires an approval gate on that class of action
• [ ] Higher-autonomy assets additionally require an enforced blast-radius scope and a declared autonomy-step budget bounding how many sequential actions may run between human checkpoints
• [ ] The highest tier additionally requires verification by a party independent of the build team
• [ ] Express each requirement as a configuration or IaC assertion a reviewer can check, not as guidance
• [ ] Critically, the approval gate must specify what makes the gate real -- who approves, at what volume, and with what authority to refuse -- because an approval step satisfied by an approver who has never refused is not a control
• [ ] Pair the block with the oversight metrics below so the gate's effectiveness is observed rather than assumed

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L2.3: Run the incident-to-pattern learning loop

Q2.3: Has the incident-to-pattern review run for at least two consecutive quarters, with each Critical or High severity root cause carrying a recorded disposition traceable to a versioned pattern change?

Evidence Required:

• [ ] Establish a standing review, at least quarterly and additionally after any Critical or High severity AI incident, in which the pattern owners walk the period's incidents and findings from IM-Software and ST-Software and classify each root cause into one of three dispositions: no pattern existed and one is now required
• [ ] A pattern existed and was not adopted
• [ ] A pattern existed, was adopted, and failed
• [ ] Each disposition has a different remedy -- author, enforce, or revise -- and each produces a tracked change with an owner and a due date
• [ ] Record the disposition on the incident record itself so the loop is auditable in both directions
• [ ] Version every resulting pattern change and note in the changelog which incident drove it, so a reader can see the library learning
• [ ] A quarter in which no incident produced a disposition is itself a finding: either the loop is not running or incident root-causing is not reaching design

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 2 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
% of Critical-tier software assets adopting the full mandated pattern set	___	___	>=95% of Critical-tier and >=80% of High-tier, trending up quarter over quarter	☐
% of patterns carrying a current tier-applicability row and named owner	___	___	100%, with no pattern past its review date	☐
% of tool-using assets whose autonomy controls match their capability tier	___	___	>=90% at Critical and High tier; zero Critical-tier assets with an unbounded autonomy-step budget	☐
Incidents with a root cause that produced a pattern disposition within 30 days	___	___	>=90% of Critical and High severity incidents, trending up	☐

Where each number is read from:

• % of Critical-tier software assets adopting the full mandated pattern set — Architecture repo pattern-adoption manifest reconciled against SM-Software inventory
• % of patterns carrying a current tier-applicability row and named owner — Pattern registry in the architecture repository
• % of tool-using assets whose autonomy controls match their capability tier — Orchestrator and IaC configuration reconciled against the pattern's autonomy block
• Incidents with a root cause that produced a pattern disposition within 30 days — Incident tracker disposition field cross-referenced with the pattern changelog

Level 2 Oversight Metrics

"A human approves it" is not a control unless you measure whether they refuse. A metric that cannot fail is not a metric.

Metric	Current	Oversight failure it detects	Met?
Approver override rate on gated side-effecting actions, per approver, per quarter	___	Rubber-stamping. An approval gate whose approver has never refused is a logging step wearing the costume of a control, and it satisfies every boolean check the pattern makes.	☐
Approval volume per approver per day, and median decision latency on gated actions	___	Approver overload. Scrutiny does not survive volume; an approver clearing hundreds of items a day is not reviewing them, and latency in the low seconds is the observable signature.	☐
Proportion of Critical-tier assets whose approver is credentialed for that asset class	___	Unqualified oversight. A human is in the loop but lacks the context to recognise the failure the gate exists to catch, so the gate passes traffic it should stop.	☐

Level 2 30-second check (E0)

• ☐ Yes ☐ No — Does each published pattern say which risk tiers must adopt it?
• ☐ Yes ☐ No — Do your higher-autonomy AI assets face stricter controls than your read-only ones?
• ☐ Yes ☐ No — Has a pattern changed in the last two quarters because of an incident?

---

Maturity Level 3, Industry-Leading

Objective: Publish software patterns as executable artifacts adopted by default, verify conformance continuously, and contribute the pattern library back to the field.

Prerequisite cells: SA-Software-L2, IR-Software-L2, ML-Software-L2

Question L3.1: Ship patterns as executable artifacts

Q3.1: Do at least 80% of reference patterns ship as executable artifacts with versioning and a migration path, and do at least 75% of new Critical- and High-tier assets adopt the artifact at creation rather than adopting the document? Evidence this in the Software domain against the agent reference pattern and the antipattern it replaces.

Evidence Required:

• [ ] Convert each reference pattern into something a team runs rather than reads: infrastructure modules, project scaffolds, libraries or configuration bundles that instantiate the pattern correctly by default
• [ ] This is the level's central move and the one that changes adoption economics, because a pattern that is faster to adopt than to circumvent is adopted without enforcement, and no amount of mandate achieves what convenience achieves
• [ ] Version the artifacts, publish their change history, and provide a migration path when a pattern changes so that existing adopters are not stranded on an old version, which is the most common reason executable pattern libraries decay
• [ ] Measure adoption of the artifact rather than conformance to the document, since the artifact is now the pattern
• [ ] In this domain the executable artifact is usually a project scaffold or an SDK wrapper that instantiates tool allowlisting, argument validation and session bounds by default, so the secure agent is the one that takes least effort to start

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L3.2: Verify conformance continuously against the running system

Q3.2: Are at least 85% of pattern invariants verified continuously against the running system with unverifiable invariants listed explicitly, and are Critical-tier breaches detected within 5 business days and routed to issue management? For Software, the assessor should be shown the agent reference pattern and the antipattern it replaces.

Evidence Required:

• [ ] Verify that what is running still matches the pattern that was approved, continuously rather than at review
• [ ] Draw the signals from configuration state, runtime telemetry and the deployment pipeline, and evaluate each pattern's invariants as checks that can fail
• [ ] Architecture drift is the defining failure of this practice at scale: designs are reviewed once and systems change continuously, so a library verified only at design time describes the past
• [ ] Publish a service level covering detection of a Critical-tier conformance breach within 5 business days, and route breaches to the issue-management practice with tier-appropriate severity rather than to a report nobody owns
• [ ] Where an invariant cannot be checked automatically, list it explicitly so the verification's real reach is known
• [ ] Conformance verification reads the running tool manifest, the active prompt template and the model reference, comparing them against what the pattern declares, which catches the tool added after review

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Question L3.3: Contribute the executable pattern library back

Q3.3: Does the organisation contribute at least four accepted artifacts per year in majority-executable form, prioritising the patterns with the highest internal adoption and therefore the most operational evidence? In Software, ask to see the agent reference pattern and the antipattern it replaces.

Evidence Required:

• [ ] Contribute at least four substantive artifacts per year, weighted toward executable form: reference implementations, infrastructure modules, scaffolds and the conformance checks that verify them
• [ ] Executable contributions carry disproportionate value in this practice, because a peer organisation can adopt a module and can only read a diagram, and the adoption gap between the two is the difference between influencing the field and describing it
• [ ] Work through OpenSSF AI, CNCF where infrastructure applies, OWASP projects and sector bodies
• [ ] Count only accepted artifacts, review each legally before release, and prefer contributing the patterns with the highest internal adoption, since those carry the most operational evidence and are the most likely to survive contact with another organisation's environment
• [ ] Software patterns contribute well because the agent and RAG shapes are near-universal, and a reference implementation transfers across organisations with little adaptation

Evidence grade for this question: ☐ E0 ☐ E1 ☐ E2 ☐ E3

Answer:

• ☐ Fully Mature (Evidence complete + ≥3 metrics meet targets)
• ☐ Implemented (Evidence complete + 2 metrics meet targets)
• ☐ Partial (Evidence partially complete + <2 metrics meet targets)
• ☐ Not Implemented (No evidence)

Evidence Location: _________________________________

Notes: ___________________________________________

Level 3 Outcome Metrics

Metric	Baseline	Current	Target	Met?	Notes
Reference patterns shipped as executable artifacts rather than documents alone	___	___	≥80% of patterns	☐
New assets adopting the executable pattern artifact at creation	___	___	≥75% of new Critical- and High-tier assets	☐
Pattern invariants verified continuously against the running system	___	___	≥85%, with unverifiable invariants listed explicitly	☐
Time to detect a Critical-tier architecture conformance breach	___	___	≤5 business days at the 90th percentile	☐
Accepted external contributions per year in executable form	___	___	≥4 accepted, majority executable	☐

Where each number is read from:

• Reference patterns shipped as executable artifacts rather than documents alone — Pattern artifact registry
• New assets adopting the executable pattern artifact at creation — Scaffold and module adoption telemetry
• Pattern invariants verified continuously against the running system — Conformance verification registry
• Time to detect a Critical-tier architecture conformance breach — Conformance verification event log
• Accepted external contributions per year in executable form — Contribution log with acceptance references

Level 3 Oversight Metrics

"A human approves it" is not a control unless you measure whether they refuse. A metric that cannot fail is not a metric.

Metric	Current	Oversight failure it detects	Met?
Pattern deviations later implicated in an incident, as a share of deviations approved	___	deviation approval that is procedurally correct and substantively wrong, which only post-hoc linkage to incidents can reveal	☐
Automated pattern conformance checks overridden by a human, with the override sustained	___	automation override becoming the routine path around a check that is inconvenient rather than wrong, which is how enforced patterns quietly become advisory	☐
Reviewer disagreement rate on a blind double-review sample of Critical-tier designs	___	reviewer drift, where the same design would be approved by one architect and refused by another, making the gate a lottery rather than a control	☐

Level 3 30-second check (E0)

• ☐ Yes ☐ No — Can a team adopt your secure architecture by running something rather than reading something?
• ☐ Yes ☐ No — Would you detect a running system drifting from its approved design?
• ☐ Yes ☐ No — Have you published an architecture module others can actually deploy?

---

Summary Scorecard

Level	Q1	Q2	Q3	Level score	Weakest evidence grade	Achieved?
L1	___	___	___	___	___	☐
L2	___	___	___	___	___	☐
L3	___	___	___	___	___	☐

Secure Architecture (SA) maturity in the Software domain: ____________

---

Document Version: HAIAMM v3.5 (generated)