The validation question
Can DRE produce repeatable evidence that the system, infrastructure, controls, and transaction-processing paths behave as designed?
DRE should progressively replace assertions with observation. “The architecture is private” is a claim. “Public network access is disabled, the private endpoint exists, private DNS resolves correctly, an authorized workload can invoke the approved model, an unauthorized path cannot, and the result is captured in a repeatable test record” is evidence.
The validation sequence
- Static validation — deployment location resolves, naming follows the
legalfirewallconvention, required resources are represented, network definitions are valid, declarations compile. - Controlled test deployment — deploy into an authorized, attributable test resource group.
- Configuration assertions — inspect deployed properties and compare them with the expected contract.
- Connectivity testing — prove the intended paths work.
- Data and model integration testing — exercise Cosmos DB and the approved model deployment.
- Evidence capture — retain a reviewable record of the run.
- Cleanup and repeatability — remove test resources and verify nothing was left behind.
Infrastructure contract assertions
| Resource | Asserted properties |
|---|---|
| Virtual network | Address space is 10.0.0.0/16; AppSubnet exists; AppSubnet uses 10.0.1.0/24. |
| Key Vault | Vault exists; Standard SKU is provisioned; deployment flags match the declared infrastructure. |
| Cosmos DB | Account exists; type is GlobalDocumentDB; Standard offer configured; primary location matches the deployment region. |
| Azure OpenAI | Account exists; S0 SKU; custom subdomain established; publicNetworkAccess equals Disabled. |
Negative validation matters most
The approved path works. The prohibited path does not.
It is not enough to prove that an approved application can reach Azure OpenAI or Cosmos DB. The completed environment should also prove that the paths which are supposed to be prohibited actually fail — and that the failure is observable. That turns architectural intent into testable security behaviour.
Testing the two processing layers separately
The deterministic engine is especially suitable for repeatable coverage because its rules and expected outcomes are explicit: clear positive and negative examples, multiple triggering conditions, boundary conditions, multiple violations in one narrative, and rule-version behaviour.
Consensus validation requires a different method because the layer is probabilistic. It splits into two testable questions: what did each agent return, and given those votes, did deterministic routing produce the expected system action? Keeping those separate makes failures diagnosable.
With exactly three agents and three valid vote values, the currently defined referee branches appear capable of covering every valid combination — which means the nominal human-escalation branch may not be reachable through ordinary valid votes. This is precisely the kind of condition validation should expose rather than allowing a safe-state path to exist only in documentation.
Evidence and repeatability
Each meaningful run should produce a reviewable record: test ID, environment, infrastructure version, application version, rule-set version, model configuration, input dataset version, individual outcomes, expected denials, failures, and cleanup result. A successful deployment that leaves uncontrolled resources, credentials, storage, or cost behind is not a completely successful validation cycle.
Production readiness is an evidence state
Not a calendar milestone. The current system should not be described as having met production criteria until the corresponding implementation and evidence exist.
