Skip to content

Commit 191f36f

Browse files
committed
Add zero-trust solutions matrix and integrate metrics into release dashboard
1 parent 7f1b76d commit 191f36f

4 files changed

Lines changed: 140 additions & 0 deletions

File tree

README.md

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -905,6 +905,37 @@ jobs:
905905

906906
The suite now includes proof-style parity tests that run the same function behavior in both legacy Java and translated Python and assert identical outputs for shared input vectors.
907907

908+
### What Is A Vector, Vectoring, And A Vector Runner?
909+
910+
In this repository, a **vector** means one structured test case: input values plus the expected output.
911+
912+
Example vector concept:
913+
- Input: `base=5`, `multiplier=10`, `premium=true`
914+
- Expected output: `75`
915+
916+
That single row is one vector. A vector file is a list of many such rows (normal, edge, and negative scenarios).
917+
918+
**Vectoring** is the testing approach where both runtimes (legacy Java and translated Python) are driven from that same shared vector dataset instead of hardcoded test values in multiple places.
919+
920+
Why vectoring is useful:
921+
- Single source of truth for migration parity expectations
922+
- Less duplicated test data across languages
923+
- Easier reviews and audits of behavioral requirements
924+
- Faster updates when business rules change
925+
926+
**Vector Runner** in this project:
927+
- `LegacyCalculatorVectorRunner.java` reads the shared JSON vectors
928+
- Executes the legacy Java function for each vector
929+
- Emits per-case output (`id, actual, expected`) for parity checks
930+
931+
This is how we prove output equivalence:
932+
1. Define vectors in shared JSON/CSV fixture files
933+
2. Run legacy Java against those vectors
934+
3. Run translated Python against those same vectors
935+
4. Assert Java output equals Python output for each vector id
936+
937+
This pattern gives an explicit migration proof: same inputs, same outputs, across runtimes.
938+
908939
| Proof Test | What It Verifies | Location |
909940
|---|---|---|
910941
| Java fixture expected-value test | Legacy Java behavior is stable and explicit | `tests/correctness/test_legacy_java_python_equivalence.py` |
@@ -939,6 +970,25 @@ Fixture sources:
939970

940971
Practical recommendation: keep a shared vector file and run both Java and Python against it, treating Java output as the initial oracle during migration.
941972

973+
### Zero-Trust Solutions Matrix
974+
975+
| Zero-Trust Control | What It Means | Project Implementation | Evidence |
976+
|---|---|---|---|
977+
| Verify identity on every request | No implicit trust by network location | JWT verification + RBAC dependency checks in API routes | `tests/negative/test_rbac_enforcement.py` |
978+
| Explicit policy decision per request | Each request must be allow/deny evaluated | Input guardrails, model lock, egress policy lock, blocked audit path | `tests/negative/test_model_blocking.py`, `tests/negative/test_egress_blocking.py`, `tests/adversarial/test_prompt_injection.py` |
979+
| Least privilege access | Users only get required capabilities | Role-permission mapping with permission-scoped endpoints | `core/auth.py`, `tests/negative/test_rbac_enforcement.py` |
980+
| Continuous verification | Runtime signals prove controls remain active | Audit report includes zero-trust rates, quality attestations, deny rate | `/api/v1/audit-report` zero-trust section |
981+
| Assume breach + contain blast radius | Treat unsafe inputs as hostile by default | Block injection/secret payloads and sanitize audit records | `guardrails/input_guard.py`, `guardrails/output_guard.py`, `tests/integration/test_audit_trail.py` |
982+
983+
The release dashboard now includes a dedicated `zero_trust` section with:
984+
- `posture`
985+
- `identity_verification_rate`
986+
- `policy_decision_rate`
987+
- `continuous_verification_rate`
988+
- `policy_deny_rate`
989+
990+
This makes zero-trust status measurable release-over-release instead of purely descriptive.
991+
942992
### Requirements-to-Implementation Mapping
943993

944994
| Requirement (README) | Test (pytest) | Security Check | Quality Gate | Coverage |

core/audit_dashboard.py

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,18 +68,26 @@ def build_release_dashboard(records: list[dict[str, Any]]) -> dict[str, Any]:
6868
action_latencies: dict[str, list[float]] = defaultdict(list)
6969
action_loadrunner_pass: Counter[str] = Counter()
7070
action_loadrunner_total: Counter[str] = Counter()
71+
identity_verified_count = 0
72+
policy_decision_count = 0
73+
continuous_verification_count = 0
7174

7275
for record in records:
7376
action = str(record.get("action") or "unknown")
7477
action_counts[action] += 1
7578

7679
status = str(record.get("status") or "unknown")
7780
status_counts[status] += 1
81+
if status in {"ok", "blocked"} or isinstance(record.get("blocked"), bool):
82+
policy_decision_count += 1
7883
if status == "ok":
7984
total_ok += 1
8085
if record.get("blocked") is True or status == "blocked":
8186
total_blocked += 1
8287

88+
if record.get("user_id") is not None or record.get("sub") is not None:
89+
identity_verified_count += 1
90+
8391
latency = record.get("latency_ms")
8492
if isinstance(latency, (int, float)):
8593
numeric_latency = float(latency)
@@ -109,6 +117,10 @@ def build_release_dashboard(records: list[dict[str, Any]]) -> dict[str, Any]:
109117
if isinstance(dpmo, (int, float)):
110118
all_dpmo.append(float(dpmo))
111119

120+
# Continuous verification evidence: request carried runtime quality signals
121+
if isinstance(record.get("ctq_metrics"), dict) and isinstance(record.get("loadrunner"), dict):
122+
continuous_verification_count += 1
123+
112124
loadrunner = record.get("loadrunner")
113125
if isinstance(loadrunner, dict):
114126
action_loadrunner_total[action] += 1
@@ -137,6 +149,20 @@ def build_release_dashboard(records: list[dict[str, Any]]) -> dict[str, Any]:
137149
"pass_rate": round(ctq_pass_counts[metric_name] / total, 3) if total else 0.0,
138150
}
139151

152+
total_records = len(records)
153+
identity_rate = round(identity_verified_count / total_records, 3) if total_records else 0.0
154+
policy_decision_rate = round(policy_decision_count / total_records, 3) if total_records else 0.0
155+
continuous_verification_rate = round(continuous_verification_count / total_records, 3) if total_records else 0.0
156+
157+
if total_records == 0:
158+
posture = "insufficient_data"
159+
elif identity_rate >= 0.99 and policy_decision_rate >= 0.99 and continuous_verification_rate >= 0.95:
160+
posture = "strong"
161+
elif identity_rate >= 0.95 and policy_decision_rate >= 0.95:
162+
posture = "moderate"
163+
else:
164+
posture = "needs_hardening"
165+
140166
return {
141167
"generated_at": datetime.now(timezone.utc).isoformat(),
142168
"summary": {
@@ -159,6 +185,18 @@ def build_release_dashboard(records: list[dict[str, Any]]) -> dict[str, Any]:
159185
"sigma_band_counts": dict(sigma_band_counts),
160186
"control_state_counts": dict(control_state_counts),
161187
},
188+
"zero_trust": {
189+
"posture": posture,
190+
"identity_verification_rate": identity_rate,
191+
"policy_decision_rate": policy_decision_rate,
192+
"continuous_verification_rate": continuous_verification_rate,
193+
"policy_deny_rate": round(total_blocked / total_records, 3) if total_records else 0.0,
194+
"signals": {
195+
"identity_per_request": identity_rate >= 0.99,
196+
"explicit_policy_decision_per_request": policy_decision_rate >= 0.99,
197+
"runtime_quality_attestation": continuous_verification_rate >= 0.95,
198+
},
199+
},
162200
}
163201

164202

tests/integration/test_audit_trail.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,5 +252,9 @@ async def test_audit_report_endpoint_aggregates_release_dashboard(engineer_clien
252252
assert "translate_project" in body["actions"]
253253
assert "performance" in body
254254
assert "quality" in body
255+
assert "zero_trust" in body
255256
assert "ctq_metrics" in body["quality"]
257+
assert "posture" in body["zero_trust"]
258+
assert 0.0 <= body["zero_trust"]["identity_verification_rate"] <= 1.0
259+
assert 0.0 <= body["zero_trust"]["policy_decision_rate"] <= 1.0
256260
assert body["viewer"] == "audit-test-user"

tests/unit/test_audit_dashboard.py

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
import pytest
2+
3+
from core.audit_dashboard import build_release_dashboard
4+
5+
6+
pytestmark = pytest.mark.unit
7+
8+
9+
def test_release_dashboard_contains_zero_trust_section():
10+
records = [
11+
{
12+
"action": "translate",
13+
"status": "ok",
14+
"blocked": False,
15+
"user_id": "u1",
16+
"latency_ms": 55.0,
17+
"performance_status": "within_control",
18+
"ctq_metrics": {"latency": "within_control", "reliability": "pass"},
19+
"six_sigma": {"dpmo": 0.0, "sigma_band": "world_class", "control_state": "in_control"},
20+
"loadrunner": {"transaction": "translate", "passed": True},
21+
}
22+
]
23+
24+
dashboard = build_release_dashboard(records)
25+
26+
assert "zero_trust" in dashboard
27+
assert dashboard["zero_trust"]["posture"] in {"strong", "moderate", "needs_hardening", "insufficient_data"}
28+
assert dashboard["zero_trust"]["identity_verification_rate"] == 1.0
29+
assert dashboard["zero_trust"]["policy_decision_rate"] == 1.0
30+
31+
32+
def test_release_dashboard_zero_trust_needs_hardening_without_identity_signal():
33+
records = [
34+
{
35+
"action": "translate",
36+
"status": "ok",
37+
"blocked": False,
38+
"latency_ms": 60.0,
39+
"performance_status": "within_control",
40+
"ctq_metrics": {"latency": "within_control"},
41+
"loadrunner": {"transaction": "translate", "passed": True},
42+
}
43+
]
44+
45+
dashboard = build_release_dashboard(records)
46+
47+
assert dashboard["zero_trust"]["identity_verification_rate"] == 0.0
48+
assert dashboard["zero_trust"]["posture"] == "needs_hardening"

0 commit comments

Comments
 (0)