Deployment reality
PRD tier
One product-requirements document per task, handed to each model in its native harness and executed against a live Cloud Foundry foundation. This is where models that cluster on the suite pull apart — a strong on-prem model can score 0.94 on the exam and still fail a binding task outright.
Each dot is one run, shaded by score. The deployment median includes every valid scheduled run: no-engagement counts as 0, while a harness-invalid run is excluded and rerun. A verified upstream serving failure is excluded from the model-performance median and remains visible as reliability evidence. Engagement remains visible as a separate diagnostic. The scored/control labels describe this topology protocol; this supplemental panel is not blended into the existing collective score.
PRD-006 — service-binding conflict
scoredTwo bound services advertise overlapping credential keys; the app picks the wrong one. Selection must be order-independent — verified by executing the resolver under both binding orders.
sample repository ↗PRD-007 — no-fault control
controlA working app and a plausible but false bug report. Scored inverted: investigating and correctly reporting "nothing is broken" is the right answer. A control — it validates that models do not fabricate faults, and does not enter the collective score.
sample repository ↗PRD-101 — exactly-once effects
scoredRepair a stateful workflow so retries cannot duplicate externally visible effects. Scoring exercises the implementation under repeated and interrupted delivery.
sample repository ↗PRD-102 — tenant isolation
scoredRestore tenant boundaries across storage and request handling. Behavioural probes require authorized access to work while cross-tenant access remains blocked.
sample repository ↗PRD-103 — event-contract migration
scoredMigrate a live event contract without breaking old producers or new consumers. The verifier checks compatibility and the migration seam rather than a static code shape.
sample repository ↗PRD-104 — binding-secret exposure
scoredRemove an application path that exposes bound-service credentials while preserving the legitimate service integration and operational behaviour.
sample repository ↗PRD-110 — retry-storm backpressure
scoredMake a failure-prone integration degrade safely under a retry storm. Scoring checks bounded work, backpressure, and recovery instead of a happy-path response.
sample repository ↗PRD tier panel. Every row is model x harness x device. Per-run scores are published beside the median because a median can misdescribe a bimodal lane. K varies per lane and is stated; PRD scores are a separate instrument from the suite composite and are never averaged into it.