Benchmarks
100% pass rate on the 291-story build benchmark
SnowCoder's Yeti Build Agent is verified nightly against a frozen set of real ServiceNow stories with auditable acceptance criteria.
Frozen acceptance criteria. Verified nightly. The Yeti Build Agent generates 42 artifact classes via the ServiceNow Fluent SDK, runs the super-audit, and auto-heals failures until every AC is met or remediation is surfaced.
Want the underlying detail?
The frozen acceptance criteria and the per-story super-audit results are available to enterprise customers and prospects under NDA. The same dataset is used to validate every nightly build of the Yeti Build Agent before it ships.
Methodology
The benchmark is built around real customer stories - not synthetic prompts. Each story carries acceptance criteria written in plain English by a human, a frozen target-instance fixture (a public ServiceNow PDI), and a frozen super-audit spec.
Every nightly run executes all 291 stories end-to-end through the Yeti Build Agent's 10-stage pipeline (Validation → Technical Spec → Data Model → Business Logic → Frontend Config → Fluent CodeGen → ATF Testing → Update Sets & Verification → Security & Script Audit → Deployment) and the result is graded against the frozen super-audit. A story passes only when every acceptance criterion is verifiably met.
The benchmark is the source of truth we use for product decisions - pipeline changes, plugin updates, model changes. We publish the headline number publicly and share the per-story detail with customers under NDA.
See it on your own backlog
The benchmark is run against public stories. Your stories are different - talk to us about a structured proof-of-value on your own backlog.