Intelligence
Notes from the frontier.
What we're learning as we build: research, live benchmarks, white papers and free tools.
Research
Sep 18, 2026 · 1 min read
Why most AI pilots never reach production — and the five things that do
We looked back at every engagement we've run. The projects that shipped had less to do with the model than with evaluation, ownership and scope.
Mohan Pasappulatti
Articles
Why most AI pilots never reach production — and the five things that do
We looked back at every engagement we've run. The projects that shipped had less to do with the model than with evaluation, ownership and scope.
Research · Sep 18, 2026
Your evals are your moat
Leaderboards measure someone else's problem. Here's how we write benchmarks for a client's actual work.
Engineering · Sep 11, 2026
Build, buy or wait: a decision guide for 2026
The model landscape changes monthly. A simple framework for knowing when to commit.
Strategy · Sep 4, 2026
What we learned deploying agents in a claims department
Three months, one very patient operations team and a lot of edge cases.
Field notes · Aug 27, 2026
The real cost of running AI in production
Token prices fall every quarter. Total cost doesn't always follow. Where the money actually goes.
Research · Aug 20, 2026
Benchmarks
Tested on real business work, re-run as models ship.
Enterprise document QA
Answering questions over long contracts, policies and filings, graded by domain experts.
Top score 92.4 · 14 models · Updated Sep 20
Contract review
Finding risky clauses against a legal playbook, scored for recall and false alarms.
Top score 88.1 · 11 models · Sep 12
Agentic task completion
Multi-step tasks across CRM, email and spreadsheets, completed end to end.
Top score 71.6 · 9 models · Updated Sep 19
Support resolution
Resolving real, anonymized support tickets with correct policy and tone.
Top score 84.9 · 12 models · Sep 6
Research briefs
Other people’s research, read closely.
The Autonomy Paradox
Why less autonomy may deliver greater business value.
Brief
Read the brief →The cost curve after the pilot
Where the money actually goes once a system is carrying real volume, measured across our own deployments.
Brief · coming soon
Evaluations that outlive the model
Writing benchmarks tied to the work rather than to the model, so a swap does not reset what you know.
Brief · coming soon
White papers
The long version.
From pilot to production
A field guide to taking AI systems live: evaluation, governance, rollout and ownership.
34 pages · coming soon
The applied AI operating model
How to organize teams, budgets and decision rights when AI becomes core to the business.
28 pages · coming soon
Evaluating frontier models for enterprise use
Our methodology for testing models on real business work, and how to run it yourself.
41 pages · coming soon
Free tools
What we use, yours to use too.
AI ROI estimator
Estimate return and payback for a use case from a few inputs.
Calculator
Model cost planner
Compare running costs across models for your expected volume.
Calculator
AI readiness check
Twelve questions to find where your organization is ready, and where it isn't.
Assessment
Eval starter kit
Templates and scripts to build your first task-specific benchmark.
Kit
Want this applied to your own numbers?
Intelligence
Notes from the frontier.
What we're learning as we build: research, live benchmarks, white papers and free tools.
Featured · Research
Why most AI pilots never reach production — and the five things that do
We looked back at every engagement we've run. The projects that shipped had less to do with the model than with evaluation, ownership and scope.
By Mohan Pasappulatti
Latest
Your evals are your moat
Leaderboards measure someone else's problem. Here's how we write benchmarks for a client's actual work.
Build, buy or wait: a decision guide for 2026
The model landscape changes monthly. A simple framework for knowing when to commit.
What we learned deploying agents in a claims department
Three months, one very patient operations team and a lot of edge cases.
The real cost of running AI in production
Token prices fall every quarter. Total cost doesn't always follow. Where the money actually goes.
Active benchmarks
Tested on real business work, re-run as models ship.
Methodology →
- RunningUpdated Sep 20
Enterprise document QA
Answering questions over long contracts, policies and filings, graded by domain experts.
92.4Top score · 14 models - UpdatedSep 12
Contract review
Finding risky clauses against a legal playbook, scored for recall and false alarms.
88.1Top score · 11 models - RunningUpdated Sep 19
Agentic task completion
Multi-step tasks across CRM, email and spreadsheets, completed end to end.
71.6Top score · 9 models - UpdatedSep 6
Support resolution
Resolving real, anonymized support tickets with correct policy and tone.
84.9Top score · 12 models
Research briefs
Other people’s research, read closely.
Brief
The Autonomy Paradox
Why less autonomy may deliver greater business value.
Read the brief →Brief
The cost curve after the pilot
Where the money actually goes once a system is carrying real volume, measured across our own deployments.
Coming soonBrief
Evaluations that outlive the model
Writing benchmarks tied to the work rather than to the model, so a swap does not reset what you know.
Coming soon
White papers
The long version.
From pilot to production
A field guide to taking AI systems live: evaluation, governance, rollout and ownership.
PDF · 34 pages · coming soonThe applied AI operating model
How to organize teams, budgets and decision rights when AI becomes core to the business.
PDF · 28 pages · coming soonEvaluating frontier models for enterprise use
Our methodology for testing models on real business work, and how to run it yourself.
PDF · 41 pages · coming soon
Free tools
What we use, yours to use too.
Calculator
AI ROI estimator
Estimate return and payback for a use case from a few inputs.
Open tool →Calculator
Model cost planner
Compare running costs across models for your expected volume.
Open tool →Assessment
AI readiness check
Twelve questions to find where your organization is ready, and where it isn't.
Open tool →Kit
Eval starter kit
Templates and scripts to build your first task-specific benchmark.
Open tool →
Signal, once a month.
New research, benchmark results and tools. No hype, easy to unsubscribe.