SutroSutro

An API for your team's judgment

Sutro makes it easy to teach AI what good looks like by your standards - not anyone else’s. Then put it to work reliably and affordably at any scale.

Get Started Today

Lead scorer score_lead()Running on:Gemma 4 31B

Northstar Health

northstarhealth.example / about-us “better care starts before the visit” HQ: Austin, TX | team: ~240? (LinkedIn says 201–500) Healthcare workflow software for multi-site clinics; intake forms, scheduling, reminders + EHR integrations. CRM notes / updated Tue: sells to ops teams, NOT a care provider. Series B per old deck. Hiring implementation + enterprise AE roles. From website footer: patient engagement / reduce no-shows / HIPAA-ready workflows. Outbound notes: 3 locations mentioned in case study; customer count unconfirmed. See duplicate account “Northstar Hlth” before routing. Tech: API docs, Salesforce integration, calendar sync. Buyer likely VP Operations. Contact history still needs deduping.

{
  "company": "Northstar Health",
  "fit": "strong",
  "priority": "high",
  "reason": "Matches our target industry and company size."
}

Candidate reviewer review_candidate()Running on:Claude Haiku 5.5

Morgan Lee

RESUME_final_v3 (2).pdf — copied text MORGAN LEE | Senior Software Engineer [email protected] / portfolio available EXPERIENCE 2023–present Northline / Sr SWE — data platform • Owned ingestion jobs + on-call; Python / SQL; migrated legacy pipelines, “~40% fewer failures” 2019 - 2023 Software Engineer Built APIs, distributed processing & internal analytics. Stack listed elsewhere: postgres, Kafka, Python3, SQL. EDUCATION: B.S. Computer Science SKILLS / Python, SQL, distributed systems [page 2] Projects: event ingestion service; warehouse migration. Formatting lost in export. Six years of professional engineering experience as of this resume.

{
  "years_experience": 6,
  "role_fit": "strong",
  "seniority": "senior",
  "summary": "Strong platform experience; leadership scope unclear."
}

Listing reviewer review_listing()Running on:GPT 6.1 Terra

Forest-green canvas backpack with a front zip pocket on a neutral studio background 
{
  "category": "backpack",
  "brand_fit": "on_brand",
  "listing_quality": "needs_work",
  "issues": [
    "Interior not shown"
  ]
}

Why you'll love it

No prompting, fine-tuning, or evals.

Just upload a sample of your data, give feedback on a handful of examples, and watch quality climb automatically.

Evaluation quality
Quality climbs with your feedback Illustrative example: quality increases from a 50% seed prompt to 57%, 78%, 81%, and 93% over four iterations. Each iteration adds 10 annotations, for 40 total and a 43 percentage point improvement. The dashed line shows seed prompt quality. 100%75%50% 10 annotations10 annotations 30 annotations30 annotations 93% 50% Seed1234 Iteration

Know which model is best for the job.

Run your tasks on any AI model. Sutro helps you find the best fit for your quality and cost needs, and easily switch when new models come out.

Cost–quality frontier
The Pareto-efficient cost–quality frontier Illustrative cost–quality comparison: three configurations above 77% accuracy highlighted in purple, with Claude Sonnet 4.5 and the lower-accuracy Nvidia model shown as unhighlighted comparisons. Costs use a logarithmic scale. 60%70%80%90%$0.2$1$5$25AccuracyCost / 1K records · log scale GPT-OSS 120B: 88.6% accuracy; $0.63 / 1K recordsClaude Sonnet 4.5: 87.1% accuracy; $24.32 / 1K recordsGemma 4 26B A4B: 85.7% accuracy; $0.28 / 1K recordsGPT-OSS 20B: 77.1% accuracy; $0.22 / 1K records Nemotron 3 Nano 30B A3B: 62.9% accuracy; $0.22 / 1K records Gemma 4 26BValue GPT-OSS 120BBest88.6% · $0.63 / 1K records

Surprisingly affordable.

Run millions of AI tasks for the price of a coffee run. Unlock use cases that seemed out of reach, while raising the quality bar.

Before
1042 · support✓
1043 · billing✓
$0
0 records processed
After
1042 · support✓
1043 · billing✓
$0
0 records processed

Hear it from our customers

“Sutro gives us a framework to develop judges that we have faith in.”
Sr. Software Engineer+60%Accuracy−94%Inference cost64 annotations per function
“Sutro saves our team countless hours, and it gives us the invaluable ability to measure, optimize, and prevent regressions against our domain expertise.”
CTOAI Data Marketplace+28%Accuracy−73%Inference cost163 annotations per function
“Sutro unlocks use cases that weren't previously possible from a cost perspective.”
Principal Data Platform Engineer+18%Accuracy−98%Inference cost30 annotations per function

*Average per customer function.

Request customer case studies

How it works

Define and Align

Sutro surfaces hard cases for your experts to annotate - then optimizes directly against their feedback.

app.sutro.sh/functions/lead-qualifier
Optimize
Collecting feedback · 1/3
Is this company a good fit for our sales team?
Example 1 of 3

Cedar Labs · website + CRM export

industry: Software Development | employees: 201–500
site: “Your embedded product team. We design & build for clients.”
crm_note: “SaaS? AE to confirm” | enrichment: B2B Software
pricing: project-based; no subscription product found
Is this a qualified lead?Reviewing example

Model Sweeps

Benchmark models on your task to build your own frontier of cost and quality. See which models deliver the best quality for your budget, then choose the tradeoff that works for you. Redraw your task frontier with a single click when new models are released.

The Pareto-efficient cost–quality frontier Eleven optimized models, with three configurations above 77% accuracy highlighted in purple. The 62.9% Nvidia model remains visible but is not highlighted. GPT-OSS 120B has the highest measured accuracy: 88.6% at $0.63 per thousand records. Model-provider logos identify each point. Costs use a logarithmic scale. 60%70%80%90%$0.2$1$5$25AccuracyCost / 1K records · log scale GPT-OSS 120B: 88.6% accuracy; $0.63 / 1K recordsGemini 3.7 Flash: 87.1% accuracy; $3.71 / 1K recordsClaude Sonnet 4.5: 87.1% accuracy; $24.32 / 1K recordsClaude Haiku 4.5: 85.7% accuracy; $8.45 / 1K recordsGemma 4 26B A4B: 85.7% accuracy; $0.28 / 1K recordsGPT-5.6 Terra: 84.3% accuracy; $6.74 / 1K recordsGPT-5.6 Luna: 80.0% accuracy; $0.73 / 1K recordsGemma 4 31B: 80.0% accuracy; $0.36 / 1K recordsGPT-OSS 20B: 77.1% accuracy; $0.22 / 1K recordsNemotron 3 Super 120B: 74.3% accuracy; $1.70 / 1K recordsNemotron 3 Nano 30B A3B: 62.9% accuracy; $0.22 / 1K records Gemma 4 26BValue GPT-OSS 120BBest88.6% · $0.63 / 1K records

Scale

Run your function where you see fit: on your own inference providers, or using Sutro's ultra cost-efficient batch inference service. Process millions of rows and billions of tokens at a time.

score_leads("january_crm_export.csv")Auto-scaling
One function, millions of rows As a dataset grows, copies of the same function run in parallel. Each input row maps to its own output row. ƒ ƒ ƒ ƒ ƒ ƒ ⋮ ⋮ ⋮
1,024,683 records

What are customers building on Sutro?

and many more

Simple pricing

All plans include a free two-week trial.

Book a demo

Platform plan

Starts at

$500 / month

Includes $100/month in inference credits

Enterprise

Custom

For enterprise and self-hosted deployments, tailored to your needs and scale.

Batch inference

Machine-time pricing means you only pay for the compute your workload actually consumes. Data can be read from your own S3-compatible storage.

Estimated price $120
Number of records 10 million
Average uncached input size

Frequently asked

Is this an evals product?

Yes. Sutro helps evaluate quality and reliability, but unlike other evals products, it directly shapes model or agent behavior from feedback, rather than just monitoring. We also provide a managed inference service, enabling end-to-end data pipeline creation, optimization, management, and deployment in one platform.

Can Sutro use image inputs and other modalities? Can it call tools?

Yes. Sutro supports images, PDFs, and web search capabilities today. Additional modalities and custom tool calling are available on a per-request basis.

What models does Sutro use?

Sutro is model-agnostic, and we support a mix of open-source and proprietary models. Small, open-source models commonly outperform larger proprietary models on expert-aligned data processing tasks. Sutro helps automatically find the best model for your needs.

Can I self-host Sutro? Do you support BYOC and BYOK?

Yes. Sutro can be managed as SaaS or entirely self-hosted. You can bring your own provider keys and cloud credentials in either case.

How does Sutro compare to prompt engineering, fine-tuning, or RL?

Sutro is designed to be more robust, accurate, and scalable than prompt engineering, and more data efficient, portable, and re-trainable than fine-tuning or RL. We encourage you to try it out and see for yourself!

How does Sutro compare to Type-One models like Jev?

Sutro builds the expert harness around models, but isn't a model company itself. Models like Jev can be used within Sutro.

What will you scale with Sutro?

Get Started