Haiku 5.5 is 75% cheaper and now actually good at browser tasks. Small-model economics just changed.
Jump to a section
See the story at a glance
Anthropic released Haiku 5.5 on October 7. The headline is 75% cheaper on average versus Haiku 4.5, which is already notable. But the number I keep coming back to is the OSWorld score.
Haiku 4.5 scored 15.7% on OSWorld 2.1's offline subset. That's a score that says: you can try to use this model in a browser-automation loop, but most of the time it will fail. Haiku 5.5 scores 72.4%. That is not an incremental improvement. That is a different product category.
What the benchmarks actually say
All numbers from Anthropic's launch post:
- OSWorld 2.1 (offline subset): 72.4% vs 15.7% for Haiku 4.5
- Terminal-Bench 4.0: 39.2% vs 0.0% for Haiku 4.5 — the model couldn't do terminal tasks at all before
- Humanity's Last Exam (no tools): 45.9% vs 10.2%
- GDPval-AA v2.1: 1620 vs 735 (Anthropic's internal agentic eval suite)
Pricing structure worth understanding carefully. Two tiers based on request length:
- Under 100K tokens: $0.10 per million input, $0.50 per million output — 90% cheaper than Haiku 4.5
- Over 100K tokens: $0.50 per million input, $2.50 per million output — 50% cheaper than Haiku 4.5
The 75% average reduction Anthropic headlines is a blended figure across both tiers. If most of your calls are under 100K tokens, the actual savings are closer to 90%.
The effort level thing
First Haiku-class model with adjustable effort. That is the feature I will actually use day-to-day.
If you've been building multi-agent workflows, you've probably been doing one of two things: either using Haiku everywhere because it's cheap (and accepting a high failure rate on harder sub-tasks), or using Sonnet everywhere and absorbing the cost. Haiku 5.5 gives you a third option: a Haiku you can turn up.
Anthropic's positioning is explicit — Haiku 5.5 as a subagent alongside Opus 5.5 and Sonnet 5.5 orchestrators. The model handles repetitive sub-tasks (summarization, classification, form-filling, short reasoning) at $0.10/M while the orchestrator runs on a more expensive model for planning.
One genuine gotcha: the tokenizer is updated. Haiku 5.5 uses slightly more tokens per equivalent input than Haiku 4.5 did. Anthropic doesn't publish a specific overhead percentage in the release post. Worth running your representative inputs through the tokenizer before assuming the 90% cost cut is fully realized. For most workloads the overhead is small. For very tight latency/cost budgets, measure it.
What I think the OSWorld number actually means
72.4% on OSWorld 2.1's offline subset. Let me be precise about what that is and isn't.
OSWorld is a well-established computer-use benchmark — the standard in the field for testing whether a model can navigate a GUI, fill forms, open applications, respond to screen state. The "offline subset" removes network-dependent tasks (tasks that require a live internet connection to complete). Full OSWorld numbers are harder, and Anthropic doesn't publish those.
So 72.4% means: on the tasks OSWorld can evaluate without a live internet connection, Haiku 5.5 succeeds almost 3 out of 4 times. That's actually usable. Haiku 4.5's 15.7% was not.
Or maybe I'm being too optimistic. The offline subset is easier than the real-world online version. Tasks that require dynamic web content, login flows, or navigation across pages might perform differently. Before I redesign a system around this benchmark, I'd run it against my actual browser-automation use cases.
The Terminal-Bench number is the one I find more decisive. Terminal-Bench 4.0: 39.2% vs 0.0%. Zero. Haiku 4.5 literally could not do terminal tasks. This isn't a regression from 30% to 39% — this is a model that gained a capability class it didn't have before. For CLI automation, scripting, and server-side agent work, that matters.
| Benchmark | Haiku 4.5 | Haiku 5.5 | Δ |
|---|---|---|---|
| OSWorld 2.1 (offline) | 15.7% | 72.4% | +56.7pp |
| Terminal-Bench 4.0 | 0.0% | 39.2% | +39.2pp |
| Humanity's Last Exam | 10.2% | 45.9% | +35.7pp |
| GDPval-AA v2.1 | 735 | 1620 | +120% |
Source spread
- Anthropic — Claude Haiku 5.5 — hype (own announcement). All benchmark numbers and pricing sourced here. Benchmarks are Anthropic-reported, not independently reproduced.
- vals.ai — Claude Haiku 5.5 model card — builder. Third-party listing confirming the October 7 release date and basic specs.
Pros & cons
What's genuinely useful:
- 90% cost reduction for sub-100K requests is a primary-source number. For high-volume classification, summarization, or short-context pipelines, this changes the economics.
- OSWorld 72.4% makes browser-use workflows viable at Haiku pricing — previously you needed Sonnet or higher for reliable computer-use.
- First effort levels on Haiku means per-request cost control without model-swapping.
- Available everywhere on day one: AWS, GCP, Azure, Claude Platform. No access gotcha.
- Cache reads at $0.01/M under 100K. Repeated system-prompt patterns are nearly free to cache.
What to watch:
- Updated tokenizer adds overhead. Measure before migrating.
- GDPval-AA and AA-Briefcase are Anthropic-internal benchmarks. No independent reproduction yet.
- OSWorld offline subset ≠ full OSWorld. Network-dependent browser tasks may perform differently.
- The 50% cost cut for over-100K tokens is real but smaller than the headline number. Long-context workloads need to do the math separately.
- Cybersecurity safeguards are tighter on Haiku 5.5 than Haiku 4.5. If you were doing security research or offensive tooling with the prior model, expect more restrictions.
- Migrate path: same API endpoints, new model ID
claude-haiku-5-5. Run your eval suite before flipping production traffic. - Measure tokenizer overhead first: run your 10 most representative inputs through both model tokenizers before migrating. The overhead is real, even if small.
- If you're using Sonnet for browser-use at scale, benchmark Haiku 5.5 first. The OSWorld number suggests it handles a significant portion of the cases Sonnet was handling.
- Effort levels: start at
lowfor classifiers and filters; testmediumandhighfor sub-tasks where Haiku was previously unreliable. This is the most underused feature in the release. - For agentic subagent roles, try: orchestrator on Sonnet 5.5, subagents on Haiku 5.5 at medium effort. That's the architecture Anthropic is implicitly suggesting with this release.
- Cache reads at $0.01/M — for pipelines with repeated system prompts, prompt caching is now effectively free.
Further reading
- Anthropic — Claude Haiku 5.5 launch post — all specs, pricing, and benchmarks
- OSWorld benchmark — the computer-use benchmark methodology and leaderboard
- Anthropic model overview — full family comparison
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.
Keep following the thread
A little more to explore.
Claude Haiku 5.5 stopped being the compromise option
Claude Haiku 5.5 landed October 7 at $0.10/M input tokens — a tenfold cut from Haiku 4.5's $1.00/M. Terminal-Bench 4.0 went from 0% to 39.2%. The model tier you routed around is worth reconsidering.
Google built its best model yet. It gave the keys to cybersecurity defenders first. That's actually the right call.
Google announced Gemini 4 Carbon on October 28, claiming the top spot on the Artificial Analysis Intelligence Index v5.0 at score 59 and 88.2% on SEC-bench Pro. Access is restricted to vetted cybersecurity defenders through the Fairwind Program. No pricing. No public API date. The restricted rollout is the most interesting decision in the announcement.
Kimi K3.1 Pro posts frontier coding numbers. The science benchmark is the honesty test.
Moonshot AI shipped Kimi K3.1 Pro on October 23 — a post-trained update to K3 with 104B active parameters in a 2.8T sparse MoE. On Terminal-Bench 4.0 it trades blows with Claude Opus 5.5. On SWE-Marathon it leads. On Terminal-Bench Science 0.1, it scores 38.1% against Opus 5.5's 63.3%. That gap says more about the model than the headline does.