AI explained. Tools tested. Ideas worth trying.By Sam Taylor & Samwise ↗

Haiku 5.5 is 75% cheaper and now actually good at browser tasks. Small-model economics just changed.

Jump to a section
See the story at a glance
72.4%
OSWorld 2.1 score vs 15.7% for Haiku 4.5
Anthropic · October 7, 2026

Anthropic released Haiku 5.5 on October 7. The headline is 75% cheaper on average versus Haiku 4.5, which is already notable. But the number I keep coming back to is the OSWorld score.

Haiku 4.5 scored 15.7% on OSWorld 2.1's offline subset. That's a score that says: you can try to use this model in a browser-automation loop, but most of the time it will fail. Haiku 5.5 scores 72.4%. That is not an incremental improvement. That is a different product category.

What the benchmarks actually say

All numbers from Anthropic's launch post:

  • OSWorld 2.1 (offline subset): 72.4% vs 15.7% for Haiku 4.5
  • Terminal-Bench 4.0: 39.2% vs 0.0% for Haiku 4.5 — the model couldn't do terminal tasks at all before
  • Humanity's Last Exam (no tools): 45.9% vs 10.2%
  • GDPval-AA v2.1: 1620 vs 735 (Anthropic's internal agentic eval suite)

Pricing structure worth understanding carefully. Two tiers based on request length:

  • Under 100K tokens: $0.10 per million input, $0.50 per million output — 90% cheaper than Haiku 4.5
  • Over 100K tokens: $0.50 per million input, $2.50 per million output — 50% cheaper than Haiku 4.5

The 75% average reduction Anthropic headlines is a blended figure across both tiers. If most of your calls are under 100K tokens, the actual savings are closer to 90%.

90%
Cost reduction vs Haiku 4.5 for requests under 100K tokens (the majority of most workloads)

→ Source: Anthropic

The effort level thing

First Haiku-class model with adjustable effort. That is the feature I will actually use day-to-day.

If you've been building multi-agent workflows, you've probably been doing one of two things: either using Haiku everywhere because it's cheap (and accepting a high failure rate on harder sub-tasks), or using Sonnet everywhere and absorbing the cost. Haiku 5.5 gives you a third option: a Haiku you can turn up.

Anthropic's positioning is explicit — Haiku 5.5 as a subagent alongside Opus 5.5 and Sonnet 5.5 orchestrators. The model handles repetitive sub-tasks (summarization, classification, form-filling, short reasoning) at $0.10/M while the orchestrator runs on a more expensive model for planning.

One genuine gotcha: the tokenizer is updated. Haiku 5.5 uses slightly more tokens per equivalent input than Haiku 4.5 did. Anthropic doesn't publish a specific overhead percentage in the release post. Worth running your representative inputs through the tokenizer before assuming the 90% cost cut is fully realized. For most workloads the overhead is small. For very tight latency/cost budgets, measure it.

What I think the OSWorld number actually means

72.4% on OSWorld 2.1's offline subset. Let me be precise about what that is and isn't.

OSWorld is a well-established computer-use benchmark — the standard in the field for testing whether a model can navigate a GUI, fill forms, open applications, respond to screen state. The "offline subset" removes network-dependent tasks (tasks that require a live internet connection to complete). Full OSWorld numbers are harder, and Anthropic doesn't publish those.

So 72.4% means: on the tasks OSWorld can evaluate without a live internet connection, Haiku 5.5 succeeds almost 3 out of 4 times. That's actually usable. Haiku 4.5's 15.7% was not.

Or maybe I'm being too optimistic. The offline subset is easier than the real-world online version. Tasks that require dynamic web content, login flows, or navigation across pages might perform differently. Before I redesign a system around this benchmark, I'd run it against my actual browser-automation use cases.

The Terminal-Bench number is the one I find more decisive. Terminal-Bench 4.0: 39.2% vs 0.0%. Zero. Haiku 4.5 literally could not do terminal tasks. This isn't a regression from 30% to 39% — this is a model that gained a capability class it didn't have before. For CLI automation, scripting, and server-side agent work, that matters.

Haiku 5.5 vs Haiku 4.5: key benchmarks
BenchmarkHaiku 4.5Haiku 5.5Δ
OSWorld 2.1 (offline)15.7%72.4%+56.7pp
Terminal-Bench 4.00.0%39.2%+39.2pp
Humanity's Last Exam10.2%45.9%+35.7pp
GDPval-AA v2.17351620+120%

Source spread

Pros & cons

What's genuinely useful:

  • 90% cost reduction for sub-100K requests is a primary-source number. For high-volume classification, summarization, or short-context pipelines, this changes the economics.
  • OSWorld 72.4% makes browser-use workflows viable at Haiku pricing — previously you needed Sonnet or higher for reliable computer-use.
  • First effort levels on Haiku means per-request cost control without model-swapping.
  • Available everywhere on day one: AWS, GCP, Azure, Claude Platform. No access gotcha.
  • Cache reads at $0.01/M under 100K. Repeated system-prompt patterns are nearly free to cache.

What to watch:

  • Updated tokenizer adds overhead. Measure before migrating.
  • GDPval-AA and AA-Briefcase are Anthropic-internal benchmarks. No independent reproduction yet.
  • OSWorld offline subset ≠ full OSWorld. Network-dependent browser tasks may perform differently.
  • The 50% cost cut for over-100K tokens is real but smaller than the headline number. Long-context workloads need to do the math separately.
  • Cybersecurity safeguards are tighter on Haiku 5.5 than Haiku 4.5. If you were doing security research or offensive tooling with the prior model, expect more restrictions.
For builders
  • Migrate path: same API endpoints, new model ID claude-haiku-5-5. Run your eval suite before flipping production traffic.
  • Measure tokenizer overhead first: run your 10 most representative inputs through both model tokenizers before migrating. The overhead is real, even if small.
  • If you're using Sonnet for browser-use at scale, benchmark Haiku 5.5 first. The OSWorld number suggests it handles a significant portion of the cases Sonnet was handling.
  • Effort levels: start at low for classifiers and filters; test medium and high for sub-tasks where Haiku was previously unreliable. This is the most underused feature in the release.
  • For agentic subagent roles, try: orchestrator on Sonnet 5.5, subagents on Haiku 5.5 at medium effort. That's the architecture Anthropic is implicitly suggesting with this release.
  • Cache reads at $0.01/M — for pipelines with repeated system prompts, prompt caching is now effectively free.

Further reading

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.

Keep following the thread

A little more to explore.

All articles ↗
Models5 min read

Claude Haiku 5.5 stopped being the compromise option

Claude Haiku 5.5 landed October 7 at $0.10/M input tokens — a tenfold cut from Haiku 4.5's $1.00/M. Terminal-Bench 4.0 went from 0% to 39.2%. The model tier you routed around is worth reconsidering.

Read the story ↗
Models6 min read

Google built its best model yet. It gave the keys to cybersecurity defenders first. That's actually the right call.

Google announced Gemini 4 Carbon on October 28, claiming the top spot on the Artificial Analysis Intelligence Index v5.0 at score 59 and 88.2% on SEC-bench Pro. Access is restricted to vetted cybersecurity defenders through the Fairwind Program. No pricing. No public API date. The restricted rollout is the most interesting decision in the announcement.

Read the story ↗
Models6 min read

Kimi K3.1 Pro posts frontier coding numbers. The science benchmark is the honesty test.

Moonshot AI shipped Kimi K3.1 Pro on October 23 — a post-trained update to K3 with 104B active parameters in a 2.8T sparse MoE. On Terminal-Bench 4.0 it trades blows with Claude Opus 5.5. On SWE-Marathon it leads. On Terminal-Bench Science 0.1, it scores 38.1% against Opus 5.5's 63.3%. That gap says more about the model than the headline does.

Read the story ↗