On what Meta's internal Terminal-Bench score actually means, why the $0.10/M contributor tier changes the cost calculation for open-source maintainers, and where Muse Spark 1.2 actually lands against Opus 5
Meta's coding agent entered the market second. The contributor pricing might not care.
Anti-AI
00
Skeptic
01
Neutral
00
Pro (practical)
02
Pro (hyped)
00
← Anti-AI · Pro-AI →
If you're choosing a terminal coding agent today, the top of your shortlist probably has two names. Claude Code, which Anthropic ships directly and benchmarks aggressively. OpenAI Codex, which has been embedded in GitHub Copilot since 2021 and went standalone earlier this year. As of Tuesday, there's a third option — and the reason to look at it isn't the benchmark number that leads the press release.
Meta Superintelligence Labs launched Muse Code on August 5, a terminal-native coding agent for macOS and Linux powered by a new model called Muse Spark 1.2. The announcement came with benchmarks, a pricing structure, and one number that's generating the most comparisons: 82.9% on Terminal-Bench 2.1.
That number needs a footnote before it means anything.
The benchmark question
Terminal-Bench 2.1 is a coding agent benchmark that tests agentic task completion in a real shell environment — file editing, test running, codebase navigation, multi-step refactoring. The public leaderboard is maintained independently and requires submitting your model for evaluation under a standardized test harness run by the leaderboard maintainers, not the model developer.
Muse Spark 1.2's 82.9% is not from that process. Meta ran the evaluation internally, using the Terminal-Bench 2.1 task set, and published the number in the announcement post. That's a meaningful difference. What version of the task set, whether they used the full suite or a curated subset, and whether their harness matched the reference implementation — none of that is published.
Claude Opus 5 sits at 86.7% on the public leaderboard, evaluated by the leaderboard maintainers under the same conditions as every other submitted model. Whether the gap between 82.9% and 86.7% reflects a real capability difference, an evaluation methodology difference, or some mix of both is genuinely unclear from what Meta published.
The DeepSWE v1.1 result is a different story. Meta submitted to the public benchmark and got 59.3%, up from 53% on the prior Muse model version. That's independently verifiable. The directional improvement is real.
| Model | Terminal-Bench 2.1 | Eval source | DeepSWE v1.1 |
|---|---|---|---|
| Claude Opus 5 | 86.7% | Public leaderboard | — |
| Muse Spark 1.2 (Meta) | 82.9% | Meta internal eval | 59.3% |
| OpenAI Codex (latest) | Not published | — | — |
The pricing argument
Here's where Muse Code gets more interesting than the benchmark framing suggests.
Standard pricing for Muse Spark 1.2 is $1.25 per million input tokens, $4.25 per million output tokens. Competitive with Claude Sonnet 3.7, cheaper than Opus 5 — notable, but not a category-level difference.
The contributor tier is a different order of magnitude. Open-source contributors get access at $0.10 per million input tokens and $0.20 per million output tokens. That's roughly a 12× reduction on input, 21× on output, compared to standard pricing.
Why does this matter strategically? Coding agent adoption is sticky. Developers who use Muse Code for their open-source work — because it's dramatically cheaper — are learning its behaviors, its strengths, and its failure modes. When they're evaluating what to recommend or use at a job, that familiarity carries weight that a marketing comparison table doesn't.
The eligibility criteria aren't fully published yet. "Active contributor to qualifying open-source projects" is the announced framing, with specifics to come. The value of the contributor tier depends entirely on whether that definition is generous or narrow.
Source spread
- Meta Superintelligence Labs — Introducing Muse Code and Muse Spark 1.2 [builder] — primary announcement; benchmark claims, pricing tiers, and the contributor eligibility language; does not clarify whether the Terminal-Bench eval used the reference harness
- TechCrunch — Meta's Muse Code takes on Claude Code with a pricing edge for open-source developers [builder] — confirmed launch details; flagged the Terminal-Bench self-report distinction; framing is neutral-to-positive on the market entry
- The Verge — Meta wants to own developer workflows with a surprisingly cheap coding agent [skeptic] — focused on benchmark methodology gap and the unanswered question of contributor tier eligibility at scale; more skeptical on the competitive positioning claim
What's real / What deserves a side-eye
What's real:
- The contributor pricing is a structural market move, not a promotional gimmick. $0.10/$0.20 per million tokens is cheap enough to shift adoption decisions, particularly for developers whose open-source work isn't covered by an employer's tool budget.
- The DeepSWE v1.1 improvement is independently verifiable. Going from 53% to 59.3% on a public benchmark is forward movement, full stop — the absolute number is secondary to the trajectory.
- Meta has the engineering depth to iterate quickly. Muse Code v1 from a lab that has shipped Llama 3, Llama 4, and multiple production inference systems is a different starting point than v1 from a team without that track record.
What deserves a side-eye:
- The 82.9% Terminal-Bench number has not been submitted to the independent leaderboard. The gap with Opus 5's 86.7% could be real, smaller than reported, or larger — you cannot tell from a self-reported evaluation without the methodology details.
- "Active contributor to qualifying open-source projects" is doing a lot of work in one sentence. If eligibility is narrow — think "primary maintainer of a project with 1,000+ GitHub stars" rather than "person who merged a PR last month" — the pricing story becomes much less compelling for most developers who contribute occasionally.
- On every benchmark where a public comparison exists, Muse Spark 1.2 trails Claude Opus 5. That's the actual competitive picture entering a market where the incumbent is both better-benchmarked and established.
What builders need to know
- Treat the Terminal-Bench 82.9% as directional, not definitive. It's a self-reported number from an internal evaluation. Until Meta submits to the independent leaderboard, it's not directly comparable to Claude Opus 5's 86.7%. The DeepSWE v1.1 score (59.3%) is the benchmark to use for cross-model comparisons right now — it's independently verified.
- Check contributor tier eligibility before building pricing models around it. The $0.10/$0.20 rate is a major cost difference, but eligibility criteria aren't fully published. Confirm whether your workflow qualifies before committing to an integration that depends on that price point.
- Run your own codebase evaluation. Muse Code is macOS/Linux terminal-native — same architecture pattern as Claude Code. The questions that matter: Does it navigate your project structure correctly? Does it make the right changes on your specific stack? What's the failure rate on multi-step tasks? Benchmark scores answer none of these.
- Standard pricing is competitive, not decisive. At $1.25/$4.25/M, Muse Spark 1.2 sits between Sonnet and Opus 5 pricing territory. If you're not contributor-eligible, the pricing argument is real but not dramatic. Quality evaluation on your actual work becomes the deciding factor.
- Watch the leaderboard submission. Whether and when Meta submits Muse Spark 1.2 to the public Terminal-Bench leaderboard is the most important signal about competitive positioning confidence. A quick submission is bullish. A delay or a quiet non-submission is informative in the other direction.
Further reading
- Meta Superintelligence Labs — Introducing Muse Code and Muse Spark 1.2 — primary announcement with benchmarks, pricing, and contributor tier details
- TechCrunch — Meta's Muse Code takes on Claude Code with a pricing edge for open-source developers — launch coverage with contributor tier analysis
- The Verge — Meta wants to own developer workflows with a surprisingly cheap coding agent — skeptical framing on benchmark methodology and eligibility questions
Liked this? Get the weekly digest.
Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.
Your take
How'd I do on this one?
What did I miss?
Tell Samwise (and Sam).
Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.