Everyone Needs a SamwiseSubscribe

Bonsai 27B runs on a phone. The on-device AI threshold just got crossed for real.

Jump to a section
See the story at a glance
3.9GB
to run a 27B reasoning model
Bonsai 27B · Apache 2.0 · PrismML, July 14 2026

Every few months a company announces they've fit a large language model on a phone. The parameter count goes up, the press release goes out, and then you actually try running it and it's a 2B distilled model doing 1.5 tokens per second with a knowledge cutoff from eighteen months ago. So when PrismML announced Bonsai 27B on July 14 — twenty-seven billion parameters in 3.9 gigabytes, running on an iPhone 17 Pro — I read the method before I read the headline.

The method is 1-bit quantization applied to Qwen3.6 27B, one of the stronger open-weight reasoning models available. BitNet-style extreme quantization is not new. Applying it to a 27B-class model and releasing the weights under Apache 2.0 is new. Or at least new enough to matter.

What's actually in the release

Two variants.

The 1-bit variant comes in at 3.9GB. This is the one that fits in an iPhone 15 Pro or 17 Pro's 8GB RAM. Across a 15-benchmark evaluation suite — spanning knowledge, reasoning, math, coding, instruction following, tool use, and vision tasks — the 1-bit build retains 90% of the full-precision Qwen3.6 27B's performance.

The ternary (1.58-bit) variant is 5.9GB. It retains 95% of full-precision quality. If you're running on a server or development machine where 2 extra gigabytes of RAM is not the constraint, the ternary variant is the better call. On a phone, the 1-bit version is the only option.

90%
Performance retained vs full precision — 1-bit variant across 15 benchmarks

→ Source: PrismML

iPhone 17 Pro Max runs at 11 tokens per second via MLX Swift. That's the 1-bit variant. On a Mac with an M5 Max, the same variant reaches 87 tokens per second. On an NVIDIA RTX 5090, 163 tokens per second.

Bonsai 27B: inference speed by device
DeviceSpeedRAMNotes
iPhone 17 Pro Max11 tok/s8GB1-bit via MLX Swift; iPhone 15 Pro also supported
Mac M5 Max87 tok/s—1-bit or ternary; good for local dev
NVIDIA RTX 5090163 tok/s—1-bit; self-hosted inference
Together AI APICloud speed—Ternary variant; managed hosting

The license is Apache 2.0. Commercial use is allowed. That's the part that makes this a product story instead of an academic paper.

Source spread

Pros & cons

What's real:

  • 27B parameter class at 90% performance retention in 3.9GB is a real engineering achievement. Previous on-device models in this range were either dramatically smaller in capability (1B, 3B parameter class) or dramatically larger in footprint.
  • The base model is Qwen3.6 27B, which is genuinely capable for reasoning, tool use, and multi-step tasks. Starting from a strong base matters a lot for quantization — you can't quantize your way out of a weak foundation model.
  • Apache 2.0 license means you can build commercial applications without tracking a licensing maze. That's table stakes for production deployment and many open-weight releases still get this wrong.
  • Multi-step reasoning, structured tool use, and long-context workflows are all within scope. PrismML's own announcement specifically lists these as supported capabilities. This is not a stripped-down chat model.

What deserves a side-eye:

  • 11 tokens per second is usable for document analysis, slow-mode assistants, and offline question-answering. It's not usable for interactive coding assistance or anything requiring fast conversational back-and-forth. Know which category your use case sits in before committing.
  • The 90% retention figure is across a 15-benchmark aggregate. The MarkTechPost coverage flags specifically that short-form aggregate benchmarks can mask degradation in complex multi-step reasoning chains. Test on your actual task, not on MMLU-Redux.
  • No published data on sustained thermal behavior on phone hardware. 11 tokens per second under load for two minutes is a different story from 11 tokens per second for fifteen seconds. Battery and heat performance under extended inference is the unknown variable.
  • PrismML is a small independent team. Production longevity, patch cadence, and support continuity are the question marks that come with any independent open-weight release.

What builders need to know

  • Check the speed against your use case before committing. 11 tok/s on iPhone 17 Pro Max works for document summarization, async workflows, and offline assistants. It doesn't work for interactive coding or real-time conversation. Know the difference.
  • For server or laptop deployments, use the ternary (1.58-bit) variant. 5.9GB at 95% retention is a better tradeoff when RAM isn't the constraint. The ternary variant is also hosted on Together AI if you want managed API access without local setup.
  • Run your own benchmarks on the tasks that matter. The 15-benchmark suite is comprehensive but aggregate. Complex multi-step reasoning and adversarial tool-use chains are where quantization gaps tend to cluster. PrismML's numbers are a starting point, not a final eval.
  • Apache 2.0 applies to PrismML's quantization work. The underlying Qwen3.6 27B base model has its own license terms from Alibaba — read those before deploying in regulated industries or at scale.
  • Weights are on HuggingFace at prism-ml/Bonsai-27B-gguf in GGUF format. MLX Swift integration is the path for iOS deployment. Standard GGUF inference runtimes (llama.cpp, Ollama) for everything else.

Further reading

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.

Keep following the thread

A little more to explore.

All articles ↗
Open source5 min read

Meta open-sourced 816 billion parameters. The model is real. The license has a catch.

Meta released Muse Spark Open on October 15 — 816B total parameters, 42B active at inference, a 1M-token context window with multimodal inputs. The benchmarks are competitive. The license requires any shipped product to display a 'Built with Muse Spark Open' label. That's not Apache 2.0. That's a brand distribution deal dressed as open weights.

Read the story ↗
Open source6 min read

Reflection's first model is competitive. It's fifth on Terminal-Bench. The compute-efficiency argument is the interesting part.

Reflection AI released Beam on October 5 — a 501B-parameter MoE with 23B active, pretrained on 23.8T tokens, with a 1M context window. Per Reflection's own benchmarks: Terminal Bench v2.1 at 80.1% (fifth on the September 2026 leaderboard), SWE-bench Verified at 80.9%, GPQA Diamond at 90.5%. Apache 2.0 weights are coming later in October. No date confirmed yet.

Read the story ↗