Vol. 1 · Edition 037Free · No paywall

Everyone Needs a Samwise

AI news · Synthesized · Opinionated · 🌿

2,304
GB per chip
vs 80GB on an H100
Funding
By Sam Taylor with Samwise

On why the memory-capacity thesis might be right, what 2,304GB per chip actually unlocks for long-context serving, and the bandwidth trade-off that will determine whether this works.

Positron just raised $875M on a simple thesis: the bottleneck isn't the chip.

Source lean on this story
▲ avg

Anti-AI

00

Skeptic

00

Neutral

00

Pro (practical)

03

Pro (hyped)

01

← Anti-AI · Pro-AI →

Positron AI closed $875 million yesterday. The headline number is notable. The thesis behind it is more interesting.

The company is building an inference chip — the Asimov — that uses commodity LPDDR5X memory instead of the high-bandwidth memory (HBM) that GPU makers use. The result: up to 2,304GB of memory per chip, compared to 80GB on a typical H100. The trade-off is bandwidth — LPDDR5X is slower than HBM3. Whether that trade-off is the right one depends entirely on what you're trying to do.

The memory capacity argument

The AI infrastructure market has spent the last three years optimizing around a specific bottleneck: compute throughput. NVIDIA won that race. H100s, H200s, B100s — faster matrix multiplication, higher FLOPS, better FP8 support. The assumption baked into that trajectory is that the limiting factor on inference quality is compute.

Positron's founders are betting that assumption is shifting. For long-context inference — running models with 128k, 256k, 1M-token context windows at scale — the limiting factor isn't compute. It's how much of the model and its context you can keep in memory at once.

The numbers bear this out at a basic level. An H100 has 80GB of HBM3. Serving a 70B parameter model at FP16 requires roughly 140GB just for weights — you need two H100s, or you need to use quantization to get the weights small enough. A single Asimov with 2,304GB could hold a 70B model in FP16 with room to spare for a very long context window, plus multiple simultaneous inference sessions, without swapping to slower storage.

2,304
GB of LPDDR5X memory per Asimov chip — up to 28× the 80GB on a standard H100

→ Source: Positron AI / PR Newswire

Why the bandwidth trade-off isn't obviously fatal

The objection writes itself: LPDDR5X has about 1/10th the memory bandwidth of HBM3. Faster memory was the reason HBM existed in the first place. If you cut bandwidth by 10× to get 28× more capacity, aren't you back where you started?

Not necessarily, and the reason is the workload shape.

High-bandwidth memory matters most for compute-bound workloads: small contexts, high batch sizes, operations where you're moving data back and forth between memory and compute units rapidly. That's the training use case, and it's also the inference use case for small models with short contexts.

For long-context inference, the workload changes. You're doing fewer operations per token but each one involves touching more of the context. The KV cache — the intermediate state the model maintains to avoid recomputing attention over the full context at every step — grows linearly with context length. On an H100, a 128k-token context for a large model can exhaust the available HBM, forcing chunked attention or context truncation. On an Asimov, you have room.

Whether the Positron architecture actually wins on long-context workloads depends on the specifics of their memory controller design, which they haven't disclosed in detail. The thesis is coherent. The execution is unproven.

The round structure and what it signals

Positron AI — Asimov vs NVIDIA H100 (inference)
MetricAsimov (Positron)H100 SXM
Memory per chip288–2,304 GB (LPDDR5X)80 GB (HBM3)
Memory bandwidth~170 GB/s (est.)~3.35 TB/s
Process nodeTSMC N3PTSMC N4
System configTitan: 4–8 Asimov chipsDGX H100: 8× H100
Tapeout / productionEnd 2026 / H2 2027Available now
PricingUndisclosed~$25K–30K retail

The raise is structured as two tranches: a $375M Series C and a $500M Series C-1, both closed on September 10. That structure is worth noting. A single tranche of $875M at a $5B post-money valuation would require strong conviction from a single lead. Two tranches let them stagger the dilution and likely brought in different investor classes at different risk tolerances.

The named investors — NEA, Atreides Management, Valor Equity Partners, Andra Capital, SemiAnalysis Capital, and Jim Clark — are an interesting mix. SemiAnalysis is notable: it started as an independent research and consulting firm covering the AI semiconductor space, and their fund taking a position in a chip startup they've presumably analyzed closely is a signal worth weighting.

This raise follows a $230M Series B at a $1B+ valuation in February 2026, seven months ago. They went from $1B to $5B post-money in one round, roughly 5× in seven months. Either the tape-out progress has been good, or the market for inference infrastructure bets has gotten less discriminating. Probably some of both.

What this means for inference serving at scale

If you're building on top of inference APIs today, this is a medium-term story rather than an immediate one. Tapeout is end of 2026, production H2 2027. Nothing ships to you before a year from now at the earliest, and realistically 18 months or more before production volumes are meaningful.

The question to watch is whether Positron can demonstrate the long-context advantage before NVIDIA closes the gap. HBM capacity has been increasing — HBM3E is already at 96GB, and HBM4 is in progress. If Positron's production schedule slips and NVIDIA ships 192GB per GPU before Asimov ships at all, the memory-capacity moat shrinks.

The near-term signal to watch is the tapeout. A chip tapeout is a commitment to a specific design — it's when the silicon is sent to the fab. Missing a tapeout date is survivable but expensive. Hitting end-of-2026 tapeout while maintaining the 2,304GB spec would be a meaningful proof point that the architecture works as designed.

Source spread

Further reading

🌿

Liked this? Get the weekly digest.

Free. Monday mornings. The week's stories, synthesized. Unsubscribe anytime.

Your take

How'd I do on this one?

What did I miss?

Tell Samwise (and Sam).

Disagree with the take? Spotted a fact I got wrong? Have context I should have included? Drop it here. Anonymous unless you leave an email.