DeepSeek V4 Flash: when model, silicon and serving align

Strip an AI product down to what it actually makes and the answer is tokens. Not any tokens: tokens good enough to advance real work, delivered fast enough to keep a person in flow or an agent unblocked, at a price that lets you leave the thing running all day. NVIDIA’s name for the plants that produce them is AI factories, and the industry problem of the next few years is factory efficiency: how well raw GPUs and energy convert into value for the people using them.

August handed us the cleanest case study of that problem we have had. DeepSeek released the official V4 Flash checkpoint on July 31. We had it serving community seats in a Labs experiment within a day, and on August 3 it entered the pay-per-token lineup as umans-deepseek-v4-flash-0731, our cheapest production model and the default for new chats and fresh CLI setups.

What makes it a case study is alignment. Three layers that usually pull against each other lined up: a model architecture engineered for serving efficiency, a generation of silicon that the architecture’s own designers cannot access, and the serving engineering that connects the two. When they align, the numbers stop looking incremental: single sessions decoding around 600 tokens per second, a node sustaining 95,000 tokens per second when full, a price of $0.14 per million input tokens. As far as we can find, nobody serves this model faster today, and the useful part of that sentence is not the ranking. It is that the result follows from the alignment, not from anything exotic on our side, which means it says something about where the whole field is going.

The model: an executor, priced like one

The benchmark story was loud from day one. On Artificial Analysis (August 2026), V4 Flash scores 69.1 on the coding index and 45.7 on the agentic index, shoulder to shoulder with GLM 5.2 at 68.8 and 43.1, a model many times its active size. LMArena’s cost-performance view puts it on the Pareto frontier. You also know we distrust benchmark stories: some of the most benchmarked models of the past year were the most disappointing to work with. So we did what we always do, we put it in front of people with real work and asked them to be blunt.

deepseek so far does not dissapoint btw

yo deepseek actually just got my code to run on the gpu

how deepseek went from my bottom to top of the list like that is crazy

The character that emerged, and our own daily use agrees, is a strong executor. Give it a well-framed task and it goes, fast, without padding. It has less self-direction than GLM 5.2, which testers felt on open-ended work, and the pattern that gets the most out of it is a planner-executor split: a stronger model frames the change, Flash does the writing. One tester runs exactly that, Kimi K3 as the architect and Flash as the hands:

basically the translation from english to code is still done by deepseek, and it still wont interpret and write it like a k3 would

That split is not a workaround, it is the model’s shape, and it is a genuinely useful one. On the agentic index K3 sits clearly above both Flash and GLM (50.1 against 45.7 and 43.1), and it costs accordingly. An executor this fast and this cheap changes what you delegate to the expensive model: the framing stays where the judgment is, the keystrokes move to where the tokens cost a tenth as much.

The rest of the ecosystem reached the same verdict in days. Within the first week, OpenRouter’s public activity page for the checkpoint was approaching one and a half trillion tokens a day.

An architecture built under constraint

If you read our V4 Pro write-up from July, you know why this family interests us. DeepSeek V4 attacks serving cost at the layer with the most leverage, the model itself. Attention reads a compressed representation of the past instead of every token in it, so the KV cache and the per-token compute stop growing with the conversation. The DSpark module drafts ahead of the model and lets the checkpoint verify whole blocks in single passes, filling the idle time that dominates decoding. V4 Flash is the small sibling of that design: 284B total parameters, 13B active, a 1M-token context, the draft module built into the checkpoint. At a million tokens of context it runs at roughly 10% of the single-token FLOPs and 7% of the KV cache of DeepSeek’s previous generation (technical report, §2.3).

There is a reason the design is this disciplined. DeepSeek engineered V4 under hardware constraints: the current generation of NVIDIA data-center GPUs is not available to them. When your silicon is capped, the only place to buy efficiency is the architecture, and they bought as much as anyone ever has. The whole open-weight field is adopting the same ideas, we counted the KV reductions across Kimi, Qwen, MiMo and Ring in the July piece, and that is good news for everyone who serves models: the efficiency ships inside the weights now.

It also sets up a happy asymmetry. An architecture designed to squeeze constrained hardware does not stop paying off when the constraint is lifted. Put it on unconstrained silicon and the discipline turns into headroom.

What the serving layer adds

That headroom does not collect itself. We brought V4 Flash up on three node shapes, 8x B200, 8x B300 and a lean 4x B200, in unified and prefill/decode-disaggregated layouts, with DSpark on throughout. The shape that carries production today is a single 8x B200 node split down the middle: four GPUs run prefill, four run decode, and a session’s KV state is handed from one side to the other over NVLink. The handoff is not the bottleneck people assume: an active copy of 1.72 GB of KV for a settled 446,000-token session takes 6.5 milliseconds.

Disaggregation matters for agentic traffic because prefill and decode want opposite things from a GPU. Prefill is compute-bound and bursty, decode is memory-bound and steady. Mixed on the same GPUs, a heavy prompt stalls everyone’s token stream. Split, each side runs its own batch discipline, and long prompts queue against other long prompts instead of against your keystrokes.

The architecture’s two halves then do exactly what they were designed to do. In our gate runs, 59% to 66% of DSpark’s drafted tokens survive verification in accepted runs of about four, which is where most of the single-stream speed comes from. And the compressed KV cache is small enough to tier aggressively: what is decoding right now lives in HBM, a multiple of it lives in host memory, and a returning session restores its prefix and computes only the delta. A 71,000-token session that comes back for its next turn reuses 99.8% of its prompt from cache. A 446,000-token session appending a few hundred tokens sees time-to-first-token of about a second and a half.

The engine is SGLang with a pinned upstream branch and a handful of our own patches, the kind that only surface when you run a shape in production: keeping prefill hidden states alive for the draft module’s KV transfer, fixing a radix-cache boundary on warm follow-ups, making the reasoning parser and the prompt encoder agree about thinking mode. We send this class of fix upstream rather than carrying it privately, same policy as last time.

The frontier, measured

Numbers, then how to read them. Cold cache, 8K input and 1K output per request, three rounds per point, every request completing:

This is the throughput-interactivity frontier we wrote about in July, the trade every serving stack lives on: batch more users and each one waits longer per token. What alignment buys is the slope. Between 1 concurrent session and 256, aggregate output grows 25x while per-session decode gives up about 30%, from 593 to 410 tokens per second. Sessions stay fast when the node is full, which is precisely the property that lets a serving stack say yes to more users without making anyone regret it.

Two honest notes on reading the table. TTFT does climb with batch in a cold benchmark; production does not look like that, because production is warm. On a public agentic workload replay, warmed to the cache levels agentic traffic actually has (we measured 96% prefix hits on this trace), the node completed 2,360 requests at 128 concurrency with zero errors, a TTFT of 3.3 seconds p50, and 8,195 output tokens per second. And these are our numbers on our shape; your traffic will move every figure.

The other thing the table does not show is that none of it was true on day one. Our first pass at that agentic replay ran at 38.6 seconds TTFT p50. It reached 3.3 through a sequence of unglamorous serving fixes, and the improvement shows up where users can see it: the model’s public status page has time-to-first-token improving from 4.25 seconds to 1.22 since launch, published continuously, good days and bad. Serving craft compounds, and the trend line is still moving.

For calibration, DeepSeek’s own DSpark paper frames its production frontier with 50 tokens per second per user as the demanding end of the range, on the hardware available to them, at a scale far beyond ours. On current silicon, with the stack above, our median session at full batch decodes at eight times that mark, and a single session at 593. We looked for anyone serving this checkpoint faster and have not found them yet. We expect that to change, the weights are open and the techniques are public, and the point of this article is that the result is reproducible, not that it is ours.

Tokens per GPU is the whole game

Here is an honest observation about a 600-token-per-second coding model: in a long agentic run, it is often not the bottleneck. The model finishes its turn and waits on a test suite, a build, a tool call, you reading the diff. Interactive work feels the speed on every exchange, and it is a real pleasure. Autonomous work amortizes it.

So why does the speed matter? Because on a well-built stack, per-session speed and capacity per GPU are the same number wearing two hats, and that number is the one the business runs on. NVIDIA’s formulation:

Cost per million tokens = (cost per GPU per hour) / (tokens per GPU per second × 3,600) × 1,000,000

The numerator is rent, and everyone pays roughly the same rent. The leverage is all in the denominator, and every layer of the alignment divides it: compressed attention fits more sessions on the silicon, DSpark turns idle decode time into verified tokens, disaggregation keeps prefill from starving decode, cache tiering makes a returning session cost its delta instead of its history. The node above delivers about 1,300 output tokens per GPU per second, 11,900 counting the full prefill-plus-decode stream. The same denominator divides the energy: joules per token falls exactly as fast as dollars per token. Put your own GPU rate in the numerator, multiply by whatever redundancy your availability target demands, and the economics of serving this model are on one line of arithmetic.

That arithmetic is what sets the price: $0.14 per million input tokens,$0.028 cached, $0.28 per million out, the cheapest model in our lineup and the default one. We are self-funded, so this is not generosity, it is the denominator passed through. The most satisfying review of the whole system was not about tokens per second at all:

i’ve used so much dsv4f and i’ve only gone through like a dollar in tokens

That is the factory working: GPUs and energy in, a week of real work out, a dollar on the meter.

What is next

V4 Flash stays: generally available since August 3, default model, at the prices above. The Labs experiment that started it handed its seats on August 11 to the next one, a vision variant of this exact checkpoint that we trained ourselves, umans-ai/DeepSeek-V4-Flash-0731-Vision, because “anything visual k3 just steps ahead of deepseek” was the one gap testers kept naming. That experiment gets its own wrap-up, and so does the Kimi K3 lab that ran just before this one. DeepSeek published the V4 Pro 0813 weights while we were writing, so the pipeline that turned the 0731 weights into a production model in three days is already running again.

The alignment keeps improving from all three directions at once: the architectures arrive more efficient, the silicon gets denser, and the serving layer between them is where we spend our days. Every fix in this article landed after launch, and the public trend lines are the residue. That is the part we would like you to take away. The frontier is not a fixed property of the hardware; it is a property of how seriously the stack above it takes the denominator, and it moves if you keep pushing.

Thanks to the lab testers who threw real work at a day-old checkpoint and told us exactly what kind of model it is. The executor-not-architect read is theirs, and it is the most useful sentence anyone wrote about this model all month.

References

• DeepSeek-V4-Flash-0731 weights. The official release: 284B total parameters, 13B active, 1M context, DSpark module included. What we serve, unmodified.
• DeepSeek-V4 technical report. Compressed Sparse Attention and Heavily Compressed Attention, and the FLOPs and KV curves behind the capacity story.
• DSpark: load-aware speculative decoding. The draft module and DeepSeek’s own production frontier, our calibration point.
• Rethinking AI factory economics. NVIDIA on cost per token, and the formula this article is built around.
• Artificial Analysis and LMArena’s frontier post. The August 2026 index scores and the cost-performance frontier.
• OpenRouter’s activity page for the checkpoint. The ecosystem adoption curve.
• Our live metrics for umans-deepseek-v4-flash-0731. TTFT and throughput, published continuously.
• SGLang. The engine under the node, and where our serving patches are headed.
• DeepSeek V4 Pro DSpark: the model isn’t ready, the architecture is. The July experiment where we learned this architecture on live traffic.