Dense or mixture of experts: what the architecture costs the buyer

Two words decide the shape of a private AI server: dense, or mixture of experts. A buyer meets them on a spec sheet, with no way to tell what either costs. The honest answer is in the model's own configuration file, which every open-weight vendor publishes, and it takes about ten minutes to read.

Concede the headline first. A mixture-of-experts model is a genuine memory win, and for high-volume, low-complexity work it is the better buy. The question is how big the win is, and where it stops.

What the two words mean

A dense model reads every weight it has to produce one token. A mixture-of-experts model splits most of its weights into experts and reads only a few of them per token. Qwen3.6-35B-A3B publishes 256 experts and 8 experts per token. So it carries 36 billion parameters and reads about 3 billion at a time.

That is the promise: the memory bandwidth cost of a small model, with the knowledge of a large one.

The two configuration files, read on the same day

Both files were read on 2026-09-23. They are the vendor's own, machine-readable, and they hold every number that decides capacity.

Three of every four layers in both models hold a fixed recurrent state instead of a growing cache. Only the full-attention layers pay per token. That is why the layer count alone tells you nothing.

The cache arithmetic, from those numbers

Each full-attention layer stores a key and a value for every token. The cost of one token is 2, for the key and the value, times the full-attention layers, times the key-value heads, times the head dimension, times the bytes per number.

The MoE holds under a third of the cache per conversation. It wins twice: it has 10 full-attention layers against 16, and 2 key-value heads against 4. Memory is the seat constraint on a single card, so this is the part of the promise that holds.

Where the promise starts to fail

The weight saving is the other half of the promise, and it is the half that does not survive a crowd.

Eight experts per token is true for one token. A server answers many people in one batch, and the batch has to read every expert that any of its tokens routed to. With 256 experts and 8 per token, 32 tokens can already reach all 256. Above that batch size the model reads something close to its whole expert set on every step, and the saving that justified the architecture is gone.

This is the reason a MoE throughput figure quoted from the single-token case is not a capacity figure. It describes one person using the machine.

The quality side of the trade

Each vendor publishes its own model's scores on its own model card. Both cards were read on 2026-09-23, and two benchmarks appear on both.

The two cards compare against different sets of rival models, so read each number as the vendor's own claim for its own model, not as a controlled head-to-head. Even read that way, the gap on both coding benchmarks is about ten points.

What this means for a buyer

The choice is not which architecture is better. It is which of the two constraints binds in your building.

• Many people, simple work: classification, summarising, document question answering, overnight batches. The MoE wins on memory and the quality gap costs you little.
• Fewer people, hard work: drafting, analysis, code, research. The dense model wins, and the seat count is rarely the thing the department feels.
• Ask any vendor for a capacity figure with four terms attached: how many concurrent users, at what context length, at what tokens per second each, with what cache precision. A number missing any of the four cannot be checked.
• For an MoE quote, ask one more thing: at what batch size was it measured. A figure taken at one request tells you nothing about thirty.