For three years the hardware conversation around AI had one answer: add more accelerators. Prompt processing, token generation, training, fine-tuning - the same chip handled every phase of every workload. That assumption started to come apart this month.
Cerebras and AMD announced a co-engineered serving platform that splits large-language-model inference across purpose-built hardware. AMD Helios racks handle prefill - the pass in which the model reads the entire input prompt at once. Cerebras wafer-scale engines handle decode - the step-by-step generation of output tokens. The partners claim up to five times more tokens per second per watt than a homogeneous stack, with availability in the second half of 2026.
Why the split makes sense
The two phases have genuinely different appetites. Prefill is a matrix-multiplication problem: it processes the whole prompt in one pass and saturates compute units efficiently. Decode reads the full key-value cache on every single step, which makes it bound by memory bandwidth rather than arithmetic - so a chip optimised for raw compute spends most of its transistor budget waiting.
Running both on one architecture means accepting a compromise on each. Splitting them removes that compromise by design. What was a research observation is now a shipping product category, and the Cerebras-AMD platform is unlikely to be the only one.
What it changes for a buyer
The economics of AI in production are increasingly set by inference, not training. A model is trained once; it is served millions of times. Efficiency gains at the serving layer therefore compound in a way that headline training benchmarks never do - and they compound directly into the cost-per-request figure an enterprise compares against a metered cloud API.
The consequence is that hardware stops being a procurement line item and becomes a design decision. The useful question is no longer “how many accelerators?” but “what serving architecture matches my workload profile?” A legal-tech firm summarising long documents has a prefill-to-decode ratio nothing like a manufacturer running real-time visual inspection. Hardware tuned for one over-provisions for the other.
The sovereignty dimension
Disaggregation raises a question that a metered API never has to answer: if prefill and decode run on different hardware, where does the data sit at each stage, and who can see it?
A hyperscaler resolves this by abstraction. The topology is deliberately invisible; the customer buys an endpoint and takes the placement of their data on trust. That is a reasonable trade for workloads where the data is not the sensitive asset.
It is not a reasonable trade for a firm whose process data is the competitive position. There, the topology has to be a stated property of the service rather than an implementation detail - which silicon processes the workload, which jurisdiction holds it at every stage, and how that maps onto the obligations the firm has to its regulator and its customers. Dedicated tenancy makes that answerable, because there is a specific, named boundary to describe.
The operational argument
There is a second effect, less discussed. When the optimal stack was a rack of identical accelerators, self-hosting was largely a purchasing exercise. When the optimal stack is a heterogeneous mix with different power, cooling, and networking profiles, the integration work grows faster than the hardware budget.
That shifts the calculation for firms who were weighing an in-house cluster against a provider. The value of a provider was once mainly capital efficiency. Increasingly it is architectural: having already solved a serving-topology problem that gets harder every quarter, and being able to explain the result to an auditor.
Sovereign infrastructure benefits from this. The complexity that makes disaggregated serving hard to buy off the shelf is the same complexity that makes a well-run dedicated environment worth paying for.