NVIDIA is still the big boss of AI GPUs, but a new Evercore ISI note suggests the conversation inside data centres is changing fast.
According to the analysts, AI engineers are not simply buying into NVIDIA’s usual performance-efficiency argument. The company’s CEO Jensen Huang has repeatedly defended NVIDIA’s premium pricing by pointing to better performance and efficiency versus rivals. But Evercore says engineers are now paying closer attention to less glamorous, very real costs: power draw, cooling, utilisation, total ownership cost and cost-per-token.
That last one matters a lot. As AI workloads shift from heavy model training to inference — the part where models actually generate answers, images, code or summaries — buyers are starting to judge chips differently. Instead of only asking which hardware has the biggest raw throughput, they are asking: how much does it cost to generate a million tokens?
For Malaysia and SEA, this is not just some Wall Street GPU drama. If cloud providers, AI startups and enterprise platforms can lower inference costs, it may eventually affect the pricing of AI tools used by local businesses, creators, developers and game studios. Cheaper backend compute could mean more affordable AI features in apps, faster local services, or lower costs for companies building AI products in the region.
The debate comes after Morgan Stanley reportedly argued that a data centre built with NVIDIA Blackwell GPUs could cost twice as much as one using custom AI chips, but that Blackwell may deliver up to eight times better performance per watt. On paper, that sounds like a strong defence for NVIDIA.
Evercore’s view is more complicated. Its checks suggest that many AI engineers are not fully convinced by headline performance claims, especially when NVIDIA’s high margins are seen as excessive. The note says the market’s move toward inference is changing buying criteria from maximum bandwidth and throughput to cost-per-token, power, cooling, utilisation and total cost of ownership.
In simple terms: if a custom ASIC, AMD accelerator, Google TPU, Amazon Trainium, Microsoft Maia or another “good enough” chip can run inference at a lower cost, some buyers may choose economics over peak NVIDIA performance. Memang practical lah — data centre operators do not care about flexing benchmark charts if the electricity and cooling bill sakit.
Evercore also cited comments from an expert at AI infrastructure provider Nebius, who said inference demand can make up as much as 95% of enterprise workload use cases. That expert also pointed to Groq chips being preferred in some cases because of higher throughput.
The bigger warning from Evercore is that NVIDIA’s share of inference workloads could fall to around 50% by 2028 as alternatives improve. That does not mean NVIDIA is suddenly finished — far from it. The company still has the ecosystem, software stack and hardware lead that made it the default choice for AI infrastructure.
But the market is maturing. Training giant models is one game. Running them cheaply at scale for millions of users is another. And once hyperscalers start optimising for every watt, every cooling rack and every generated token, custom silicon becomes much more tempting.
No consumer RM pricing is involved here, since these are enterprise data centre chips rather than retail GPUs. But Malaysian tech readers should still watch this closely: the cheaper AI inference gets globally, the faster AI-powered services can become normal, affordable and competitive in our own market.
Source: Wccftech Gaming