Can AMD’s MI350X and MI355X GPUs Close the Gap with Nvidia?

Posted on by io-fund

This article is a continuation of our free newsletter from June 19, AMD vs Nvidia: The AI Stock That Could Win by 2028

Find out the following below: 

  • We compare AMD’s MI350X and MI355X with Nvidia’s B200s and GB200s to decipher if AMD has what it takes to close the gap with the AI leader 
  • Clear conclusions on the next 1-2 years that are tailored for stock investors and how we plan to position our portfolio  
  • The SKU that all investors should know about 

Last week, AMD introduced its Instinct MI350 series GPUs, including MI355X with up to 4X performance over the previous MI300X generation and up to 40% more tokens per dollar compared to Nvidia’s B200 accelerators. The company also previewed its Helios rack-scale server architecture featuring the MI400s for 2026 deployments. 

According to Tom's Hardware AMD is claiming the eight-GPU MI355X system is 1.3X faster than Nvidia’s DGX GB200s systems with Llama 3.1 and up to 1.2X faster than the B200 HGX systems in inference for DeepSeek R1 with equivalent performance as Llama 3.1 when tested at FP4. 

Here are a couple of key points in terms of how AMD is starting to close the gap with Nvidia for inference purposes: 

Floating point precision: 

AI accelerators are increasingly offering lower floating-point formats to help reduce memory consumption and bandwidth requirements, which in turn speeds up computation and lowers power consumption. For example, FP8 delivers better throughput and energy efficiency in LLM inference compared to FP16. The newer generations of GPUs will offer FP4 formats to further alleviate memory-bandwidth bottlenecks and improve performance for large matrix operations. 

I elaborated on the importance of floating-point precision in my analysis “Here’s Why Nvidia Will Reach a $10 Trillion Market Cap” when I stated: “The difference is that the smaller bit size allows for an economical way to achieve more speed when giving up a small amount of accuracy doesn’t make a critical difference. As discussed, this also helps in the face of a slowing Moore’s Law.” 

With the MI350X and MI355X, AMD is introducing FP4 along with the smaller formats of FP8, FP6 and FP4, which are especially helpful for inference. In the CDNA 4 architecture, the FP6 data rate shares the same peak PFLOP/s as FP4 — which for inference purposes means it will be comparable to or slightly exceed Nvidia’s B200s.  

ServetheHome states, “AMD is doing the higher performance (at a transistor cost) option of adding FP6 to the FP4 pipeline to give it a big boost.” 

Source: Tom’s Hardware, pictured above – FP6 performance is on par with FP4 performance.  

HBM3E and HBM4 Memory: 

AMD is attempting to compete on memory by slightly beating Nvidia with the MI355X having 1.6x more memory capacity than the B200s. This allows AMD to load full model weights into memory for fast inference and avoids having to share resources between multiple GPUs. The higher amount of memory also increases the batch size, which increases throughput while lowering latency.  

It’s important to keep in mind that Nvidia is preparing to send a shockwave through the AI market, once again, with its NVL72 and NVL36 systems. These systems combine 72 GPUs and 36 GPUs to think like one GPU, which I’ve covered recently here.covered recently here. 

Rather than AMD taking head-on Nvidia’s NVL72s and NVL36s right now – which are earth-shattering SKUs — the company is instead attempting to compete at the 8-GPU system level. Memory is a big part of that attempt. Inference craves low latency, thus having the model fit entirely in memory for inference purposes is a part of that strategy.  

What’s Important About the MI350X and MI355X: 

To put it plainly, on the AI accelerator front, this will be the first time that AMD will overlap Nvidia in terms of benchmarks on GPUs. Please do note, the amount of time that AMD’s current generation of GPUs and Nvidia’s GPUs overlap will be brief – and will only be at the single GPU and 8-GPU system level. AMD was originally expected to ship the MI350s at the end of this year yet are moving the shipments up – which fits with AMD’s tradition of underpromising and overdelivering.  

However, the accomplishment is noteworthy as it’s setting the tone as the inference market begins to ramp. In other words, AMD ceded the training market to Nvidia – but I do not expect that to be the case with the inference market. 

When Blackwell Ultra ships, the B300s will offer FP4 TFLOP/s that is 1.3X faster than AMD’s current MI350X and MI355X. With that said, because AMD has prioritized competing on memory — its bandwidth and capacity is expected to be on par with Blackwell Ultra. 

AMD’s CDNA 4 Architecture: 

The primary architectural changes of CDNA 4 were aimed at increasing memory capacity and bandwidth per compute unit. The lower precision compute capacity was also increased, favoring FP6 and FP4. 

AMD’s architecture is built on a chiplet design, and similar to the Zen-2 architecture discussed above, the chiplet design offers power efficiency improvements from monolithic designs by offering a dozen chiplets on a single processor.  

Although monolithic used to be preferable, to compare, Nvidia’s has evolved its architecture to utilize multi-die modules (MCM) which combines two reticle-limit dies. By utilizing high bandwidth connections, the two dies function as a single die to forego reticle-size constraints, helping to improve yields and results in higher performance. 

However, keep in mind that AMD was first to market with chiplets in the Zen architecture that helped stage the company’s comeback. Nvidia is the world’s best AI semiconductor design company, yet the point is that AMD is not necessarily a follower. In some design areas, AMD leads. 

A few more things to highlight from last week’s announcements: 

  • 3D packaging with CoWoS-S from the MI300s remains with XCDs, HBM3 memory, I/O Dies and the Infinity cache 
  • There are a total of 256 compute units with eight 32 CDNA per XCD. This is less than the last generation yet with the 3nm, each compute unit delivers more power 
  • There are two larger I/O Dies rather than four for better efficiency. The I/O Dies are built on a 6nm process. 
  • More memory at 288GB of HBM3E with 8TB/s 

MI400s “Helios” Will Close the Gap on Larger AI Clusters 

The market is forward-looking, which means investors should be too. AMD is closing the gap on single GPUs and 8-GPU systems, yet the MI400s will mark a pivotal moment as AMD will attempt to compete on rack-scale systems with Helios, its 72-GPU systems. If things go as planned, AMD will be competitive with Nvidia on GPU, memory and interconnect performance — while potentially taking the lead on memory capacity and bandwidth.  

By using UALink and potentially Broadcom’s scale-up ethernet, AMD will be able to deliver considerable bandwidth, with projections of 31 TB of HBM4 memory and 1.4PB/sec of bandwidth, which would beat Nvidia’s offerings by 50%. 

UALink, or Ultra Accelerator Link, is an open industry-standard interconnect that enables high-speed and low-latency communication for AI clusters. This is a joint venture between a consortium of Nvidia competitors, including AMD, Intel and Broadcom, to take-on Nvidia’s proprietary NVLink. The first generation of UALink supports 1.28 TB/s of bandwidth for systems of 4 to 8 accelerators while future generations will support racks of 72 accelerators and more. 

Conclusion: 

Judging by the poor stock performance over the past 1-2 years, the market thinks AMD is down for the count. I think it’s the nuances of AI training versus inference (and timing of those markets) that has made AMD appear to be inconsequential to AI hardware. Although I do not foresee AMD surpassing Nvidia in terms of market cap by a long shot, I believe it’s highly probable that AMD’s returns outpace the AI leader due to the sheer amount of revenue growth hidden within AI inference. Specifically, inference is expected to be a larger market than training, and AMD’s strengths will finally be on full display.  

My current prediction is that AMD does not need to even come close to overtaking Nvidia on revenue or market cap for the stock performance to exceed Nvidia's over the next few years. Rather, the unforeseen second wind from GPUs and AI systems will be enough to make second place the most rewarding era in AMD’s history.

Please note: The I/O Fund conducts research and draws conclusions for the company’s portfolio. We then share that information with our readers and offer real-time trade notifications. This is not a guarantee of a stock’s performance and it is not financial advice. Please consult your personal financial advisor before buying any stock in the companies mentioned in this analysis. Beth Kindig and the I/O Fund own shares in NVDA and AMD at the time of writing and may own stocks pictured in the charts.

Recommended Reading:

Leave a Reply

Your email address will not be published. Required fields are marked *