The question of “why did Crowdstrike sell-off” doesn’t seem to be satisfied by the $10 million miss on forward revenue and ARR.
Forward Q4 revenue was expected to be $634M and the company guided $619M to $628M for a miss of about $10 million, if we take a midpoint of $624 million (about 1.5% miss). ARR was $2.34 billion compared to analyst expectations of $2.35 billion, for a $10 million miss (less than 1% miss).
Although this likely contributed, I believe the analyst we quoted in our Pre-ER write-up that was modeling for net new ARR of $224M to $230M-plus may be providing a missing link between analyst expectations for this key metric and actual results of $198 million. At the midpoint, this would be more of a miss of 14.6%.
“An analyst note from Barclays’ Saket Kalia is modeling ARR net addition of $224 million “but thinks upside could be $230M-plus given strong pipeline commentary.” At $230M, it would represent 5% sequential growth and 35% YoY growth. This would be down from 15% sequential growth in the previous quarter and 45% YoY.”
The reason we flagged this is because the net new ARR at high point of $230M would still mark a strong deceleration to 5% sequential growth down from 15% sequential growth last quarter. This means this would have to be met or we would be nearing flat to negative sequential growth on net ARR.
With the actual of $198 million reported, this drops the net new ARR at negative sequential growth of negative (9%) down from $218 million last quarter. This marks a change compared to the comp of 13% in sequential growth from Q2 2022 to Q3 2022.
The market is nervous with cloud becoming the other shoe to drop as enterprise budgets will slow long after consumer slows due to annual billing cycles, annual budget reviews (i.e., likely to produce budget cuts) and due to higher switching costs (or in cloud’s case, slower to switch off than consumer or ad spending, for example).
In my opinion, this is why outsized pressure is being placed on sequential growth. The market does not care about YoY because it’s assuming enterprise spending wasn’t affected yet.
CrowdStrike Q3 Overview:
CrowdStrike beat both top line and bottom line for Q3. In fact, an area where CrowdStrike continues to stand out from its peers is the health of the bottom line and both Q3 actual and Q4 guide was no exception in this regard. For example, the free cash flow margin of 30% is exceptional.
The company reported revenue of $581 million for growth of 53% compared to revenue of $574 million expected for growth of 51%. This is a slight deceleration from 58% last quarter.
For Q4, the company guided for revenue of $619 million to $628 million compared to expectations of $634 million. At the midpoint of $623.5million, this is a $10.5 million miss.
The GAAP EPS of ($0.24) compares to ($0.22) EPS from the year ago quarter and ($0.25) EPS last quarter.
Adjusted EPS for Q3 came in at $0.40 compared to $0.32 expected. This compares to $0.36 last quarter and $0.17 in the year ago quarter.
Adjusted EPS guide for Q4 also beat at $0.42 to $0.45 compared to $0.34 EPS expected.
GAAP gross margin was 72.7% which was in line with a range of 73% to 74% over the past five quarters. The adjusted gross margin this quarter was at 75% compared to 76%-77% over the past five quarters. Subscription gross margins were also in line.
GAAP operating margin of (9.70%) compares to (9%) last quarter and (10.5%) in the year ago quarter. This resulted in GAAP operating loss of ($56.4) million which is a tad higher than the $48 million losses last quarter and the $40 million losses in the year ago quarter.
The adjusted operating margin was a beat in Q3 and Q4. This was a bright spot in the report with adjusted OM of 15.4% compared to 13% estimated. This compares to 16% Adj OM last quarter and Adj OM of 13% last year. This was essentially flat and it’s important it did not contract.
The guide on adjusted operating income of $87.2M to $93.7M implies an adjusted operating margin of 14.5%.
The GAAP net margin of (9.4%) and adjusted net margin of 16.5% was in line with previous quarters. The guide for adjusted net margin is also in line at 16.6%.
CrowdStrike is very strong on cash flow margins and is one of the top ranking cloud stocks in this regard. This quarter the company reported a free cash flow margin of 30% for FCF of $174 million. The company is guiding for a FCF margin of 28% to 30% next quarter. The operating cash flow was $242.9 million for a margin of 41.8%. There is $2.47 billion in cash on the balance sheet.
The company paid $140 million in stock-based compensation for a margin of 23.7%.
Key Metrics:
As stated in the Intro, the key metrics are likely causing the sell-off.
RPO was up 44% year-over-year for $2.797 billion and was up 11.6% sequentially. However, management reminded analysts that ARR is the leading key metric for their business.
Ending ARR grew 54% year-over-year to $2.34 billion and grew 9.3% sequentially. Therefore, because ending ARR was strong, the net new ARR could be easily underestimated in terms of impact. The net new ARR at $198 million in fiscal Q3 compared to $218 million net new ARR in fiscal Q2 indicates a 9% sequential decline.
The market has the jitters right now so the sequential decline is important to pay attention to especially because management said to expect further weakness in the upcoming Q4 quarter. Here is what the CFO said:
“Even though we entered Q3 with a record pipeline, we are expecting the elongated sales cycles due to macro concerns to continue, and we are not expecting to see the typical Q4 budget flush given the increased scrutiny on budgets. While we do not provide net new ARR guidance given the current macro uncertainty, we believe it is prudent to assume that Q4 net new ARR will be below Q3 by up to 10%.”
If I understand the CFO correctly, then this implies a net new ARR of $178.3 million for Q4 (10% lower than the current quarter at $198.1M) compared to net new ARR of $216 million in the year ago quarter. This is important because it’ll mark not only a sequential decline but a year-over-year decline in net new ARR. The market had already sold off for what I presume was a sequential decline in Crowdstrike’s leading key metric, and management then stated the decline would be steeper for Q4 on the call. Once the comment above was made, we were certainly not going to see a reversal in the stock price from the earnings call.
Customer count was strong at 44% growth. The mix of domestic versus international was slightly lower than usual for North America at 69% with EMEA being slightly higher at 15%.
Deferred revenue grew 56.4% year-over-year and backlog grew 19%.
Additional Commentary:
CrowdStrike was transparent about the importance of ARR even in the face of net new ARR being lower than expected. Here is what was said by the CFO:
“And then finally, just to comment on ARR. You pointed out that's how we run our business. ARR, though, is really an X-ray into the contracts themselves. And as we view that as the most important — or most transparent metric into the outlook for our business, that's the one where we're focused on. So, hopefully, that gives some more clarity on how we think about cRPO and ARR.”
Later on, an analyst did zero-in on the (9%) decline.
“Andrew Nowinski
Great. Thank you for taking the question this afternoon. So total ARR of $2.3 billion, growing 54% is still absolutely amazing, I was – and it's at scale. But I was wondering, were you surprised that the net new logos that you added were down 9% this quarter?
Burt Podbere
Thanks, Andy. So when we think of the net new logos, it really corresponds to what we talked about in terms of what we saw in that SMB space. The SMB space is the one that drives the velocity of our net new logos. And as we talked about, we saw an 11% increase in our sales cycle in the SMB space. And that actually equated into $15 million in terms of deals in that space that could push out. And so when you think about 15 million in that space and what it means in terms of logos, where you can do the math, it's a pretty big number.
So that's how we think about net new logos corresponding to what we saw in net new ARR from the SMB space. So from that perspective, we weren't surprised at the end of the day when we saw that what happened with respect to the increased sales cycles and the amount of money that got pushed out in the SMB space.
My note: Just to be clear, when they say “push out” they are referring to a delayed sales cycle for an impact of $15 million.
The CFO did reiterate the 10% further sequential decline in net new ARR between Q3 and Q4 when he said:
“When we do talk about net new ARR, I did talk about in the prepared remarks about how we think about up to 10% headwinds going into Q4 from Q3, and that's just to coincide with some of the headwind activity that we saw accelerated at the end of this quarter. So that's how we think about that.”
Conclusion:
Given the tough macro, our goal is to fully understand why the market may favor some stocks and deeply discount others after an earnings report. The market is getting nervous on cloud. We talked about this with Microsoft and also saw this following Datadog’s report.
“Microsoft is guiding down for next quarter with analyst expectations for the December quarter at $56.04 billion compared to management guidance on the call for revenue of $52.75 billion, at the midpoint. This represents 2% growth. […] That’s a 11% deceleration over the next few months. Some of this may be coming from Azure as the company is expected Azure to decline 5% next quarter for its current growth rate. This will be 37% growth on a constant currency basis, down from 42% this quarter.”
“RPO decelerated and is a concern. The deceleration we noted in our last earnings report and our pre-earnings write-up where we noted the deceleration went from 85% to 51%. This quarter, the deceleration steepened to 31% year-over-year growth for $941 million. RPO is still up on a sequential basis with $858M in RPO in Q1, $881M in RPO in Q2 and $941M in RPO this quarter. If it were to decline on a QoQ basis, the stock would be deeply penalized, so we will monitor this as we go along.”
What we saw today from CrowdStrike sounded very familiar, in my opinion. The market is nervous about cloud and is swiftly discounting these stocks on slowing revenue plus any additional signs revenue may slow in the future. We will need to see more information to draw any conclusions, most especially we will need SentinelOne’s report coming next week.
This article was originally published on Forbes on Nov 23, 2022,12:52pm ESTForbes on Nov 23, 2022,12:52pm EST
Nvidia has overcome strong headwinds over the past few years, including United States-China tensions, supply chain disruptions spanning many components, tough comps on the data center, tough comps on gaming, and a less-than-rosy macro environment. However, the most impactful of all has been Ethereum’s merge to Proof of Stake (POS), which led to a $2.5 billion cumulative miss in revenue.
In September, we made a prediction in the analysis entitled “Nvidia Stock Is Ready to Rumble with RTX 40 Series and H100 GPUs” that Nvidia’s new gaming release would soften the blow when we said the following:
“First, Nvidia is restricting supply on its current gaming model. Per the CFO: ‘Across those two quarters, the Q2 of ‘23, the Q3 of ‘23, we have likely undershipped gaming to our end demand significantly.’
[…] We estimated for our premium members that the amount undershipped is a minimum of $1 billion. The reason behind this is to help keep prices stable and to increase demand for the RTX 40 Series.
Second, Nvidia announced its GeForce RTX 40 Series at the GTC 2022 Conference this week.
The new Ada Lovelace architecture uses 76 billion transistors and a 4nm production process. In the keynote, the CEO stated: ‘Nvidia engineers worked closely with TSMC to create the 4N process optimized for GPUs. This process let us integrate 76 billion transistors and over 18,000 CUDA cores, 70% more than the Ampere generation.’
The improvement from 8nm to 4nm means more transistors on the GPU, which results in better performance as the 4nm processes data faster.
In the gaming world, this much anticipated release is expected to be 2-4X faster than the RTX 3090 Ti. The flagship AD102 GPU model will have 144 individual streaming multiprocessors (SMs) in one die compared to 84 SMs in the Ampere architecture. As stated, the AD102 will also have a 70% increase in CUDA cores over the RTX 3090 Ti […]
The popularity of this release will help determine if Nvidia can stage a comeback in the gaming segment.” You can read the full analysis here. Fast-forward and not only was the GeForce RTX 40 Series with Ada Lovelace architecture popular, management stated “the Ada launch was a homerun.” Below, we look at the most recent earnings report and then we break out additional details that support Nvidia’s Q3 reaching a gaming bottom.
Sign up for I/O Fund's free newsletter with gains of up to 221% – Click HereSign up for I/O Fund's free newsletter with gains of up to 221% – Click HereClick Here
What Q3 Earnings Results Says About a Gaming Bottom
First, I’ll provide a general overview of Nvidia’s earnings results before I discuss what to expect in the gaming segment specifically.
Nvidia reported as expected for Q3 ending in October with revenue of $5.93 billion for growth of (17%) which matched management guidance of $5.90 billion. Analyst consensus for revenue was $5.85 billion, or (17.7%) growth.
Fiscal Q4 ending in January was a slight miss with guidance of $6 billion compared to analyst consensus of $6.17 billion. This represents growth of (21%).
Nvidia reported adjusted EPS of $0.58 which missed adjusted EPS estimates of $0.71. This compares to the July quarter of $0.51 adjusted EPS.
Management indicated that profitability will increase from here: [GAAP and non-GAAP operating expenses were] primarily due to higher compensation expenses related to headcount growth and salary increases and higher data center infrastructure expenses. Sequentially, both GAAP and non-GAAP operating expense growth was in the single-digit percent, and we plan to keep it relatively flat at these levels over the coming quarters.”
In Q3, the GAAP gross margin was 53.6% and the adjusted gross margin 56.1%. This was a miss from management Q3 guidance of 62.4%. The reason for the miss related to China: “Gross margins reflect $702 million in inventory charges largely related to lower data center demand in China, partially offset by a warranty benefit of approximately $70 million.”
Nvidia is signaling that gross margin will return to normal next quarter with a guide for GM of 63.2%.
For the most part, Nvidia’s bottom line showed signs that last quarter was a bottom for the company with marginal, yet crucial improvement sequentially. As long as the company does not increase operating expenses, which the CFO stated the opex would be flat and not increase, then these margins should improve from here.
The company reported GAAP operating profit of $601 million for an operating margin of 10.1%. This compares to an operating margin of 7.44% last quarter. Nvidia’s typical OM is in the 37%-38% range.
The adjusted operating profits of $1.56 billion with a margin of 25.9% in Q3 compares to an adjusted operating margin of 19.76% in Q2. This is down from Nvidia’s typical adjusted OM of 47%.
The adjusted net margin of 24.5% in Q3 compares to an adjusted net margin of 19.27% last quarter.
The free cash flow margin was (2.6%) for free cash flow of ($156) million compared to a 12% margin last quarter for free cash flow of $824 million. The company has $13.14 billion in cash and $10.95 billion in debt.
The company returned $3.75 billion to shareholders with share repurchases and cash dividends. with $8.3 billion remaining under the share repurchase authorization through December 2023.
The biggest names in tech are reporting their earnings right now, and our premium members are getting updates almost daily. Learn more about about our premium membership here.The biggest names in tech are reporting their earnings right now, and our premium members are getting updates almost daily. Learn more about about our premium membership here.Learn more about about our premium membership here.
Evidence that Gaming Bottomed in Q3
Gaming revenue was down (51%) for revenue of $1.57 billion. Admittedly, even if Q3 is the bottom, there is still quite a ways to go before the company returns to growth in this segment. The reason it’s important to identify a fundamental bottom is because it typically correlates with a bottom in the stock price.
Nvidia Management Points Toward Q3 as the Bottom in the Earnings Callas the Bottom in the Earnings Call
In addition to saying “the Ada launch was a homerun,” management expects gaming to grow sequentially from Q3 to Q4. Management also stated “our new Ada Lovelace GPU architecture had an exceptional launch” and “we sold out quickly in many locations and are working hard to keep up with demand.”
Most importantly, management stated gaming will return to sequential growth in Q4 and that channel inventory will “approach normal levels” as the company exits Q4 to where the company can more adequately match supply with demand.
On the call, Kress discussed that the sell-through rate across two quarters for gaming is $5 billion total, which helps prove the popularity of Nvidia’s RTX 40 series. The CEO also stated: “That 4090 — we shipped a large volume of 4090s because as you know, we were prepared for it. And yet within minutes, they were sold out around the world. And so, the reception of 4090 and the reception of 4080 today has been off the charts.”
The first release date for the RTX4090 models was October 12th with a starting price of $1,599. There was a second release date in November for the RTX4080 models with prices of $1,199 and $899. Notably, the mid-range RTX 40 series outperforms the previous generation’s high-end models, which also helps to drive demand because customers receive an upgrade at the $899 and $1,199 level. This is due to the Ada Lovelace architecture which offers 1,400 Tensor TFLOPs versus 320 Tensor TFLOPs which means the DLSS is superior and the high-end RTX 30 Series cannot compete with the mid-range RTX 40 series.
Deep learning super sampling (DLSS) refers to using AI to predict the next pixel. The new DLSS 3.0 not only predicts pixels but will also use AI to predict frames. This results in “up to four times” better performance over traditional rendering.
In addition to this, Nvidia released a new feature powered by Shader Execution Reordering (SER) which will improve ray-tracing performance by 3X with 25% faster frame rates. Rather than deliver workloads sequentially, the GPUs are able to reorder the workloads to process more workloads at once which results in more power and better performance.
Conclusion:
Despite a historic revenue miss, Nvidia is rising to the occasion with the perfectly timed Ada Lovelace architecture. As we said in our previous analysis, Nvidia is flexing their product muscles by meeting head-on the wave of negative sentiment on the stock. Investors should keep in mind, that despite enormous headwinds, Nvidia has been the best performing mega cap stock over the past few years (reference our analysis here for more details).
The company’s swift and concise answer to the crypto mining selloff helps illustrate why Nvidia stands apart from its peers – primarily, that its products are superior, end-market demand remains strong, and management has many levers it can pull to quickly reverse a bottom.
The I/O Fund targeted NVDA on October 13th for a price of $108. After a 50% gain in less than a month, we trimmed some NVDA around $162 with real-time trade alerts. We did this to raise cash, so that we can buy more at lower levels. Please join us next week, Thursday, 12/1, at 1:30 PST, for our premium webinar. We will discuss NVDA in depth and lay out our targets for adding to this position.
If you would like notifications when my new articles are published, please hit the button below to "Follow" me. You can also sign up for my newsletter here to receive weekly tech stock analysis for free.
There were a few key things discussed on the call:
Why the H100 will ramp faster than the A100 with Q1 being the estimated time when we should see the H100 driving forward data center growth (we should get an acceleration this quarter in the DC segment).
How the H100 helps drive enterprise software revenue as it’s been optimized with Nvidia’s software stack.
The strength of Nvidia’s networking business following the acquisition of Mellanox.
The CEO believes even if hyperscalers slowdown spending in 2023, that Nvidia is more insulated because their systems are optimized for AI acceleration, which is a top priority within capex spending, and because their systems reduce costs and improve efficiency.
My specific investment thesis is this: “the A100 GPU is what led the company’s gains since Q2 2020 (detailed here) and the Hopper H100 GPU is what will lead the company’s gains for the next two years.”I detail what I think is the most important Q&A from the call below plus other moments that provide a glimpse into what Nvidia investors can expect from here.
Note: I did not cover gaming in this analysis because I am covering Nvidia’s gaming bottom for my free newsletter. I’ll make sure to post this separate analysis on the forum early next week. The H100 will be absent since we are entering an actionable phase with the CEOs discussions about Q1. Actionable analysis, for the most part, is reserved for premium.
Financials:
Note: calendar months are provided to avoid confusion due to Nvidia’s off calendar fiscal year. This upcoming report will be Q3 FY 2023
Nvidia reported as expected for Q3 ending in October with revenue of $5.93 billion for growth of (17%) which matched management guidance of $5.90 billion. Analyst consensus for revenue was $5.85 billion, or (17.7%) growth.
Fiscal Q4 ending in January was a slight miss with guidance of $6 billion compared to analyst consensus of $6.17 billion. This represents growth of (21%). The guidance has led to slightly lower estimates for Q1 ending in April with revisions from (19.8%) to (22.5%) for revenue of $6.41 billion.
As stated on the forum in the pre-earnings write-up, analysts have Nvidia returning to positive growth by July 2023 and to strong growth of 35%+ by October of 2023. This is helped by the low comps that we are currently experiencing.
Nvidia reported adjusted EPS of $0.58 which missed adjusted EPS estimates of $0.71. This compares to the July quarter of $0.51 adjusted EPS. Management indicated that profitability will increase from here: [GAAP and non-GAAP operating expenses were] primarily due to higher compensation expenses related to headcount growth and salary increases and higher data center infrastructure expenses. Sequentially, both GAAP and non-GAAP operating expense growth was in the single-digit percent, and we plan to keep it relatively flat at these levels over the coming quarters.”
An analyst did bring up that stock based compensation has been increasing each quarter at $700 million in the current quarter, up from $648 million and $578 million in the two previous quarters.
In Q3, the GAAP gross margin was 53.6% and the adjusted gross margin 56.1%. This was a miss from management Q3 guidance of 62.4%. The reason for the miss related to China: “Gross margins reflect $702 million in inventory charges largely related to lower data center demand in China, partially offset by a warranty benefit of approximately $70 million.” Nvidia is signaling the gross margin will return to normal next quarter with a guide for GM of 63.2%.
The company reported operating profit of $601 million for an operating margin of 10.1%. This compares to management’s guidance for an operating margin of 18.5% with Nvidia’s typical OM at 37% to 38%.
The adjusted operating margin of 25.9% is down from the typical range of 47%.
GAAP net margin of 11.5% for net profit of $680 million was up from a GAAP net margin of 9.8% in the previous quarter. The adjusted net margin of 24.5% for an adjusted profit of $1.46 billion compared to 19.3% in the previous quarter.
For the most part, Nvidia’s bottom line showed signs that last quarter was a bottom for the company with marginal, yet crucial improvement sequentially.
Nvidia had lower cash flow margins than it did last quarter at a 6.61% operating margin for operating cash flow of $392 million compared to a margin of 18.9% last quarter for operating cash flow of $1.27 billion. The free cash flow margin was (2.6%) for free cash flow of ($156) million compared to a 12% margin last quarter for free cash flow of $824 million.
The company had stock-based compensation of $745 million in the quarter, up from $648 million last quarter. There is $13.14 billion in cash and $10.95 billion in debt. The company returned $3.75 billion to shareholders with share repurchases and cash dividends. There is $8.3 billion remaining under the share repurchase authorization through December 2023.
Nvidia Discusses Why the H100 Will Ramp Faster than the A100
Since our thesis is that the H100 will drive sales and the stock price over the next couple of years, similar to the A100, we want to make sure we are getting confirmation of how the H100 is performing now that it has been on the market for about a month.
Why the H100 is Special
1. Enterprise Software
The first question from C.J. Muse discussed how the H100 is bundled with Enterprise Software, and the timing of when software monetization will begin to occur. The answer from the CEO was effectively “now.”
Here is what Huang said:
“Every company we’re talking to would like to have the agility and the scale, flexibility of clouds. And so, over the last year or so, we’ve been working on moving all of our software stacks to the cloud – all of our platform and software stacks to the cloud. And so today, we announced that Microsoft and ourselves are going to standardize on the NVIDIA stack, for a very large part of the work that we’re doing together so that we could take a full stack out to the world’s enterprise. That’s all software included.
If they would like to use it in the cloud, it’s per GPU instance hour; if they would like to utilize our software on-prem, they could do it through software license and so — license and subscription. And so, in both cases, we now have software available practically everywhere you would like to engage it.
2. The CEO stated H100 is going to Ramp faster than the A100
This was the discussion I felt was most important to Nvidia investors on the call. Second place would be the discussion around the Gaming bottom. Enterprise software is certainly important to as the software stack will eclipse hardware at some point. However, today, Nvidia is a hardware company and visibility into the pace of H100 adoption is key for our 2023 position and allocation.
Notably, I believe there will be positive surprises in the data center segment as we go along into 2023. It’s prudent for analysts to be cautious as we don’t have big tech capex numbers yet and the H100 has only been out for a month. Eventually, enthusiasm for Nvidia will return and it’ll the H100 that drives the positive sentiment.
Here was the question, which is being quoted in full due to its importance to our thesis:
William Stein:
I’m hoping you can discuss the pace of H100 growth as we progress over the next year. We’ve gotten a lot of questions as to whether the ramp in this product should look like a sort of traditional product cycle where there’s quite a bit of pent-up demand for this significant improved performance product and that there’s supply available as well. So, does this rollout sort of look relatively typical from that perspective, or should we expect a more perhaps delayed start of the growth trajectory where we see maybe substantially more growth in, let’s say, second half of ‘23?”
Jensen Huang
H100 ramp is different than the A100 ramp in several ways. The first is that the TCO, the cost benefits, the operational cost benefits because of the energy savings because every data center is now power limited, and because of this incredible transformer engine that’s designed for the latest AI models.
The performance over Ampere is so significant that I — and because of the pent-up demand for Hopper because of these new models that are — that I spoke about earlier, deep recommender systems and large language models and generative AI models. Customers are clamoring to ramp Hopper as quickly as possible, and we are trying to do the same. We are all hands on deck to help the cloud service providers stand up the supercomputers.
Remember, NVIDIA is the only company in the world that produces and ships semi-custom supercomputers in high volume. It’s a miracle to ship one supercomputer every three years. It’s unheard of to ship supercomputers to every cloud service provider in a quarter. And so, we’re working hand in glove with every one of them, and every one of them are racing to stand up Hoppers. We expect them to have Hopper cloud services stood up in Q1. And so, we are expecting to ship some volume — we’re expecting to ship production in Q4, and then we’re expecting to ship large volumes in Q1. That’s a faster transition than Ampere. And so, it’s because of the dynamics that I described.
My translation: Per the CEO, Q1 should be good to us Nvidia investors!
3. Grace Hopper and the CPU, GPU, DPU Trifecta
Grace Hopper is Nvidia’s new CPU that is meant to further accelerate and be integrated with Nvidia’s GPUs and DPUs. Notably, AMD is doing the same – where their CPUs are optimized and integrated to further accelerate AMD’s GPUs and DPUs.
Mark Lipacis
Jensen, I think for you, you’ve articulated a vision for the data center where a solution with an integrated solution set of a CPU, GPU and DPU is deployed for all workloads or most workloads, I think. Could you just give us a sense of — or talk about where is this vision in the penetration cycle? And maybe talk about Grace — Grace’s importance for realizing that vision, what will Grace deliver versus an off-the-shelf x86 [CPU], do you have a sense of where Grace will get embraced first or the fastest within that vision? Thank you.
Jensen Huang
Thanks Mark. Grace’s data moving capability is off the charts. Grace also is memory coherent to our GPU, which allows our GPU to expand its effective GPU memory, fast GPU memory by a factor of 10. That’s not possible without special capabilities that are designed between Hopper and Grace and the architecture of Grace […] It all needs to be fast, so that you can make a recommendation within milliseconds to hundreds of millions of people using your service.”
Networking is Showing Surprising Strength
According to an analyst on the call, their calculations show networking driving most of the sequential growth. He is referencing Mellanox acquisition which we covered a few years ago in this analysis.
Ambrish Srivastava
I actually had a couple of clarifications. Colette, on the data center side, is it a fair assumption that compute was down Q-over-Q in the reported quarter because the quarter before, Mellanox or the networking business was up as it was called out. And again, you said it grew quarter-over-quarter. So, is that a fair assumption?
Collette Kress
So, looking at our compute for the quarter is about flattish. Yes, we’re seeing also growth, growth in terms of our networking, but you should look at our Q3, compute is about flattish with last quarter.
Additional comments on Networking:
“Your data center networking business, I believe, is driving about $800 million per quarter in sales, very, very strong growth over the past few years” – Harlan Sur
“Jensen, can you help us understand like where your InfiniBand networking sits relative to like traditional data center switching?” – Aaron Rakers
“Yes. Thanks, Aaron. The math is like this. If you’re going to spend $20 billion on an infrastructure and the efficiency of that overall data center is improved by 10%, the numbers are huge. And when we do these large language models and recommender systems, the processing is done across the entire data center. And so, we distribute the workload across multiple GPUs, multiple nodes and it runs for a very long time. And so, the importance of the network can’t be overemphasized.”
We’ve been using Big Tech capex as a proxy for our semiconductor positions. According to one analyst on the call, the presumption is capex from the Big 3 will be flat in 2023. These are still sizable budgets, but the concern is if capex flatlines in 2023, what level of growth will the data center segment be capable of?
Here was the question on the call from Vivek Arya:
“And then, Jensen, the question for you. A lot of concerns about large hyperscalers cutting their spending and pointing to a slowdown. So if, let’s say, U.S. cloud CapEx is flat or slightly down next year, do you think your business can still grow in the data center and why?”
The answer from the CEO focused on Nvidia driving growth from AI acceleration, rather than general purpose compute, which implies that Capex can be flat while Nvidia will be serving the most valuable piece in the stack. AI acceleration, according to the CEO, will not be flat or down.
“Vivek, our data center business is indexed to two fundamental dynamics. The first has to do with general purpose computing no longer scaling. And so, acceleration is necessary to achieve the necessary level of cost efficiency scale and energy efficiency scale, so that we can continue to increase workloads while saving money and saving power. Accelerated computing is recognized generally as the path forward as general purpose computing slows. The second dynamic is AI. And we’re seeing surging demand in some very important sectors of AIs and important breakthroughs in AI.”
The CEO discussed deep recommender systems, large language models driven by Transformers, and generative AI for generating images and videos. He ended the answer with this: “And so, you could see that our company is indexed to two things, both of which are more important than ever, which is power efficiency, cost efficiency and then, of course, productivity. And these things are more important than ever. And my expectation is that we’re seeing all the strong demand and surging demand for AI and for these reasons.”
My translation: Capex can be flat and the CEO foresees Nvidia will take a higher percentage of this capex because they’re serving demand where few companies can across the three major AI breakthroughs he pointed out. My other comment would be that we won’t have a full picture of capex for next year until we get Q1 reports and 2023 full year guides around end of January. This is when we did a deep dive analysis on capex spending last year, and we will revisit this. So, keep an eye out for that.
Note: Nvidia Expected to Change Reporting on Data Center
There was a discussion on the call that Nvidia plans to start breaking out the data center segment to account for internet service companies in addition to hyperscalers. My understanding is internet service providers would mean 5G providers or other internet services related to edge computing. This was not directly stated but it makes the most sense given where edge computing is headed, which could rival the hyperscalers.
Matt Ramsay
I guess, Colette, I heard in your script that you had you talked about maybe a new way of commenting on or reporting hyperscaler revenue in your data center business. And I wondered if you could maybe give us a little bit more detail about what you’re thinking there and what sort of drove the decision?
Jensen Huang:
[…] And these are internet service companies that offer services, but they’re not public cloud computing companies. The second factor has to do with cloud computing […] [hyperscalers] are two things to us, therefore, a hyperscaler can be a sell to customer; they are also a sell with partner.”
Conclusion:
The market has been discouraging this year. The gaming selloff for Nvidia and PC selloff for AMD were brutal. But if you listen to these calls, it is crystal clear something monumental is going on. We want to capture this as fully as possible. Perhaps we will have 40% allocation in two positions (NVDA or AMD) or perhaps we will have to trim to 15% across two positions and layer back up to 30% allocation. We will do this as skillfully as possible.
If 2021 to 2022 taught us anything, it’s that only the strong survive. That goes for stocks/companies and investors. There is no doubt that NVDA and AMD will weather what’s ahead and we want to stay close to our AI bellwethers. Whatever the tide brings us, you can expect us to obsessively cover these companies and to actually increase our coverageincrease our coverage as we go along. There is no limit to the research needed if we are building positions with conviction.
This article was originally published on Forbes Oct 14, 2022,09:31am EDTForbes Oct 14, 2022,09:31am EDT
Semiconductors have been rocked this year due to slower consumer spending on PCs, mobile and also slowing enterprise budgets that further affect hardware purchases, including PCs, notebooks and servers. The silver lining is Capex spending by Big Tech companies, which we’ve covered in the past for our premium members, when we stated that the increased Capex from companies like Google, Microsoft and Amazon and other big tech companies greatly benefit the semiconductor market.
The news has been in an uproar about crypto mining and the consumer-related PC markets. However, it has been our stance for some time that Big Tech capex is the true leading indicator for AI semiconductor companies. Despite an enormous increase in Big Tech capex primarily driven by data centers, this line item does not get the attention it deserves in terms of follow-through to the semiconductor industry. Below, we look at FY2022 budgets to draw the conclusion that H2 spending on data center chips is equal if not greater than the first half of 2022.
Market Opportunity
According to Gartner, the overall IT spending is expected to grow 3% to $4.5 trillion in 2022. It is lower than the 10% growth in 2021. The slowdown was mainly due to the cutdown in spending on personal computers, tablets, and printers.
The Data Center Systems segment, however, is expected to grow fastest among all the segments. It is expected to grow 11% YoY to $212 billion, higher than the 6.4% growth in 2021.
Hyperscale data centers, which are very large data centers primarily operated by Amazon, Microsoft and Google, are expected to outpace overall data center systems.
According to data from Allied Market Research, the global hyperscale data center market is expected to grow from $59 billion in 2020 to $585 billion by the year 2030, representing a Compound Annual Growth Rate of 26% from 2021 to 2030.
Similarly, the Artificial Intelligence chip market is expected to grow from $8.02 billion in 2020 to $194.90 billion by the year 2030, representing a CAGR of 37% from 2021 to 2030.
According to a report published by Dell’Oro Group, the global data center Capex is expected to be $377 billion by the year 2026 – which implies the majority of the growth noted by Allied Market Research will occur in the next few years.
The private markets are also signaling growth will continue as there has been quite a bit of deal activity in data centers.
According to data from Synergy Research Group, 87 data center focused merger and acquisition deals were closed in the first half of 2022, worth $24 billion. There is an additional $18 billion of pending deals in the pipeline that are agreed and are yet to be officially closed. The research group mentioned that 209 deals were closed in 2021 for over $48 billion, up 41% from 2020.
One of the more significant deals this year that was completed is the acquisition of CyrusOne for $15 billion by KKR and Global Investment Partners. John Dinsdale, Chief Analyst at Synergy Research Group, said, “There is an ever-increasing demand for data center capacity, driven by rapidly growing cloud markets, aggressive expansion of hyperscale operator networks and continued growth of data-rich digital services.”
Earnings season kicked off this week and our premium members are receiving deep-dive tech earnings analysis straight to their inbox each week. We also offer real-time trade notifications, weekly webinars, a completely transparent portfolio of 20+ positions and more. Learn more about our premium membership.premium members are receiving deep-dive tech earnings analysis straight to their inbox each week. We also offer real-time trade notifications, weekly webinars, a completely transparent portfolio of 20+ positions and more. Learn more about our premium membership.
Big Tech Capex H2 2022
Alphabet’s Q2 Capex grew by 24% YoY to $6.9 billion. Ruth Porat, CFO of Alphabet, said, “Turning to CapEx. The largest investments in the second quarter were in servers followed by data centers and office facilities.” were in servers followed by data centers and office facilities.” The company had invested $24.6 billion in Capex in the year 2021, up 11% YoY. The management expects Capex to rise in 2022. In the Q2 2022 earnings call, Ruth Porat said, “We continue to expect an increase in CapEx in 2022 versus last year. For the balance of 2022, the increase will be particularly reflected in investments in technical infrastructure globally with servers as the largest component.” Earlier this year, the company announced its plan to invest about $9.5 billion in data centers and offices in the U.S. for the year 2022. This is up from about $7 billion spent in 2021.
Similarly, Microsoft’s Capex including financial leases, grew by 19% YoY to $8.7 billion in the Q4 FY2022 quarter (i.e., Q2 CY2022). Amy Hood, CFO of Microsoft, said, “Maybe let me start by talking about Q4's capital spend. Obviously, the big driver of our growth this quarter was in data center spend, both new and newbuilds as well as adding capacity to existing data centers. We are seeing, obviously, good demand signal.” data center spend, both new and newbuilds as well as adding capacity to existing data centers. We are seeing, obviously, good demand signal.” The management expects a sequential decrease in the next quarter due to the normal variability in the quarterly spend. In the CY 2021, Microsoft’s Capex including financial leases, grew by 33% YoY to $27.5 billion.
Amazon incurred capital expenditures, including equipment financial leases, of about $60 billion in 2021. About 40% of this is made up of technology infrastructure supporting AWS and worldwide stores business. The management expects Capex to increase over the last year with the increase in technology infrastructure.
Brian Olsavsky, senior VP and CFO, said in the Q2 2022 earnings call, “For full-year 2022, we do expect to spend slightly more on capital investments than last year, but the proportion of capital spending shifts among our businesses. We expect technology infrastructure spend to grow year-over-year, primarily to support the rapid growth in innovation we are seeing with AWS. We expect infrastructure to represent a bit more than half of our total capital investments in 2022.”
Meta’s capital expenditures in Q2, including principal payments on finance leases were $7.75 billion, up 64% YoY. The company’s CFO, Dave Wehner, said in the Q2 earnings call, “Capital expenditures, including principal payments on finance leases, were $7.7 billion, driven by investments in servers, data centers and network infrastructure. The big step-up in CapEx, both year-over-year and sequentially related to server spend, including for our AI infrastructure.”driven by investments in servers, data centers and network infrastructure. The big step-up in CapEx, both year-over-year and sequentially related to server spend, including for our AI infrastructure.”
The company expects 2022 capital expenditures, including principal payments on financial leases, to be $32 billion at the mid-point of the guidance, representing a 66% YoY growth. Tracking the Capex in the first two quarters, Meta Platforms had spent $13.3 billion which suggests the spend will be higher in 2H 2022. When we deduct from the mid-point of the guidance, it comes to $18.7 billion for H2.
Meta also recently announced its plan to expand the Eagle Mountain data center project. It is Phase 3 expansion plan and brings the total investment in the project to over $1.5 billion.
Earnings season kicked off this week and our premium members are receiving deep-dive tech earnings analysis straight to their inbox each week. We also offer real-time trade notifications, weekly webinars, a completely transparent portfolio of 20+ positions and more. Learn more about our premium membership.premium members are receiving deep-dive tech earnings analysis straight to their inbox each week. We also offer real-time trade notifications, weekly webinars, a completely transparent portfolio of 20+ positions and more. Learn more about our premium membership.
Conclusion
Thanks to very big Big Tech capex budgets, Nvidia’s data center revenue grew 71% YoY to $7.6 billion in 1H 2022. Similarly, AMD’s data center revenue grew by 83% YoY to $1.5 billion in Q2 2022 and doubled in Q1 2022.
Due to consumer-related weakness, the data center is now the leading segment for these companies, which we had predicted would occur in 2018 in my free weekly newsletter. We also provide regular deep dives for our premium research Members on a more granular level as to what will happen next in the semiconductor industry.
Royston Roche, Financial Analyst for the I/O Fund contributed to this article.
Please note: The I/O Fund conducts research and draws conclusions for the Fund’s positions. We then share that information with our readers. This is not a guarantee of a stock’s performance. Please consult your personal financial advisor before buying any stock in the companies mentioned in this analysis.
Marvell’s management team did an excellent job of acquiring Inphi and executing. Typically, we avoid M&A for a year to allow the financials to merge, yet in this case, leaning into the acquisition was a good choice.
The Marvell management team’s execution skills are needed once again because Marvell has an opportunity to greatly increase its revenue and profits if management can execute in a new market one more time. The opportunity is a new architecture called CXL that disaggregates memory from the CPU. CXL is attracting a lot of attention at industry events, such as Hot Chips 2022, because it’s focused on optimizing one of the most expensive parts of the data center – which is memory.
Before we go into the 2023-2024 Marvell product road map, and why it’s key to the company’s future, I want to discuss the fiscal Q2 2023 earnings.
Fiscal Q2 2023 Earnings Overview
The market is concerned over Marvell’s data center guidance of 20% growth next quarter. This is a slowdown from the most recent quarter at 48% YoY growth and earlier quarters at >100% growth.
At an estimated $600 million, it will also mean a sequential decline both from Q2 and Q1, which were at $643M and $640M, respectively. Marvell stated it’s the on-premise business weighing on their cloud data center business and supply issues (more below).
Notably, Q2 of last year was an important moment for the company when 56% sequential data center growth grew from $277 million to $434 million in the span of three months following the close of the Inphi acquisition in April 2021. From there, the company has sustained Inphi’s already high growth levels for over a year.
The company is now at an annualized run rate of $6 billion, which the CEO reminded analysts, was the target for October of 2023. The company met the target originally provided at the October 2021 Investor Day one year earlier than expected. Notably, this was six months after Inphi was closed so M&A not a factor here.
Marvell’s Segment Overview:
The data center represents 42% of revenue at $643 million and grew 48% year-over-year.
The carrier infrastructure segment, which is wired and wireless and reflects 5G growth, reported 45% YoY to $285 million.
Enterprise networking grew handily at 53% YoY to $340 million and is expected to grow at 70% next quarter. We break this segment down below.
Consumer was down (1%) to $164 million and is expected to be down (10%) next quarter. Marvell has exposure to the storage market and this can weigh on the more robust segments.
Automotive was up 46% YoY to $84 million and is expected to be up 40% YoY next quarter. We also break down this segment below.
Marvell Financial Overview
Marvell was reporting negative top line revenue when we first covered it in 2019 and Marvell took another hit on revenue during Covid before accelerating to the 50%-74% revenue growth range.
The current quarter’s top line revenue in Q2 was at 41% which is a deceleration from Q1 with 74% revenue growth. The company guided for 29% year-over-year growth, which was a slight miss as analysts were expecting 30.3% growth in the fiscal Q3 quarter. The company reported EPS in line with adjusted EPS of $0.57. The guidance on EPS was a slight miss, however, at $0.59 reported versus $0.61 adjusted EPS estimated.
Semiconductors make a tougher investment as analysts can’t model too far into the future beyond what management teams provide. That is why there were many questions looking for help with how to factor in the “acceleration” in the data center the Marvell team is expecting in Q4 and what this will mean for CY2023.
An analyst asked if they can assume 10% QoQ in the data center for $1.7 billion overall revenue and the CEO said it sounded “a little on the high side.” This has led to analysts modeling $1.65 billion in revenue in Q4, for 22.5% growth. Therefore, despite a single-digit acceleration in the data center segment, there will still be a top line deceleration, if today’s forecast does not change.
The company’s margins and cash flow are a bright spot, and I believe this is being overlooked. If we get an acceleration in the data center into next year, then Marvell is fundamentally a much stronger company than it was during the previous data center streak.
On a GAAP basis, the gross margin was at 51% in the most recent quarter, up from 35% in the year ago quarter and up from 46% in FY2022. The company is guiding for the same GM of 51% next quarter.
The GAAP operating margin has improved quite a bit YoY to 8.3% in the current quarter compared to (25%) in the year ago quarter. This is also an improvement from Q1 with GAAP OM of 4.80%. The adjusted operating margin “hit a record” at 36.5% and is guided for 37% next quarter. Stock based compensation was at $139 million in the most recent quarter.
Cash flow is also improving with operating cash flow at $332 million, or 22% of revenue. This compares to $194 million last quarter and $819 million in FY2022. However, the company carries debt of $4.6 billion and has $617 million of cash on the balance sheet. This is a 1.8X net debt to EBITDA ratio.
Therefore, there has been substantial improvement yet Marvell does have a weaker debt profile than a company like AMD or Nvidia.
Source: YCharts
Note on Supply:
Marvell is aligned with AMD in that they believe supply chain issues will ease in Q4 and into 2023. Here is what Marvell said in the opening remarks:
“Therefore, for our overall data center end market, we project revenue in the third quarter to decline sequentially in the mid-single digits on a percentage basis. However, we expect our data center revenue in the fourth quarter to increase on a sequential basis, anticipating an improvement in supply and new product ramps in cloud.”
Here is what AMD said:
“The visibility with our customers, especially our large cloud customers’ second half of this year into next year is very good. And we’re planning really for the next four to six quarters, and that gives us good visibility” and later provided many references toward supply coming online in Q4, such as: “But overall, the 7% increase [in gross margin], I think, is very well supported given all of the new product ramps that we have going on in addition to some additional supply that’s coming in as we get into the fourth quarter.”
It never hurts to have two management teams agree on the larger broad-based issue. However, since those reports, we’ve seen analysts cast doubts on the effects of macro for the rest of the year: “[Mizuho analyst Rakesh] checks show hyperscale orders are seeing "pushbacks" but no cancels, with Q3 trending flat quarter-over-quarter and Q4 "potentially soft." Rakesh lowered estimates for AMD "with macro headwinds clouding the near-term outlook."
Marvell’s Products:
In six brief years, Marvell has pivoted away from consumer (storage) products as the revenue mix was previously 62% consumer/38% infrastructure to being 11% consumer/89% infrastructure today.
This was driven partly by hyperscalers building data center infrastructure and AI/ML driving the need for faster data speed. Inphi also contributed to this.
Data Center Segment
PAM Solutions:
Marvell offers 200-gig and 400-gig PAM-based electro-optics — and the company recently added 800-gig solutions. This market sees tailwinds from the need for more bandwidth as the electro-optics connect short distances and long distances to increase data rates. PAM4 has replaced NRZ data transmission with the benefit of doubling the bit rate.
Hyperscalers are going through an upgrade cycle that requires high bandwidth and port density. PAM4 connects networking ASICs and machines, like servers and AI machines. Digital-based PAM4 uses analog-to-digital converters to clean up the signal in the digital domain before converting it back to analog to transmit.
Artificial intelligence and machine learning drives demand for the 800-gig PAM to increase the speed of input-output and to process the data flows. This doubles the throughput (bandwidth) due to an 8x100Gpbs optical transceiver for inside and between AI clusters.
In the fiscal Q1 results ending in April, management had stated: “our first quarter results benefited from a ramp in volume shipments of our 800-gig PAM solutions at two large customers.” The company has also stated that their products will see increase demand with the release of more powerful CPUs.
COLORZ 400:
COLORZ allows regional data centers to be linked together in the same metro region to function as one single mega data center. COLORZ silicon photonics technology allows data centers in the same metropolitan region to function like a mega data center through a “network fabric.” This facilitates faster edge computing within an 80/120 km distance for 30-megawatt data centers as they will be linked together and function like a 120-megawatt data center.
“As artificial intelligence (AI), machine learning (ML) and high-performance computing (HPC) applications continue to drive greater bandwidth requirements, cloud-optimized 400G solutions are needed to support high-speed data center interconnections. These requirements can only be met through high bandwidth connectivity offered in a small, cost-effective form factor. The Marvell COLORZ II 400ZR enables cloud data centers the ability to increase the speed of data movement while keeping the power and cost low.”
Another press release stated the company shipped 100,000 units.
Here is what was said on the call about how/why the growth in the data center can continue:
Harlan Sur
Good afternoon. Thanks for taking my question. On the cloud optical connectivity business, this is both inside and between data centers, the upgrade cycles have been this really great multi-year tailwind for the team.
And if I look into next year, I believe that there's still at least one of the top four US hyperscale titans that's going to start the 400-gig PAM4 transition. You still have China CSPs that need to fire. You've got multiple customers on DR that's going to fire as well. Historically, like these transitions, I don't think have been impacted by a slowing macro demand environment. They're viewed as, I think, very strategic.
But is that how your cloud customers are thinking about these upgrades and your views on continued upgrade momentum in this segment for next year? And just relatedly, is the Innovium team on track to drive $150 million in revenues this year?
Matt Murphy
Hey. Thanks, Harlan. Yes, I think the first part of it, you got pretty well in terms of the transition on 200 and 400 gig PAM4 inside the data center. And then, the new ramps we're seeing in 400 gig ZR for DCI between data centers.
What I'd add on top of that is — which has been extremely strong and also, in some ways, a little bit of a constraint we've seen in terms of being able to keep up is, the demand on 800 gig, which is happening right now really around, obviously, very advanced AI workloads.
That is an area where, if we could obviously produce more material, we would be shipping it in Q3. So that's also a positive trend. So you've got sort of the transition going on all the way up to 800 gig, and that continues to look pretty good.
NOTE: Innovium is an acquisition that closed in 2021 and at time of acquisition was expected to add $150 million in revenue for CY2022/FY2023.
Compute Xpress Link (CXP): 2024-2025 Data Center Catalyst
Marvell is launching a new product line called CXL, which will improve how data centers add memory. Right now, a server must be opened to add DRAM and the DIMM slots are limited in number and don’t pass service history or bit-error history, which is needed by hyperscalers.
Memory pooling allows memory to scale independently from processors by taking memory for a task and then releasing the memory. The new fabric removes the need for local DRAM, which adds a bit of latency from 100ns to 140-160ns, however, there’s a possibility of adding a CXL accelerator to be more “cache coherent.”
The CXL switch will be used to accelerate protocol-level processing across ethernet, DPUs, SmartNICs and solid-state drive controllers (SSD).
What Marvell is proposing with CXL is a new server architecture to “dynamically assign memory resources between servers.” The result is boosted memory bandwidth and also the ability to enable memory pooling. The company sees a future where a new architecture will separate compute, memory and I/O racks with the interconnect being CXL.Partially-disaggregated racks are expected to deploy in 2024-2025.
Marvell is at the forefront of the shift toward “disaggregate memory from the CPU” because it currently supplies the optics that this new fabric will disrupt. Inphi is the leader in silicon optics, PAM-4, and the encoding of PAM-4 for PCIe 6.0.
2024 seems like a long ways off yet the market will be paying attention to this In Q2/Q3 2023.
“As you recall from our discussion last quarter, we see CXL as the next big evolution in cloud data centers that will enable us to increase our reach into the memory ecosystem and presents a multibillion-dollar SAM expansion opportunity for Marvell.
This includes a host of new products such as CXL expanders, cooling devices, switches and accelerators and the potential to embed CXL IP and a broad range of our data center products. Events and presentations at FMS strongly validated our excitement around CXL. This is the hottest topic at FMS with standing room-only presentations by many leading industry participants.
The Marvell booth, we demonstrated the industry's first CXL memory pooling solution, addressing the challenges related to memory scaling and cloud data centers. While the industry is still in the early stages of CXL adoption, we are working on closing significant opportunities right in front of us at key customers and envision a strong design win pipeline.”
Why Marvell for CXL?
There are a handful of companies going after the CXL opportunity. Marvell could be front runner as the company already works closely with memory OEMs by supplying HDD controllers, SSD controller and preamplifiers. The company also has an aggressive PCIe roadmap with the company shipping Gen 5 sockets whereas most SSD device are shipping Gen 4 solutions. Marvell is already investing in Gen 6, which in turn, attracts more Tier 1 memory OEMs.
Marvell acquired Tanzanite, a developer of advanced CXL technologies. The company plans to expand to CXL expanders, cooling devices, switches and accelerators.
The company has stated this will drive “a multibillion-dollar PAM expansion opportunity driven by CXL overtime.” (Note: Marvell is referring to PAM, their premiere product)
We will focus on this more next year. You can listen to a recent tech talk here on CXL. The presentation is located here. This is an article about Microsoft’s interest in CXL with a statement that “50% of their server costs are taken up by DRAM.”
Carrier Infrastructure:
The OCTEON processors and platform is an Arm-based compute architecture for embedded applications, such as wireless networking equipment including 5G, including switches, routers, firewalls and monitoring solutions.
The OCTEON DPU is used with SmartNICs and security accelerators with a 5nm design that delivers to the infrastructure industry the same processing node as consumer smart phones and high performance computing and shipped in 2021. The most recent release from last year was the OCTEON 10 DPU and Prestera carrier switches which combined consumes 50% less power than competitors (according to Marvell).
Marvell’s processors help 5G networks meet latency and bandwidth demand while also allowing the networks to upgrade as cellular standards evolve. Marvell also offers customized solutions, which is ideal for Tier 1 customers who can combine their IP with Marvell’s Arm v8 processors and accelerators.
Recently, Dell and Marvell partnered to develop a server-class accelerator card for 5G base stations based on Marvell’s arm-based OCTEON Fusion processor. The hardware accelerators deliver more processing power including processing solutions for smart radio heads to support massive MIMO antenna rays.
We wrote about MIMO a few years back in a reference guide: “Massive Multiple Input and Multiple Output (MIMO) sends the data through multiple data streams called layers, which increases parallelism and throughput. MIMO helps avoid lost signals with multipathing, which allows the base station to send multiple copies of the same signal for increased redundancy.
Note: The antenna array is one fundamental change to 5G infrastructure. The initial 5G rollout will use existing cell towers, however, newer, dedicated 5G network infrastructures will require many more antennas than used in previous generations. Read more.”
The distributed unit (DU) shares the load with the radio unit by running L1 functions on the RAN protocol. Marvell has been a proponent of OpenRAN with the O-RAN platform, which is an open protocol and open platform that allows Marvell’s hardware to be used with various software vendors. Facebook (Meta) is a partner with Facebook Connectivity.
DPU processors, or digital processing units, are gaining traction for 5G transport, 5G RAN intelligent controllers, edge computing and cloud data center workloads. These hardware accelerators enable high speed connectivity and can improve packet processing rates by 5X. DPUs are ideal for power sensitive edge applications. Marvell’s strength in DPUs is one reason it may be able to stave off competition, which in the narrow field of 5G base stations includes Qualcomm/HPE and Analog Devices. Beyond 5G, Marvell has other competitors for DPUs such as AMD/Pensando and Nvidia.
Regarding 5G, over 7 million of the Octeon processors have been used in 3G, 4G and 5G base stations with Tier 1 customers. In the past, we reported that Samsung and Nokia use Marvell, and supplying these particular companies was a tailwind when Huawei was blacklisted. More recently, Marvell has stated they have design wins with four of the top five global OEMs and next-tier OEMs building base station equipment. These design wins are based on the 5nm platform.
Marvell uses TSMC for the 5nm OCTEON DPUs and this is an advantage because Marvell has the 5nm now and is able to move quickly on a 3nm release.
Notably, 5G has been a long time coming but I do believe it will reward investors over the next few years. Technavio has a CAGR of 67% for 5G equipment through 2025. The growth trend of 5G/edge computing is not one that we plan to complacent on as it will provide the next leg up for substantial capex spending similar to data center capex spending.
Enterprise Networking:
Marvell sells ethernet switches and ethernet PHYs to IT managers and networking equipment manufacturers. The company uses DSP technology for CAT5e ethernet cables to supply data rates up to 5Gbps with support for CAT6 and CAT6a.
Management discussed on the call that the main driver for this market right now is wireless, specifically WiFi 6 as the wireless rate line is now faster than the wired rate. The call also pointed toward content per port going up in the transition to multi-gig. According to the CEO, “it's not like 10%, 20%, 30%. It's sort of multiples on a per port basis of where it was before.”
Increased enterprise share and content gains from wired and wireless enterprise networking drove 53% YoY revenue growth and 19% QoQ revenue growth.
Automotive:
Similar to the networking that Marvell supplies enterprises and the data center, Marvell also supplies auto manufacturers with ethernet PHY transceivers, camera bridges and switches for in-vehicle networks. This is used for things like collision detection, lane warnings, and autonomous driving.
Marvell believes Ethernet will be the backbone for connected and autonomous vehicles to connect the electronic control unit (ECUs), cameras, sensors, and central compute devices. The Ethernet device is called Brightlane.
ON Semi has partnered with Marvell on use cases such as pairing a standardized protocol, such Ethernet PHY, with ON’s portfolio of ultra-dynamic range image sensors.
Automotive was up 46% to $84 million, yet was down 6% sequentially. Management cited supply issues rather than demand. Marvell counts eight of the largest 10 OEMs worldwide and 36 OEMs total. The company believes revenue growth will be 40% next quarter.
Note on Consumer Market:
Marvell sells hard disc drives (HDD) and solid state disc (SSD) controllers. This is a weaker segment, declining 1% YoY and 8% sequentially to $164 million. For next quarter, Marvell expects revenue to be down 10% YoY and flat sequentially.
Conclusion:
There is a new, powerful trend on the way that is on par with the cloud computing trend. This trend of edge computing will rely on distributed computing rather than centralized processing. Both will exist and rely upon each other but edge computing will have a stronger growth trend when it breaks ground (by virtue of being new/rapidly expanding). Much of this will be in sync with the 5G buildout.
Marvell has the potential to be a strong stock during this buildout as the company provides the base station hardware, supports MIMO antenna rays, beamforming, and accelerates 5G transport and controllers which results in high-speed connectivity.
The company also provides electro-optics and silicon photonics for increased data rates and a network fabric for edge computing. The edge is defined as many things, but what all definitions can agree on, is that the edge needs superior connectivity/networking. Electro-optics, silicon photonics, DPUs, SmartNICs and ethernet in the data center are a warmup for Marvell supplying edge servers and edge devices. As this occurs, the demand for Marvell’s product suite will increase.
In addition to this, Marvell is thinking outside the box by focusing on restructuring memory while most companies are focused on more powerful chips. CXL drives down costs on DRAM and is likely to rapidly adopted by hyperscalers once it becomes available. There’s no guarantee that Marvell will be the one to win the contracts but it’s certainly a front runner.
This article was originally published on Forbes on Sep 23, 2022,04:33pm EDTForbes on Sep 23, 2022,04:33pm EDT
Nvidia had a big week with GTC 2022 and management is clearly ready to rumble against any excess inventory from crypto mining. The negative catalyst from crypto mining and Nvidia's price action is eerily similar to Q4 2018/Q1 2019 —- yet the company is not the same company it was four years ago. This is apparent by Nvidia flexing some major product muscle by timing it's best-ever gaming release and it's best-ever AI chip to hit the market in October.
We draw important parallels (pun intended) between the last crypto mining selloff and this selloff with key reasons as to why this time the stock's comeback will be quicker.
Nvidia stock has been in the clutches of a steep drawdown after the company has faced nearly every headwind imaginable: United States-China tensions, supply chain disruptions spanning many components, tough comps on the data center, tough comps on gaming, and a less-than-rosy macro environment.
The most impactful headwind, however, was Ethereum’s merge to Proof of Stake (PoS), which ultimately lowers demand for gaming GPUs. This contributed to a $2.5 billion cumulative miss in revenue driven by the gaming segment.
Nvidia’s stock performance in 2018 and 2022 feels eerily similar as the stock sold off 54% in 2018 specifically because of a gaming miss tied to crypto mining. Today, Nvidia is currently 57% YTD.
It took eighteen months for Nvidia to recover its all-time high from the Q4 2018 selloff (Sept 2018 through Feb 2020). Despite the uncanny similarity that 2018 and 2022 may have — Nvidia is actually a much stronger company today than it was four years ago.
Below, we discuss a few key reasons Nvidia stock will recover quicker this time around.
Drilling into Parallels Around the Gaming Miss
During the Q3 2018 results released in November 2018, Nvidia gave Q4 2018 revenue guidance of $2.7 billion, below the analysts’ consensus estimate of $3.4 billion. In January 2019, the company again lowered revenue guidance from $2.7 billion to $2.20 billion, which suggests a total revenue miss of $1.2 billion. Gaming revenue in Q3 2018 was $1.76 billion, up 13% YoY and down 2% QoQ. In Q4 2018, gaming revenue was $954 million, down 45% YoY and down 46% QoQ.
In the most recent quarter ending July 2022, the company missed on gaming with revenue of $2.04 billion, which is 33% lower than the year ago quarter and 44% lower sequentially. The company is expecting a further decline in gaming sequentially for Q3. According to one analyst on the call, they are modeling for a further 30% sequential decline in gaming and professional visualization offset by low to mid-single digit growth in data center and automotive. The CFO affirmed this understanding is correct.
After 2018, although it took Nvidia eighteen months to reclaim its all-time highs, in 2020-2021, Nvidia would go on to stage a remarkable turnaround as the stock led tech mega cap stocks in gains. This was not simply because all tech performed well during those years – if you compare Nvidia to Meta, Amazon and Google, you’ll see something unique occurred with Nvidia that caused the stock to outpace its peers. In all cases except Apple, Nvidia doubled, tripled or quadrupled the performance of other mega cap stocks.
Source: YCHARTS
Perhaps most impressive, Nvidia is still in the lead over all mega cap stocks despite a 57% drawdown this year. It’s the company’s past performance that makes it well worth the time to answer: can Nvidia do it again?
Sign up for I/O Fund's free newsletter with gains of up to 403% – Click hereSign up for I/O Fund's free newsletter with gains of up to 403% – Click hereClick here
Nvidia’s GeForce RTX 40 Series is Perfectly Timed
Next quarter, Nvidia was expected to report $6.92 billion and the company guided for $5.9 billion. This is down from $7.10 billion in Q3 of last year. This will be a 17% decline in revenue. Due to this, analysts expect Nvidia to end fiscal year 2023 with 0.8% revenue growth, or $27.13 billion in total revenue.
It’s not only the top line valuation that is affected by this cut in guidance but it’s the bottom line, as well. In previous quarters, high average sales prices drove $2 billion to $3 billion in operating profits and net profits, whereas in the most recent quarter, the company is reporting $500 million and $656 million, respectively.
The GAAP EPS reported was $0.26 compared to $0.94 in the year ago quarter. Adjusted EPS was $0.51 versus $1.04 for the year ago quarter.
Although it’s tempting to redirect the conversation toward higher-growth segments, the $2.5 billion total miss between two quarters came from gaming and it’s prudent for investors to start here (for now) when analyzing the stock for a potential recovery.
The company stated the miss was driven by both lower units and lower average sales prices including reduced consumer demand. The company is not commenting on crypto as they state they have no visibility here as to how the GPUs are being used, however, it’s certainly contributing to the bulk of this decline.
Notably, AMD reported gaming growth of 32% to $1.7 billion which provides a better picture of reduced gaming demand minus crypto. Nvidia believes some of their weakness is also from preparation for a new product generation that will be announced this month.
Per the earnings call, there are two ways that Nvidia plans to overcome the crypto mining selloff which could produce a faster rebound than 2018.
First, Nvidia is restricting supply on its current gaming model. Per the CFO: “Across those two quarters, the Q2 of ‘23, the Q3 of ‘23, we have likely undershipped gaming to our end demand significantly.”
Following the call, we estimated for our premium members that the amount undershipped is a minimum of $1 billion. The reason behind this is to help keep prices stable and to increase demand for the RTX 40 Series.
Second, Nvidia announced its GeForce RTX 40 Series at the GTC 2022 Conference this week.
The new Ada Lovelace architecture which uses 76 billion transistors and a 4nm production process. In the keynote, the CEO stated: “Nvidia engineers worked closely with TSMC to create the 4N process optimized for GPUs. This process let us integrate 76 billion transistors and over 18,000 CUDA cores, 70% more than the Ampere generation.”
The improvement from 8nm to 4nm means more transistors on the GPU, which results in better performance as the 4nm processes data faster.
In the gaming world, this much anticipated release is expected to be 2-4X faster than the RTX 3090 Ti. The flagship AD102 GPU model will have 144 individual streaming multiprocessors (SMs) in one die compared to 84 SMs in the Ampere architecture. As stated, the AD102 will also have a 70% increase in CUDA cores over the RTX 3090 Ti.
In addition to this, Nvidia is releasing a new feature called Shader Execution Reordering (SER) which will improve ray-tracing performance by 3X with 25% faster frame rates. Rather than deliver workloads sequentially, the GPUs are able to reorder the workloads to process more workloads at once which results in more power and better performance.
Deep learning super sampling (DLSS) refers to using AI to predict the next pixel. The new DLSS 3.0 not only predicts pixels but will also use AI to predict frames. This results in “up to four times” better performance over traditional rendering.
The first release date for the RTX4090 models is October 12th with a starting price of $1,599. There is a second release date in November for the RTX4080 models with prices of $1,199 and $899. Notably, mid-range RTX 40 series will outperform the previous generation’s high end models. This is due to the Ada Lovelace architecture which offers 1,400 Tensor TFLOPs versus 320 Tensor TFLOPs which means the DLSS is superior and the high-end RTX 30 Series cannot compete with the mid range RTX 40 series.
The popularity of this release will help determine if Nvidia can stage a comeback in the gaming segment. Here is what analysts are saying:
“Morgan Stanley analyst Joseph Moore said his "biggest takeaway" from the keynote at Nvidia's GTC conference were the higher prices of gaming GPUs, which increases his conviction about the pace of gaming revenue recovery next year. Prices that are 28% higher than the baseline price from two years ago for the higher volume 4080 should drive material growth in revenue, said Moore, who sees revenues in the gaming segment rebounding from the current quarter run rate of $5.5B or so to $9.5B next year.”
“Given the channel inventory work downs in the July and October quarters, the products should be "strong demand catalysts" into 2023, Harlan Sur of Chase tells investors in a research note.”
Nvidia Continues to Build a GPU Moat with H100
In 2018, we stated in our free newsletter that Nvidia had built a moat in the GPU-powered data center. This was a bold statement as the company would go on to have negative year-over-year data center revenue in 2019. Yet, fast-forward and it’s quite clear that Nvidia is unshakeable in this segment, which has surpassed gaming as Nvidia’s most valuable segment.
I’ve written quite a bit about Nvidia, which you can reference here and also here. However, I will keep it simple by saying the A100 GPU is what led the company’s gains since Q2 2020 (detailed here) and the Hopper H100 GPU is what will lead the company’s gains for the next two yearsdetailed here) and the Hopper H100 GPU is what will lead the company’s gains for the next two years.
In the most recent quarter, data center revenue of 61% is down from 83% last quarter yet accelerated YoY from 35% growth in the year ago quarter. The earnings call reviewed some of the challenges Nvidia faced in the quarter that led to the 1% sequential growth.
First, Chinese hyperscalers slowed their infrastructure investment this year yet the slowdown is unlikely to last much longer. Due to being a large market for Nvidia, the data center growth was impacted by this. The reason Nvidia was able to meet expectations is because “North America doubled year-over-year in revenues.” As of now, supplying the Chinese military is restricted for Nvidia, but this does not include supplying the hyperscalers.
Second, demand continues to outstrip supply yet there are many components to Nvidia’s systems and they are experiencing supply chain issues.
“We were challenged this quarter with a fair amount of supply chain challenges because as you know, we don’t just sell the GPU chip, but these systems are really complex with a large number of chips in the system components that we offer like HGX […] all of the components that have to come together for us to be able to deliver the final component.”
H100 Hopper Coming in October
On the earnings call, an analyst asked if the company expects data center growth to re-accelerate when Hopper ships: “Do you think that Hopper, as that comes fully available, it sounds like in fiscal 4Q, that you actually see Data Center growth reaccelerate as that product cycle materializes.”
The CFO Kress stated: “Our Data Center yes, we do expect it to grow. It may grow about what we just saw between Q1 and Q2. We’ll continue to look at it.”
I believe this means the data center will accelerate above 61% but not to exceed the 83% from Q1. Ultimately, the CFO may not have full visibility into Hopper sales until the units ship and are tested by customers, who in turn, often buy more if the product exceeds expectations.
On that note, the new 4nm chips are bound to impress. The H100 GPUs and the DGX H100 server pods and super pods offer Nvidia the next leg-up as the company has solved an important bandwidth issue.
Hopper tackles some of the bigger issues around previous generations like speeding up algorithms by offering dynamic programming on GPUs to break down problems to simpler subproblems. The new GPUs also boost bandwidth by 3X with SHARP in-networking computing and Infiniband Switches, and the H100 can leverage NVLink to connect eight H100s into one giant GPU for 640 billion transistors, 32 petaflops, 640GB of HBM3, and 24 terabytes per second of memory bandwidth.
The H100 has about 50% more memory and interface bandwidth than the A100. That’s 1.5X more bandwidth with the NVLink connection and PCIe 5.0 doubling the bandwidth of PCIe 4.0. The H100 will ship with support for 80GB of HBM3 memory at 3 TB/s speed
Where the H100 really stands apart is the leap in performance with about 3X more performance than the A100 and the H100 is up to 6X faster. The A100 lacked support for FP8 compute at default whereas the H100 will leverage a transformer engine to switch between FP8 and FP16, depending on the workload.
According to Nvidia, the H100 delivers 9X more throughput in AI training, and 16X to 30X more inference performance. The company also states in HPC application-specific workloads, the H100 is 7X faster. The goal of the H100 was not only to add more transistors and make the H100 faster, but to also offer function-specific optimizations. This is achieved through the transformer engine.
Last week, MLPerf published artificial intelligence performance tests. The parent company MLCommons provides the industry standard for benchmarking deep learning, AI training, AI inference and HPC. The H100 Tensor Core GPUs delivered 4.5X more performance than the A100 in offline scenarios and 3.9X more in the server scenario compared to its predecessor the A100.
The Hopper H100 GPUs are in full production and availability starts next month and will have over 50 server models by the end of the year and “dozens more in the first half of 2023.”
Sign up for I/O Fund's free newsletter with gains of up to 403% – Click hereSign up for I/O Fund's free newsletter with gains of up to 403% – Click hereClick here
Nvidia’s Automotive Opportunity is Massive
Nvidia’s lead in automotive across dozens of OEMs requires its own analysis, which we will write for our free newsletter subscribers next year. Hyperion 8 is shipping in 2024 and Hyperion 9 will ship in 2026. However, as long-term Nvidia investors, now is a good opportunity to remind my readers of the long-term vision for yet another large and sweeping revenue segment.
Although a small segment today of only $220 million, automotive grew 59% sequentially and 45% year-over-year. The company has a $11 billion automotive design win pipeline.
At GTC this week, Nvidia announced a new superchip named “Thor” which will deliver 2,000 teraflops of performance, up from 200 teraflops from the current generation “Orin.” The chip has a transformer engine which can process video data as a single perception frame and offers 8-bit floating point (FP8) precision to avoid task loss when converting model data from one platform to another platform.
More on the Omniverse
We’ve covered the Omniverse platform in the past including an interview with Nvidia’s Richard Kerris you can view here.
At GTC this week, Nvidia launched Omniverse Cloud, which is a infrastructure-as-a-service software offering to reduce the complexity around building 3D virtual worlds and assets. This removes the need for local compute power and opens up the ability for more creators to access 3D world creation.
Regarding the China Restrictions
The United States government is restricting sales of high-performance chips to China as Nvidia’s AI chips could be used for military purposes. A spokesperson for Nvidia stated the products where the new licensing requirement applies is the A100, H100 and systems that include DGX.
The restrictions apply to Russia yet Nvidia has stated there is no exposure to Russia for their products. In a recent SEC filing, the company stated: The Company’s outlook for its third fiscal quarter provided on August 24, 2022 included approximately $400 million in potential sales to China which may be subject to the new license requirement if customers do not want to purchase the Company’s alternative product offerings or if the USG does not grant licenses in a timely manner or denies licenses to significant customers.
At this time, Nvidia has applied for an exemption and there has also been a clarification that Nvidia can continue to develop the H100 in China through September 1, 2023 through the company’s Hong Kong facility.
Per the SEC Filing dated August 31, 2022:
The U.S. government has authorized exports, reexports, and in-country transfers needed to continue NVIDIA Corporation’s, or the Company’s, development of H100 integrated circuits after the Company filed its Current Report on Form 8-K with the U.S. Securities and Exchange Commission on August 31, 2022. The authorization also allows the Company to perform exports needed to provide support for U.S. customers of A100 through March 1, 2023. Additionally, the U.S. government authorized A100 and H100 order fulfillment and logistics through the Company’s Hong Kong facility through September 1, 2023.
Some analysts have stated that being granted an exemption is “feasible.” Mark Lipacis of Jefferies is modeling for a $200 million hit to October rather than the $400 million identified risk. Harlan Sur of JP Morgan noted AMD is working on getting export licenses for its customers and helping them transition to products that fall below the performance threshold to help mitigate the downside risk.
According to a new report, Nvidia has asked TSMC to rush high-end GPU orders before the US sanctions begin. The report says that TSMC has a special program to speed delivery of orders at a higher negotiated price and can help to cut the delivery time in half. This could lead to a surprise bump in Q4 revenue for the company.
Conclusion
Nvidia is not the same company that it was four years ago. In 2018, Nvidia was a gaming company with promising AI tailwinds. Today, Nvidia’s AI products serve nearly every enterprise company’s artificial intelligence and machine learning ambitions.
The company has an impressive launch schedule starting in October for two flagship products – the RTX 40 Series and the H100 GPU. The timing of these releases is no coincidence as it’s a rapid two months following the crypto/gaming revenue miss. Suffice to say, Nvidia’s management team is prepared to rumble —- putting its very best release in gaming and its most powerful AI chip to-date up against the crypto mining selloff. If history is any indication, the turnaround will only be a matter of time.
Please note: The I/O Fund conducts research and draws conclusions for the company’s portfolio. We then share that information with our readers and offer real-time trade notifications. This is not a guarantee of a stock’s performance and it is not financial advice. Please consult your personal financial advisor before buying any stock in the companies mentioned in this analysis. Beth Kindig and the I/O Fund own shares in NVDA at the time of writing and may own stocks pictured in the charts.
The Hopper architecture is ramping and it’s yet again going to disrupt the GPU and AI accelerator market. I’ve written quite a bit about Nvidia, which you can reference here. However, I will keep it simple by saying the A100 GPU is what led the company’s gains since Q2 2020 (detailed here) and the Hopper H100 GPU is what will lead the company’s gains for the next two years.
But first, we have to get over the gaming hump. This has singlehandedly taken Nvidia’s revenue down to +3% growth this quarter with Nvidia expected to report $8.2 billion in revenue which came in at $6.7 billion.
For next quarter, Nvidia was expected to report $6.92 billion and the company guided for $5.9 billion. This is down from $7.10 billion in Q3 of last year. This will be a 17% decline in revenue. The company is expected to end fiscal year 2023 with 1.2% revenue growth, or $27.24 billion in total revenue.
It’s not only the top line valuation that is affected by this cut in guidance but it’s the bottom line even more so. In previous quarters, high average sales prices drove $2 billion to $3 billion in operating profits and net profits, whereas in the most recent quarter, the company is reporting $500 million and $656 million, respectively. The GAAP EPS reported was $0.26 compared to $0.94 in the year ago quarter. Adjusted EPS was $0.51 versus $1.04 for the year ago quarter.
Data center revenue of 61% decelerated sequentially down from 83% last quarter yet accelerated YoY from 35% growth in the year ago quarter. Gaming revenue fell 33% YoY whereas it had grown 31% YoY in the previous quarter. Professional Visualization also fell 4% whereas it had grown 67% in the previous quarter and had 100% growth in previous quarters, as well. Automotive was up 45% and along with data center helped to absorb the fall-off from Gaming and ProViz.
Gaming Hump: How Long Will It Last?
The company missed on gaming with revenue of $2.04 billion, which is 33% lower than the year ago quarter and 44% lower sequentially. The company is expecting a further decline in gaming sequentially for Q3. According to one analyst on the call, they are modeling for a further 30% sequential decline in gaming and professional visualization offset by low to mid-single digit growth in data center and automotive. The CFO affirmed this understanding is correct.
This is driven by both lower units and lower average sales prices including reduced consumer demand. The company is not commenting on crypto as they state they have no visibility here as to how the GPUs are being used, however, it’s certainly contributing to the bulk of this decline.
Notably, AMD reported gaming growth of 32% to $1.7 billion which provides a better picture of reduced gaming demand. Nvidia believes some of their weakness is also from preparation for a new product generation that will be announced next month.
Here was the first question on the call:
C.J. MuseC.J. Muse
I think the question we all have is what is normalized revenues for gaming for you guys? Obviously, this is a challenge to you as well. But curious how you’re thinking about it today. Is the fiscal ‘20 recovery post the first half ‘19 correction an appropriate framework, or was that inflated by crypto as well? And I guess, as part of that, how do we think about the cascading in of the new product cycle? And is there potential for future reserves needed to be taken if gaming does not meet your new updated outlook? Thanks so much.I think the question we all have is what is normalized revenues for gaming for you guys? Obviously, this is a challenge to you as well. But curious how you’re thinking about it today. Is the fiscal ‘20 recovery post the first half ‘19 correction an appropriate framework, or was that inflated by crypto as well? And I guess, as part of that, how do we think about the cascading in of the new product cycle? And is there potential for future reserves needed to be taken if gaming does not meet your new updated outlook? Thanks so much.
Management avoided the crypto question and instead answered the following:
The CFO Collette Kress stated: “Across those two quarters, the Q2 of ‘23, the Q3 of ‘23, we have likely undershipped gaming to our end demand significantly. We expect that sell-through or essentially our end demand for those combined two quarters of Q2 and Q3 to be approximately $5 billion […].”We expect that sell-through or essentially our end demand for those combined two quarters of Q2 and Q3 to be approximately $5 billion […].”
She is referring to about $1 billion being under shipped (or reduced sell-in) if we assume flat growth for gaming next quarter as the company attempts to rebalance inventory. It would be even more of an under shipment if gaming does decline sequentially.
Note: the next-generation GeForce RTX 40 Series the company is referring to is to be announced in September at GTC 2022.
The CEO Jensen Huang stated: “Our strategy is to reduce the sell-in — reduce the sell-in this quarter, next quarter to let channel inventory correct. Obviously, we’re off the highs, and the macro condition turned sharply worse. And so, our first strategy is to reduce sell-in in the next couple of quarters to correct channel inventory. We’ve also instituted programs to price position our current products to prepare for next-generation products.”And so, our first strategy is to reduce sell-in in the next couple of quarters to correct channel inventory. We’ve also instituted programs to price position our current products to prepare for next-generation products.”
The next question was similar and also about gaming, which the CEO responded again that they are rebalancing the supply and demand by reducing the sell-in (or essentially limiting the supply side).
“We believe that by the end of the year, we’ll be in a good shape going into next year. And so, I hope that answers your question. But, the important thing is our sell-in rate is far below what is happening in the market for sell-throughs. The sell-through is solid, has increased 70% since pre-COVID. And so, the gaming market is really quite vibrant.”We believe that by the end of the year, we’ll be in a good shape going into next year. And so, I hope that answers your question. But, the important thing is our sell-in rate is far below what is happening in the market for sell-throughs. The sell-through is solid, has increased 70% since pre-COVID. And so, the gaming market is really quite vibrant.”
My takeaway is that we have two more quarters before gaming rebalances. Management said this again toward the end of the call: “Still, the fundamentals of gaming are strong. We’ll get through this over the next few months and go into next year with our new architecture.” Nvidia states their gaming GPUs command the Top 15 list for Steam with 1,350 titles and there are 20 million registered GeForce NOW members.
Data Center Has More Runway
The information on the call about the data center was especially interesting because the company met expectations at 61% growth yet saw many challenges in the quarter. The challenges resulted in 1% sequential growth. As detailed below, revenue from North American hyperscalers doubled revenue year-over-year and it was Chinese hyperscalers that weighed on growth.
Demand continues to outstrip supply yet there are many components to Nvidia’s systems and they are experiencing supply chain issues.
“We were challenged this quarter with a fair amount of supply chain challenges because as you know, we don’t just sell the GPU chip, but these systems are really complex with a large number of chips in the system components that we offer like HGX […] all of the components that have to come together for us to be able to deliver the final component.
And then furthermore, these data centers sit idle until the last piece comes together. And the last piece includes very complicated switches and very complicated NICs and networkings and cables. And so these — building these high-performance computing data centers at very large scale for the world’s cloud is not particularly easy. And so the supply chain challenges have been somewhat disruptive. But the demand is there.”
The CFO elaborated by saying: “Some of our supply arrived very late in the quarter. We had very little time from a logistics and availability to get those things out. Customers were impacted as well by availability of key third-party other components that we weren’t offering, which were slowing down some of their deployments. So what we did in our Q2 orders that couldn’t be delivered in Q3, given that some of these supply constraints existed, and we had Q3 demand where we did have supply in Q2.”
Management also discussed how Chinese hyperscalers slowed their infrastructure investment this year and how this slowdown can’t last forever. Due to being a large market for Nvidia, the data center growth was impacted by this. The reason Nvidia was able to meet expectations is because “North America doubled year-over-year in revenues.”
We’ve discussed in detail the Hopper H100 GPUs and the DGX and HGX systems, as well as the Grace CPUs, which you can reference here. According to management “With respect to Hopper, we’re in full production now. And we’re racing to get Hopper 2, all of the CSPs are dying to get them […] We expect to ship substantial Hoppers in Q4.”
An analyst snuck a question in asking if the company expects data center growth to re-accelerate when Hopper ships: “Do you think that Hopper, as that comes fully available, it sounds like in fiscal 4Q, that you actually see Data Center growth reaccelerate as that product cycle materializes.”
The CFO Kress stated: “Our Data Center yes, we do expect it to grow. It may grow about what we just saw between Q1 and Q2. We’ll continue to look at it.”
My note: the data center was at 83% growth for Q1 and 61% growth in Q2.
The CEO Huang stated: “The first thing I’d say, Aaron, is that we are selling in or we’re selling far below the market demand, far — excuse me, far below the market sell-through. And the reason for that is to allow the inventory the channel inventory, the OEM inventories to correct. And this allows us to prepare for our next generation. And our next generation has Hopper for compute, but we also have the next generation for computer graphics that will be coming to market.”
The takeaway is that the data center is very likely to re-accelerate from Hopper.
Transformers
Since our new thesis published in July discussed the importance of transformers, I wanted to pull out some comments on the call as it was the primary growth driver the CEO discussed. Notably, he discussed it many times in an effort to explain the importance of transformers to the company’s strategy moving forward.
“And then, of course, over the last several years, a very important model has emerged called transformers. You and I’ve spoken about this model several times in the past. And it’s been found that this transformer model, this large language — this language model, which when scaled up in size, exhibits really spectacular and effective capabilities for — to be used to learn skills with either few shots or almost no shot, meaning it could learn skills, it could perform skills that it has never learned because the knowledge was somehow encoded from the large amount of data that it had learned from.”it could perform skills that it has never learned because the knowledge was somehow encoded from the large amount of data that it had learned from.”
The CEO elaborated again on Transformers when he was asked about whether Hopper can help re-accelerate the company’s data center revenue:
“Hopper is a giant new generation because it is designed to perform this new type of AI model called Transformers. It has an engine inside it called Transformer engine with numerical formats and pipelines that allows us to do a spectacular job on Transformer-type of models, which includes large language models, but it also includes computer vision models that are now able to be processed with this new type of AI model called Transformers.Hopper is a giant new generation because it is designed to perform this new type of AI model called Transformers. It has an engine inside it called Transformer engine with numerical formats and pipelines that allows us to do a spectacular job on Transformer-type of models, which includes large language models, but it also includes computer vision models that are now able to be processed with this new type of AI model called Transformers.
And so I fully expect Hopper 2 to be the next springboard for future growth. And — and the importance of this new model, Transformer, can’t possibly be understated and can’t be overstated. This is the impact of this model across robotics, computer vision, languages, biology, chemistry, drug design is just really quite spectacular. And I’m sure that you’ve been hearing about this new breakthrough in AI, and Hopper was designed for this.”And so I fully expect Hopper 2 to be the next springboard for future growth.And — and the importance of this new model, Transformer, can’t possibly be understated and can’t be overstated. This is the impact of this model across robotics, computer vision, languages, biology, chemistry, drug design is just really quite spectacular. And I’m sure that you’ve been hearing about this new breakthrough in AI, and Hopper was designed for this.”
And, there were more comments which I’m inclined to continue quoting because I think this company is doing very important things that are being overlooked by the gaming miss. So, bear with me as I provide yet another quote:
“Hopper was designed for transformers. The new transformers was going to be important. Nobody could have predicted the profound importance of large language models […] And to have AI that was never trained on a particular skill and yet within 1 shot or 1 shot of trying or even no shots, are able to perform that skill is beyond anybody’s expectations, I would think. And so I think the — the success of Hopper is — reflects the amount of work and pent-up demand for large training systems that Hopper is going to go into. If that’s an indicator, I think Hopper is going to be a spectacular success.”And so I think the — the success of Hopper is — reflects the amount of work and pent-up demand for large training systems that Hopper is going to go into. If that’s an indicator, I think Hopper is going to be a spectacular success.”
Automotive
The word “inflection” was used for automotive. Although a small segment of only $220 million, it grew 59% sequentially and 45% year-over-year. The company has a $11 billion automotive design win pipeline. This segment was the focus of a recent deep dive so I’ll keep it simple for now and just say there are promising things happening here and this may have been the first quarter of many where we see automotive continue to grow quickly.
Professional Visualization
Professional Visualization was a blemish this quarter and is expected to decline sequentially next quarter. Of the 30% sequential decline expected in Q3 in gaming and professional visualization, one-fourth will come from this segment and three-fourths from the gaming segment.
Analysts were poking around to see if this means enterprise spending is weaker than anticipated but I believe it simply means the Omniverse is discretionary compared to automotive and data center (which are industries that are very competitive at the moment).
Conclusion:
We have a high conviction company taking a breather on growth and each investor should approach this in a way that’s best for them. Some will decide to hold and ignore the noise, and others will want to re-allocate for the next quarter to a stronger company fundamentally in CY2022. There could be signs of a stock bottoming but this is different than a stock rallying. How I/O Fund handles this is subjectively up to us, but we will of course disclose our trades in the event they are useful.
I’ve given Knox the green light to trim from Nvidia 2-3% and add this to AMD, which I believe is a bit fundamentally stronger right now. AMD doesn’t have a gaming hump to get over and I don’t have a strong feeling if one has the leading allocation over the other for a period of time so we will see how we adjust here as they are both high conviction. Certainly, Nvidia is a high valuation, and considering the time-out the company is going to be taking for a quarter or two on revenue growth, we have to be realistic on what the stock price will be capable of compared to its peer AMD. Those are my thoughts fundamentally, which I covered here for AMD, but we will use technicals to guide this, as well. I wrote something similar in a brief note on Wednesday night.
Anything we trim from Nvidia now will be added back to take full advantage of the Hopper-inspired data center growth and the automotive (positive) surprises we have in store over the next few years.
Note: Information above has been updated 08/27 to reflect new analyst expectations for fiscal year 2023 of 1.2% revenue growth, or $27.2 billion.
This article was originally published on Forbes on Aug 12, 2022,01:21 pm EDTForbes on Aug 12, 2022,01:21 pm EDT
Semiconductor stocks have gained prominence due to growth drivers such as artificial intelligence, high-performance computing, 5G, robotics, machine learning, and electric vehicles. Despite semiconductor companies underperforming YTD, there is evidence that more supply will come online by the end of the year that will be met with equal or greater demand. Here is what AMD stated in their most recent earnings call:
“Certainly, on the Embedded side, we were supply constrained in the second quarter. And even on the Server side, we were tight in the second quarter. We have additional supply that’s coming online, especially as we get towards the end of the year. That will help us really meet more of the demand from customers. So, we feel pretty good about all of those puts and takes.”
Below, we review the stocks in the sector to find out which companies stand out in terms of revenue growth, profits, cash flows, and earnings surprise.
Sign up for I/O Fund's free newsletter with gains of up to 403% – Click hereSign up for I/O Fund's free newsletter with gains of up to 403% – Click hereClick here
Top Semiconductor stocks with the highest revenue growth rates for the current fiscal year
Revenue Growth Estimate for Current Fiscal Year – SOURCE: YCHARTS AND SEEKING ALPHA
Indie Semiconductor is leading with the expected year-over-year growth of 131% in the current fiscal year. The company is benefitting from the growth trend in advanced-driver assistance systems and electric vehicles. The company expects to be profitable by the end of 2023. The company has a Serviceable Addressable Market (SAM) of $40 billion by 2026. The company supplies chips and software to the automobile sector. Its chips power sensor capabilities like LiDAR and Radar, and vehicle electrification.
Monolithic Power Systems (MPWR) is expected to grow 50% in the current fiscal year. The company’s recent Q2 2022 results were strong. Revenue grew by 57% YoY to $461 million, beat the analysts' estimates by $30.41 million. The adjusted EPS came at $3.25 and beat estimates by $0.31. The Storage & Computing revenue grew by 112% YoY to $122 million; enterprise data revenue grew by 118% YoY to $65 million, and automotive grew by 25% YoY to $61 million. The management expects Q3 revenue of $490 million, representing a 51% YoY growth at the mid-point of the guidance. It was also significantly higher than the analysts' initial estimate of $400 million.
Top Semiconductor stocks with the highest revenue growth rates for the next fiscal year
Revenue growth estimate for the next fiscal year – SOURCE: YCHARTS AND SEEKING ALPHA
Aehr Test Systems has developed a unique technology that provides tangible benefits for testing emerging semiconductor components, such as silicon carbide and silicon photonics. Silicon carbide (SiC) is increasingly being used in EVs, while silicon photonics is being integrated into edge computing data centers. Tesla was the first to start using SiC in its vehicles with its Model 3. More EV manufacturers could follow suit due to SiC’s ability to withstand hostile conditions, improve efficiencies, and lower failure rates.
The company’s recent fiscal year ending May 2022 results were strong as revenue grew by 206% YoY to $50.8 million. The adjusted net income was $11.7 million or $0.42 per share compared to an adjusted net loss of $3.2 million or $(0.13) per share in the previous year. The management has guided revenue of $65 million for the FY ending May 2023, representing a YoY growth of 28% at the mid-point. The analyst expects revenue to grow 22% in FY ending May 2023 and 60% in the next fiscal year ending May 2024.
Sign up for I/O Fund's free newsletter with gains of up to 403% – Click hereSign up for I/O Fund's free newsletter with gains of up to 403% – Click hereClick here
Semiconductor Stocks with Top Forward P/S multiples
PS Ratio (Forward) – SOURCE: YCHARTS
The companies that outperform the market deserve a premium valuation. Nvidia is leading the sector. Nvidia has a solid long-term growth prospect in AI data centers and from the automotive chips. Similarly, Wolfspeed, which is a leading company in Silicon Carbide Technology, has a premium valuation.
Ambarella is another notable company trading at a fwd P/S ratio of 10. The company’s chips which were previously popular for using in drones and cameras have recently found a niche in the automobile sector. The company’s AI computer vision chips benefit from the Internet of Things, ADAS, and autonomous driving.
Quarterly Revenue Surprise
Quarterly Revenue Surprise – SOURCE: YCHARTS
Semiconductor Equipment Company ACM Research crushed the analyst’s consensus revenue estimates by 44%. The company’s Q2 revenue grew by 94% YoY to $104.4 million. The revenue also included $12.9 million that could not be shipped in Q1 due to the Covid-related restrictions in China. The company also maintained the revenue guidance for the year 2022 in the range of $365 million to $405 million, representing a YoY growth of 48% at the mid-point of the guidance.
Texas Instruments beat analysts' revenue estimates by 12%. The company’s Q2 revenue grew by 14% YoY to $5.2 billion. Susquehanna analyst Christopher Rolland in a note to the clients said, "[Texas Instruments] reported better results and guidance, in part as management overestimated China shutdown impacts of ~10% of [second-quarter] sales (~$500mln), and in part on the back of solid Automotive and Industrial demand,"
Top ranked semiconductor stocks based on Free Cash Flow Margin
Top ranked semiconductor stocks based on Free Cash Flow Margin – SOURCE: YCHARTS
Companies with a high cash flow margin also have a premium valuation. ASML Holding is leading the sector with the highest free cash flow margin. This is an important financial metric in the current environment, and we have noticed in the last few earnings seasons that shares were sold off when companies fell short on this metric.
Top ranked semiconductor stocks based on Net Profit Margin
Top ranked semiconductor stocks based on Net Profit Margin – SOURCE: YCHARTS
Texas Instruments leads the sector in this metric with a 44% net profit margin in the company’s recent quarterly results. Leading foundry, Taiwan Semiconductor, ranks second with a 41% net profit margin. TSMC’s revenue growth was strong, with good profits and cash flows also helped by the hike in chip production prices for its clients.
Royston Roche, Equity Analyst at the I/O Fund, contributed to this article.
Please note: The I/O Fund conducts research and draws conclusions for the Fund’s positions. We then share that information with our readers. This is not a guarantee of a stock’s performance. Please consult your personal financial advisor before buying any stock in the companies mentioned in this analysis.
If you were to guess, when do you think we wrote the following paragraph?
“When a thesis is not reflected in the revenue segments yet, there are typically lower entry points and ongoing volatility. You’ll see in the technical analysis that although I could not be more bullish on this stock long-term, there is weakness in the semiconductor sector and we hope this translates to a lower entry point for our readers.
The market is also in a fierce debate between AMD, Intel, and Nvidia and is also distracted by other chips, such as Micron and NXP. In my analysis, I look for growth. How big is the market relative to how big the company is now?
You can ignore Nvidia’s gaming revenue and other segments for the main trajectory that we are focused on. Gaming is great for stability and earnings reports, but the growth will not be from gaming (a market where Nvidia is already a mature, market leader). I’m also not focused on PC sales or the CPU-powered cloud, as the first is not a growth market and the second is not the piece in the cloud stack that will accelerate future technologies.”
That was written in 2019 yet the far majority of those concerns could be stated verbatim right now. Do we care about PC sales or gaming consoles? No, although our stance is that we have to expect these concerns will affect our semiconductor positions at times. The good news for Nvidia and AMD investors is that as time goes on, the less consumer-related hardware will have an impact. The 2022 Nvidia Investors Presentation provided numbers which show in detail how consumer exposure will become less of a concern in the future for these AI heavyweights.
When do you think we wrote this analysis?
“Over the past few weeks, I have read many lagging explanations on the chip shortage – too many fabless semiconductor companies, too few foundries, automobile manufacturers paused ordering in March and didn’t prepare for the sharp rebound, tensions with China, and even a fire at the Asahi Kasei plant that specifically manufactures sensing devices for the automobile industry.
While all of these are true, the overarching issue is that the role of semiconductors has changed from a commodity to the primary accelerant of future technologies. This is because connectivity, automation, and ultimately AI, will disrupt every corner of every industry.
We saw this happen with data and cloud but now we must accelerate this to the next level for AI/ML and the common denominator is semiconductors. Automotive is only the beginning. We can add renewables to the list and even e-commerce as AR/VR and AI/ML attempt to prop up the leaders who are competitive enough to add these features first.
As a tech stock analyst, I don’t have the luxury of lagging analysis of any kind. My subscribers require (and deserve) forward-looking, and with my intense focus on semiconductor chips, I don’t think my readers are surprised that semis are under pressure due to an increasingly important role.
I have repeated (perhaps too many times) that there is no way forward without the semis. We are seeing this manifest in automotive right now, but as investors, we should get used to hearing about semiconductor shortages.
You and I can debate Palantir, Snowflake or C3.AI, for example, and the valuations or the right angle for AI/ML-driven software, but the common denominator to these companies is the need for semiconductors to drive forward AI and 5G.
Now, we add the enormous push for auto manufacturers to compete with Tesla, Apple, Lucid Motors and what we have is a bottle neck where the automotive industry filters into semiconductors.
My guess is the demand won’t be letting up for many years as we are no longer in the cyclical pattern that semis are notorious for. Instead, demand will outpace supply for years to come.
Is this a bad thing or a good thing for our stocks? As investors, we can either listen to the news or listen to management. In this case, they are not aligned. Machines trade off news and natural language processing (NLP) but as human investors, we have the advantage of looking deeper into the issues.
I have written volumes of analysis leading up to the triple-digit growth we are seeing now in the data center from AI accelerator chips. Most of this was written when data center growth was negative. For instance, my Nvidia thesis was set end of 2018 — and in 2019 Nvidia reported negative data center revenue year-over-year for four quarters in a row.reported negative data center revenue year-over-year for four quarters in a row.
I mention this because following a trend’s trajectory is more important than immediate gratification from the market. The trend will always win out over time.
I have maintained that chips will eventually lead the AI market and are the best angle for investing in edge computing. I have also defended our stocks against custom silicon. Now we have the first of what I predict will be many semiconductor shortages and bullish to me.
The shortage is that there are hundreds (thousands really) of companies that rely on semiconductors. This will come to a head with AI and 5G as those who go-to-market soon with these features will have an enormous competitive advantage.”
That was written at the height of the bull market in February of 2021. My goal is to illustrate there has always been headlines to worry about for the semiconductors. We’ve firmly held these stocks and bought during dips. In the past, from 2018-2019, I focused on the GPU-powered cloud and the CUDA moat here and here. Our 2020 coverage centered on the A100 GPU which we discussed at time of launch for premium here and continued coverage on the A100 about a year later on the free side.
Here is background on the A100:
“Nvidia released the Ampere architecture and A100 GPU as an upgrade from the Volta architecture. The A100 GPUs are able to unify training and inference on a single chip, whereas in the past Nvidia’s GPUs were mainly used for training. This allows Nvidia a competitive advantage by offering both training and inferencing. The result is a 20x performance boost from a multi-instance GPU that allows many GPUs to look like one GPU. The A100 offers the largest leap in performance to date over the past 8 generations.”
Nvidia's AI Dominance Will be Propelled Forward by Software:
I wanted to go back through a bit of Nvidia’s history – what was the thesis and how did the thesis evolve? – before I go into how Nvidia will continue to dominate. In my opinion, I believe this is the most important analysis I have ever written on Nvidia because the company is changing rapidly into a software company.
The shift that Nvidia is going through has gone unnoticed and that’s to our benefit. Because we have been hell bent on finding what companies will dominate AI hardware, I’ve been asked frequently who do I think will dominate AI software (Palantir? Snowflake? Google?)
I’m prepared to give you that answer today: I believe Nvidia will be one of the biggest or perhaps the biggest AI software stack company in the world.the biggest AI software stack company in the world. The analysis below kickstarts our in-depth coverage on this new thesis — and I fully believe I will be quoting this analysis in five years from now when we check back on how the AI software thesis played out.
Before I go into semiconductor jargon where I risk losing your attention, I want to make sure our Members are fully aware that the segment where Nvidia will dominate with AI software is automotive. I am not talking about a few OEMs that trickle into a little bump in revenue. I am saying that Automotive is scheduled to become Nvidia’s number one segment – even over data centers – and to the tune of it being 3X larger than its gaming segment.
Don’t take my word for it because the CFO said exactly that (more on this below) and there is ample evidence that this is happening, which I also detail for you. Wall Street won’t be giving this the credit it deserves until 2023 at the earliest but you will hear non-stop “Nvidia Automotive” coverage by 2024-2026 as this segment ramps. I go over why those are the target dates below.
But first, let’s talk about the H100 and how this new GPU architecture will also help Nvidia lead on AI software at the enterprise level. There is plenty going on outside of Automotive that we need to cover so I kept automotive for last.
GTC Highlights: The Hopper H100 GPU
In March at GTC 2022, Nvidia announced the Hopper H100 GPU with 80 billion transistors and will be released in Q3 of this year. For reference, the A100 has 54 billion transistors. This is Nvidia’s solid attempt to keep their stake in the ground in leading high-performance computing over AMD’s Instinct MI250/250X and the newly announced MI210.
It’s easy to focus on hardware with Nvidia (and AMD) yet these companies are becoming more software-driven each year. By owning the majority of the market for AI accelerators, these two companies are afforded an opportunity to also own the software layer as a means to lower the barrier to entry for training models, deploying inference across various frameworks, and other workloads related to deep learning, conversational AI, video conferencing algorithms, and more. By supplying the hardware, these companies have natural inroads to machine learning operations (MLOps).
The H100 is the New Artificial Intelligence Infrastructure
DGX, DGX Pods and DGX SuperPods:
The H100 will power all AI and high-performance computing systems including the PCI express accelerator for mainstream servers and many H100 GPUs can be combined to power advanced AI through the following systems: DGX, DGX Pod and DGX SuperPod.
The difference between the A100 and H100 is the performance will be two to three times faster. The H100 GPUs and the DGX H100 server pods and super pods offer Nvidia the next leg-up as the company has solved an important bandwidth issue.
Hopper tackles some of the bigger issues around previous generations like speeding up algorithms by offering dynamic programming on GPUs to break down problems to simpler subproblems, boosting bandwidth by 3X with SHARP in-networking computing and Infiniband Switches, and the H100 can leverage NVLink to connect eight H100s into one giant GPU for 640 billion transistors, 32 petaflops, 640GB of HBM3, and 24 terabytes per second of memory bandwidth.
The chip is custom built by Taiwan Semiconductors with a 4nm design making it the world’s fastest 4nm GPU. The H100 has about 50% more memory and interface bandwidth than the A100. That’s 1.5X more bandwidth with the NVLink connection and PCIe 5.0 doubling the bandwidth of PCIe 4.0. The H100 will ship with support for 80GB of HBM3 memory at 3 TB/s speed.
The NVLink is now able to link together server nodes to build a data center-sized GPU. NVLink was originally designed to bypass the PCIe slot and has become an important tool for chip-to-chip connectivity, especially for high-speed operations. There is a dedicated chip called the NVSwitch which has increased the NVLink’s bandwidth. The ultimate goal is to run 32 servers with their own operating systems to run a single job. NVLink will complement the InfiniBand networking for high-performance computing and NVLink will be default for all of Nvidia’s chips, including GPUs, CPUs, DPUs and SoCs.
Where the H100 really stands apart is the leap in performance with about 3X more performance than the A100 and the H100 is up to 6X faster. The leap in performance is measured by H100’s ability to deliver up to 4,000 TFLOPS of FP8 compute, 2,000 TFLOPS of FP16 compute and 1,000 TFLOPS of TF32 compute and 60 TLOPS of general purpose FP64 compute. The A100 lacked support for FP8 compute at default whereas the H100 will leverage a transformer engine to switch between FP8 and FP16, depending on the workload.
According to Nvidia, the H100 delivers 9X more throughput in AI training, and 16X to 30X more inference performance. The company also states in HPC application-specific workloads, the H100 is 7X faster. The goal of the H100 was not only to add more transistors and make the H100 faster, but to also offer function-specific optimizations. This is achieved through the transformer engine.
The architecture aims to answer one of the bigger challenges facing superfast compute, which is that moving data into traditional servers overloads the CPU and system memory and becomes bottlenecked by PCI-Express.
By improving the bandwidth issue, Nvidia’s goal is to create more demand for their DGX Pod and SuperPod Systems, which in turn, will create more demand for their software.
The H100 DGX Pod is a 32-node, 256-GPU system. The H100 DGX Pod connects 32 DGX systems using the NVLink Switch System to scale into a super-GPU capable of 768 terabytes per second. To compare, the entire internet requires 100 terabytes per second. This results in 1 exaflop of AI computing.
From there, multiple H100 DGX Pods can connect through the Infiniband Switch to scale DGX Superpods with thousands of H100 GPUs. DGX SuperPods are turnkey systems that power enterprise AI. DGX SuperPods were also available with the A100 yet the H100 will have 6X better performance with 1 exaflop of FP8 AI performance to run trillions of parameters (more on this below).
Spectrum-4 Ethernet Platform
Perhaps one of Nvidia’s most important advancements for the H100 is the ability to attach the network directly to the GPU to avoid bottlenecks at the CPU. This is accomplished by sending data with direct memory access at 50 gigabytes per second. Hopper HGX and DGX are networking and interconnects that facilitate moving data with an advanced networking processor called the CX7. The result is the H100 CNX that avoids bandwidth bottlenecks and frees the CPU and system memory to process other parts of the application.
The Spectrum Ethernet platform, which consists of a Spectrum-4 Switch, CX7 SmartNIC and Bluefield-3 DPU will be used for several of Nvidia’s AI platforms, such as Riva, Merlin and Omniverse. These workloads include natural language processing, recommenders, and digital twins and will be supported by a networking system that helps exchange large databases between nodes. Whereas traditional workloads required many connections exchanging small amounts of data, the workloads of the future will require data to be shared quickly between GPUs and storage. This is accomplished by bypassing the CPU and sending data directly to the GPU while using the network hardware to move the data.
This is ideal for enterprise use cases where people are more likely to use Ethernet while AI and HPC workloads continue to use the Quantum-2 based off Mellanox’s InfiniBand. Quantum-2 allows for in-network computing to do data reductions in the network. It’s also more likely that Ethernet is used for 5G and sensors.
Eos: The First Hopper AI Factory
Nvidia is building AI factories to compete with AI supercomputers, which are blueprints for AI infrastructure that can be adopted by cloud partners and enterprises.
Eos is built with 18 H100 SuperPods, with 576 DGX H100 systems and 360 NVLink Switches. Nvidia states EOS is 1.4X faster than the fastest supercomputer and offers 4X the AI processing of the world’s fastest supercomputer. This will deliver 18 EFLOPS of FP8 AI compute or 9 EFLOPS of FP16 compute.
Previously, FP16 was the standard for AI whereas FP8 is gaining more support to become the industry standard. Depending on what AI compute you use, benchmarks will not be apples-to-apples if FP8 is compared to FP64 performance. Right now, AMD’s Frontier supercomputer is #1 with 1.1 exaflops of FP64 performance compared to the upcoming Venado supercomputer’s 10 exaflops of FP8 performance.
The difference is that the smaller bit size allows for an economical way to achieve more speed when giving up a small amount of accuracy doesn’t make a critical difference. This also helps in the face of a slowing Moore’s Law. FP8 is most commonly used for inference yet may be used for training in the future due to boosting throughput. Following the release of the Hopper H100, Intel released Gaudi2 which supports FP8. Chip makers Graphcore, AMD and Qualcomm have recently pushed for an industry-standard for the low precision floating point format FP8 rather than integer formats.
Here is what Nvidia said in the GTC keynote:
But the trend in AI computing has been toward developing neural nets that lean on the lowest precision that will still yield an accurate result. The smaller formats compute faster and more efficiently, and they require less memory and memory bandwidth. The addition of 8-bit floating-point units in the H100 leads to a significant speedup—double the throughput compared to its 16-bit units”
DPX Instructions (ISA):
The H100 improves dynamic programming with DPX Instructions that will help specific AI Algorithms to perform up to 7X faster than previous GPUs and 40X faster than CPU-based algorithms. As algorithms require more complexity, the new set of DPX instructions will help break the complex problems down into simpler subproblems using GPUs instead of CPUs or FPGAs.
The DPX ISA are expected to be broadly available with the CUDA 12.0 release. Examples of where this will be useful include disease research and drug discovery where the process can be sped up 35X for real-time processing to match the rate of DNA sequencing. Route optimization and finding the shortest distance between destinations for use in factories and autonomous driving systems, or Floyd-Warshall acceleration, is boosted up to 40X compared to CPU-only servers. These instructions will also be used for quantum computing and SQL queries as dynamic programming can help find the optimal order for joining a set of tables.
GPU Confidential Computing:
Data is encrypted at-rest and in-transit yet is often unprotected during use. Meanwhile, the data used to train AI models is worth millions in investments and is trained from domain knowledge and company-proprietary data. The new H100 offers confidential computing whereas previously only CPUs offered the protection of both data and applications during use.
Nvidia is Becoming a Leading AI Software Company
It would be easy to read the information above and to assume Nvidia is improving its hardware. However, the company’s future resides in software which will remove some of the cyclicality of hardware revenue. I believe once Nvidia’s software revenue begins to reveal itself in earnings reports, the market will finally piece together the true potential of this AI powerhouse.
It’s both the hardware and the software stack that led me to say Nvidia will surpass Apple in 5 years. You know this story well: the relationship between a hardware company leveraging their position to capture the lion’s share of the software — because that’s exactly what Apple did.
There are four layers to Nvidia’s full-stack accelerated computing: hardware, system software, platform software and applications. Below, I discuss a few ways that Nvidia is capturing more of the software stack due to vendor lock-in effects from their dominance in hardware.
As stated, in the past, our focus was the GPU-powered data center. This was a four-year thesis from 2018 and we doubled up on the thesis in June of 2020 for the A100 release. I want to make sure and emphasize that Nvidia’s lesser-known catalyst is actually the software.
The H100 is helpful in maintaining a lead in GPUs, which is critical turf to protect with GPUs being the most popular AI accelerator, however — the AI/ML catalyst will be further fueled by the Nvidia’s lead in software. This is why the majority of who will remain the AI leader will be up to developers and not the C-suite partnerships on hardware that characterized Intel’s lead over the past few decades. The developers choose the frameworks, the SDKs, libraries and the other parts of the software stack, and because of this, they also choose the GPUs they build on rather than IT departments.
Transformers
The transformer engine is one of the key aspects of the H100. Transformers are becoming one of the most popular neural-network models by applying self-attention to detect how data elements in a series influence and depend on one another.
Sequential text, images and video data are used for self-supervised learning and pattern recognition, which results in more data being used to create better models. Prior to transformer models, labeled datasets had to be used to train neural networks. Transformer models eliminate this need by finding patterns between elements mathematically, which substantially opens up what datasets can be used and how quickly. Transformers are partial to the parallel processing that GPUs offer.
Google first introduced transformer models in 2017 and transformers are used in Google and Bing Search. Transformers also led to BERT models, which stands for Bidirectional Encoder Representations from Transformers, and is commonly used for text sequences. Transformers are also used in GPT-3 (it’s the T in GPT) which improved from 1.5 billion parameters to 175 billion parameters. GPT-3 has the ability to report on queries it has not been specifically trained on.
Nvidia and Microsoft recently worked on a Mega transformer model with 530 billion parameters and the future for AI engineers is trillion-parameter transformers and applications. The H100 is already prepping for this. According to Nvidia, the training needs for transformer models will increase 275-fold every two years compared to 8-fold for other models. The H100 GPU with its Transformer Engine supports the FP8 format to speed up training to support trillion-parameter models. This leads to transformer models that go from taking 5 days to train to becoming 6X faster to only taking 19 hours to train.
The transformer engine is software combined with the new hardware in the H100’s tensor cores. As discussed, the A100 was designed for floating-point numbers to 16 bits while the H100 is designed for 8 bits. This is helpful because AI models are moving toward neural nets that lean on the lowest precision and yet still yields an accurate result. In this case, 8 bits double the throughput of 16-bit units, compute faster and more efficiently, and they require less memory and memory bandwidth.
The main feature from the Transformer Engine is the ability to choose what precision is needed for each layer in the neural network at each step, transitioning between 8-bits, 16-bits, 32-bits, plus the H100 is able to do matrix math with two forms of 8-bit numbers with either 5-bits as the exponent or 4-bits as the exponent: E5M2 and E4M3. This is important because the E4M3 may be favored for back propagation while E5M2 may be favored for inferencing.
Pictured above: Nvidia is prepped to support model sizes growing up to 275X every two years
Triton Inference Server:
Nvidia offers AI frameworks to reduce time for developers throughout the AI workflow from data processing and ETL to deep learning model training and large-scale inferencing. These libraries include Dali, Rapids, Triton and Magnum I/O. The library supports all popular frameworks and offers pre-trained models and data pipelines.
Triton is open-source inference software that helps developers deploy models across GPUs and CPUs, it supports Tensor Flow and PyTorch, any query type and any model – such as Transformers or CNNs (used for image recognition) and RNNs (used in speech recognition). The inference engine helps developers take AI development from experimentation to production by removing the need for multiple inference servers and simplifying machine learning infrastructure on the backend.
MLOps (machine learning operations) helps developers with less ML expertise to train and deploy models yet there were limited use cases with little help in deploying custom models. Triton offers high performance inference and scalability with Dockers and Kubernetes while serving up to hundreds of models with the model control API. By supporting all popular frameworks, Triton helps developers avoid framework lock-in due to the consistent interface regardless of training framework or hardware.
Nvidia will Power the Lion’s Share of Automotive – and that means software licensing
Nvidia’s lead in automotive across dozens of OEMs requires its own deep dive. The reason I haven’t prioritized this is because Hyperion 8 is shipping in 2024 and Hyperion 9 will ship in 2026. However, as long-term investors, we should touch base now on the long-term vision for yet another large and sweeping revenue segment. In fact, automotive promises to be Nvidia’s largest segment by 2030 – so on that alone, imagine what Nvidia investors have in front of us.
Nvidia’s Orin SoC (system-on-a-chip) is designed for the neural networks that run robots and AVs at the edge. This is the central computer for the car. The Orin SoC is capable of 254 trillion operations per second by combining Nvidia GPUs with Arm CPU cores and TensorRT APIs. The goal is to help OEMs move from Level 2 autonomous systems to the elusive Level 5 and it stiffens the competition with Tesla’s FSD. Notably, at the release two years ago, Tesla pointed towards Orin’s power consumption as a potential issue for EV batteries but this has not stopped many competing EVs from adopting Nvidia’s in-vehicle hardware and DRIVE software stack.
The EV manufacturers that have already moved forward with Nvidia DRIVE Orin include: Nio, Xpeng, BYD, Lucid Group, Mercedes and Land Rover, GM Cruise — you name it, it’s probably in production with Nvidia at this moment. The company’s current automotive pipeline exceeds $11 billion over the next six years – expect this small blip of pipeline to grow exponentially.
Nvidia recently announced an upgrade to Orin called Atlan with 1,000 TOPS on one chip, or more than then Level 5 compute in AVs today. This chip will catapult forward the computing performance of AVs and is expected to be released in 2023.
Nvidia DRIVE is the operating system and software stack for vehicles that offers an execution environment and includes both security and over-the-air updates. DriveWorks is an SDK that enables self-driving applications. Drive AV offers key ingredients to an autonomous system, such as perception, mapping and planning modules. Regarding mapping, Nvidia DRIVE Map is a multi-modal drive engine that can map independently and has two map engines. Drive IX is open-source software that offers vision, voice and graphics for the user experience. (I will do a separate deep dive on Nvidia Automotive in 2023).
The entire autonomous platform is called Hyperion, which includes the compute and sensor toolkit. This includes the hardware plus a 360-degree camera, radar, lidar and ultrasonic sensor suite. As stated, Hyperion 8 ships in 2024 with Hyperion 9 shipping in 2026, which will double the processing speed and offer an increase in sensors. Nvidia offers open-source developer kits to help increase its compatibility across various projects.
Rather than train the vehicles on the road, Nvidia trains in simulation and can create virtual world obstacles for the vehicles to learn from. This is a different approach from companies like Tesla who have millions of cars on the road collecting data which they then augment for unusual events with a photorealistic simulator.
Tesla has the most data of any car manufacturer which helps the company competitively as more data equals better performing models especially in terms of object detection. More data from millions of cars on the roads also helps with prediction as Tesla collects data from incorrect predictions that can be added to the training set. By leveraging a prediction neural network, Tesla does not need to use human labeling or annotation and can instead use what’s called a temporal sequence of events — in other words, Tesla rewinds events and labels objects automatically with the use of a supercomputer.
The advantage here is that training neural networks correlates with the miles (which again, are substantial due to size of fleet on the road compared to competitors) rather than correlating with the need for human labeling. The result of automatic labeling is that Tesla is able to predict rare situations with more accuracy.
Where Nvidia delivers a strong advantage is the company has decades of history with graphics and simulation due to its gaming roots. As stated, Tesla also uses imitation learning and has a photorealistic simulator which uses vector space for labels and functions like a game engine. However, Nvidia has been quietly working on their simulation platform for many years internally despite only recently marketing Omniverse to the public. In this case, Nvidia has such a high-level of confidence in their simulation skills that they forego the real-life fleet to primarily train virtual 3D models. The company is also packaging the simulation platform for many other uses cases, such as AI factories, 5G networks, power plants and climate research. Developers can work with 3D tools through Python-based development.
To keep it simple, Tesla’s primary advantage is the data they have collected as no other EV/AV has collected this level of data from real drivers. To contrast, Nvidia has arguably the best simulation platform due to decades of graphics work. These digital twins are only now being widely marketed despite being in development for over 5 years. The license costs $9,000 and Nvidia has estimated its current addressable market is 20 million engineers. Notably, Nvidia’s Hyperion will also be deployed in millions of vehicles over time, offering similar levels of data as Tesla’s fleet.
The Tesla VS Nvidia debates have not formally begun but they are certainly in our future … so brace yourself. Ultimately, the way Nvidia stands apart is the company does not directly compete on manufacturing vehicles. This is something anyone can agree on. That means many OEMs will use Nvidia’s DRIVE system whereas Tesla is less likely to commercialize their software as they’re viewed as a main competitor.
As long as Nvidia continues to innovate and maintain a lead, the popularity of its DRIVE system is likely to remain due to the company’s strategic advantages in AI and supercomputing. The company did an excellent job of tackling the edge computing use case of autonomous vehicles first.
Hardware is only part of the equation. The long-term plan is for Nvidia to license software for autonomous vehicles, which will create a recurring revenue stream. The licensing fees will go well beyond Omniverse to include the actual owner of the vehicle paying a subscription fee to Nvidia for its software. Tesla does this with their AutoPilot software which has grown from $5,000 to $12,000 per vehicle.
$150 billion from Omniverse software – fees are charged to both users and robots/digital twins
$300 billion from Automotive – primarily software
What Nvidia is communicating is that software revenue will surpass hardware revenue long-term.
Here is what Kress stated: "Our software content per vehicle can be in the thousands of dollars over the lifetime of the vehicle compared to the hundreds of dollars for the hardware. And second, software scales with the installed base of vehicles, not annual production.”
Note on CUDA:
The software discussion on Nvidia is not complete without a mention of CUDA. We called this Nvidia’s moat back in 2018 and we continue to believe it provides an important moat. The CUDA-related libraries include frameworks that span quantum computing, robotics, 5G networks, cybersecurity and drug discovery. The universal skills taught around CUDA and Nvidia’s SDKs help to drive more business for Nvidia’s GPUs.
The primary risk right now is valuation as Nvidia trades 2X higher than its peers on both the top line sales valuations and on the bottom line with earnings and cash-based valuations. There’s probably equal risk in waiting for Nvidia to drop another 50% as there is in buying Nvidia at the 2X valuation. One reason Nvidia may be valued here is because it’s slowly becoming a software company. Regardless, Knox’s technicals help immensely in determining if the market will continue to award Nvidia it’s gold medal valuation or if the market will discount Nvidia based on sentiment-driven headlines. This is a position we plan to keep on building so you can keep an eye out for those trade alerts over the next few years.
Conclusion:
Finding great companies is only half the battle, fighting negative sentiment is the other half – and semis have no shortage of this in any market – hence our beginning quotes from 2019 and also 2021.
Nvidia is the strongest company in terms of product on the market today. That doesn’t mean semis won’t be a roller coaster – we should fully expect that semis will undulate in sentiment and price while we hold our stocks over many years. We can’t change the way Wall Street works — which is a pendulum that swings between value stocks and growth stocks — but we can describe in great detail why concerns around gaming and consumer electronics slowing down is not going to change our position. We do not care to perfectly time entries or to find a perfect bottom – you’ll be hard pressed to find any legendary investor recommend that this be an investor’s goal. What we care about is finding quality companies and building those positions over time. Nvidia fits this description.
If you were to guess, when do you think we wrote the following paragraph?
“When a thesis is not reflected in the revenue segments yet, there are typically lower entry points and ongoing volatility. You’ll see in the technical analysis that although I could not be more bullish on this stock long-term, there is weakness in the semiconductor sector and we hope this translates to a lower entry point for our readers.
The market is also in a fierce debate between AMD, Intel, and Nvidia and is also distracted by other chips, such as Micron and NXP. In my analysis, I look for growth. How big is the market relative to how big the company is now?
You can ignore Nvidia’s gaming revenue and other segments for the main trajectory that we are focused on. Gaming is great for stability and earnings reports, but the growth will not be from gaming (a market where Nvidia is already a mature, market leader). I’m also not focused on PC sales or the CPU-powered cloud, as the first is not a growth market and the second is not the piece in the cloud stack that will accelerate future technologies.”
That was written in 2019 yet the far majority of those concerns could be stated verbatim right now. Do we care about PC sales or gaming consoles? No, although our stance is that we have to expect these concerns will affect our semiconductor positions at times. The good news for Nvidia and AMD investors is that as time goes on, the less consumer-related hardware will have an impact. The 2022 Nvidia Investors Presentation provided numbers which show in detail how consumer exposure will become less of a concern in the future for these AI heavyweights.
When do you think we wrote this analysis?
“Over the past few weeks, I have read many lagging explanations on the chip shortage – too many fabless semiconductor companies, too few foundries, automobile manufacturers paused ordering in March and didn’t prepare for the sharp rebound, tensions with China, and even a fire at the Asahi Kasei plant that specifically manufactures sensing devices for the automobile industry.
While all of these are true, the overarching issue is that the role of semiconductors has changed from a commodity to the primary accelerant of future technologies. This is because connectivity, automation, and ultimately AI, will disrupt every corner of every industry.
We saw this happen with data and cloud but now we must accelerate this to the next level for AI/ML and the common denominator is semiconductors. Automotive is only the beginning. We can add renewables to the list and even e-commerce as AR/VR and AI/ML attempt to prop up the leaders who are competitive enough to add these features first.
As a tech stock analyst, I don’t have the luxury of lagging analysis of any kind. My subscribers require (and deserve) forward-looking, and with my intense focus on semiconductor chips, I don’t think my readers are surprised that semis are under pressure due to an increasingly important role.
I have repeated (perhaps too many times) that there is no way forward without the semis. We are seeing this manifest in automotive right now, but as investors, we should get used to hearing about semiconductor shortages.
You and I can debate Palantir, Snowflake or C3.AI, for example, and the valuations or the right angle for AI/ML-driven software, but the common denominator to these companies is the need for semiconductors to drive forward AI and 5G.
Now, we add the enormous push for auto manufacturers to compete with Tesla, Apple, Lucid Motors and what we have is a bottle neck where the automotive industry filters into semiconductors.
My guess is the demand won’t be letting up for many years as we are no longer in the cyclical pattern that semis are notorious for. Instead, demand will outpace supply for years to come.
Is this a bad thing or a good thing for our stocks? As investors, we can either listen to the news or listen to management. In this case, they are not aligned. Machines trade off news and natural language processing (NLP) but as human investors, we have the advantage of looking deeper into the issues.
I have written volumes of analysis leading up to the triple-digit growth we are seeing now in the data center from AI accelerator chips. Most of this was written when data center growth was negative. For instance, my Nvidia thesis was set end of 2018 — and in 2019 Nvidia reported negative data center revenue year-over-year for four quarters in a row.reported negative data center revenue year-over-year for four quarters in a row.
I mention this because following a trend’s trajectory is more important than immediate gratification from the market. The trend will always win out over time.
I have maintained that chips will eventually lead the AI market and are the best angle for investing in edge computing. I have also defended our stocks against custom silicon. Now we have the first of what I predict will be many semiconductor shortages and bullish to me.
The shortage is that there are hundreds (thousands really) of companies that rely on semiconductors. This will come to a head with AI and 5G as those who go-to-market soon with these features will have an enormous competitive advantage.”
That was written at the height of the bull market in February of 2021. My goal is to illustrate there has always been headlines to worry about for the semiconductors. We’ve firmly held these stocks and bought during dips. In the past, from 2018-2019, I focused on the GPU-powered cloud and the CUDA moat here and here. Our 2020 coverage centered on the A100 GPU which we discussed at time of launch for premium here and continued coverage on the A100 about a year later on the free side.
Here is background on the A100:
“Nvidia released the Ampere architecture and A100 GPU as an upgrade from the Volta architecture. The A100 GPUs are able to unify training and inference on a single chip, whereas in the past Nvidia’s GPUs were mainly used for training. This allows Nvidia a competitive advantage by offering both training and inferencing. The result is a 20x performance boost from a multi-instance GPU that allows many GPUs to look like one GPU. The A100 offers the largest leap in performance to date over the past 8 generations.”
Nvidia's AI Dominance Will be Propelled Forward by Software:
I wanted to go back through a bit of Nvidia’s history – what was the thesis and how did the thesis evolve? – before I go into how Nvidia will continue to dominate. In my opinion, I believe this is the most important analysis I have ever written on Nvidia because the company is changing rapidly into a software company.
The shift that Nvidia is going through has gone unnoticed and that’s to our benefit. Because we have been hell bent on finding what companies will dominate AI hardware, I’ve been asked frequently who do I think will dominate AI software (Palantir? Snowflake? Google?)
I’m prepared to give you that answer today: I believe Nvidia will be one of the biggest or perhaps the biggest AI software stack company in the world.the biggest AI software stack company in the world. The analysis below kickstarts our in-depth coverage on this new thesis — and I fully believe I will be quoting this analysis in five years from now when we check back on how the AI software thesis played out.
Before I go into semiconductor jargon where I risk losing your attention, I want to make sure our Members are fully aware that the segment where Nvidia will dominate with AI software is automotive. I am not talking about a few OEMs that trickle into a little bump in revenue. I am saying that Automotive is scheduled to become Nvidia’s number one segment – even over data centers – and to the tune of it being 3X larger than its gaming segment.
Don’t take my word for it because the CFO said exactly that (more on this below) and there is ample evidence that this is happening, which I also detail for you. Wall Street won’t be giving this the credit it deserves until 2023 at the earliest but you will hear non-stop “Nvidia Automotive” coverage by 2024-2026 as this segment ramps. I go over why those are the target dates below.
But first, let’s talk about the H100 and how this new GPU architecture will also help Nvidia lead on AI software at the enterprise level. There is plenty going on outside of Automotive that we need to cover so I kept automotive for last.
GTC Highlights: The Hopper H100 GPU
In March at GTC 2022, Nvidia announced the Hopper H100 GPU with 80 billion transistors and will be released in Q3 of this year. For reference, the A100 has 54 billion transistors. This is Nvidia’s solid attempt to keep their stake in the ground in leading high-performance computing over AMD’s Instinct MI250/250X and the newly announced MI210.
It’s easy to focus on hardware with Nvidia (and AMD) yet these companies are becoming more software-driven each year. By owning the majority of the market for AI accelerators, these two companies are afforded an opportunity to also own the software layer as a means to lower the barrier to entry for training models, deploying inference across various frameworks, and other workloads related to deep learning, conversational AI, video conferencing algorithms, and more. By supplying the hardware, these companies have natural inroads to machine learning operations (MLOps).
The H100 is the New Artificial Intelligence Infrastructure
DGX, DGX Pods and DGX SuperPods:
The H100 will power all AI and high-performance computing systems including the PCI express accelerator for mainstream servers and many H100 GPUs can be combined to power advanced AI through the following systems: DGX, DGX Pod and DGX SuperPod.
The difference between the A100 and H100 is the performance will be two to three times faster. The H100 GPUs and the DGX H100 server pods and super pods offer Nvidia the next leg-up as the company has solved an important bandwidth issue.
Hopper tackles some of the bigger issues around previous generations like speeding up algorithms by offering dynamic programming on GPUs to break down problems to simpler subproblems, boosting bandwidth by 3X with SHARP in-networking computing and Infiniband Switches, and the H100 can leverage NVLink to connect eight H100s into one giant GPU for 640 billion transistors, 32 petaflops, 640GB of HBM3, and 24 terabytes per second of memory bandwidth.
The chip is custom built by Taiwan Semiconductors with a 4nm design making it the world’s fastest 4nm GPU. The H100 has about 50% more memory and interface bandwidth than the A100. That’s 1.5X more bandwidth with the NVLink connection and PCIe 5.0 doubling the bandwidth of PCIe 4.0. The H100 will ship with support for 80GB of HBM3 memory at 3 TB/s speed.
The NVLink is now able to link together server nodes to build a data center-sized GPU. NVLink was originally designed to bypass the PCIe slot and has become an important tool for chip-to-chip connectivity, especially for high-speed operations. There is a dedicated chip called the NVSwitch which has increased the NVLink’s bandwidth. The ultimate goal is to run 32 servers with their own operating systems to run a single job. NVLink will complement the InfiniBand networking for high-performance computing and NVLink will be default for all of Nvidia’s chips, including GPUs, CPUs, DPUs and SoCs.
Where the H100 really stands apart is the leap in performance with about 3X more performance than the A100 and the H100 is up to 6X faster. The leap in performance is measured by H100’s ability to deliver up to 4,000 TFLOPS of FP8 compute, 2,000 TFLOPS of FP16 compute and 1,000 TFLOPS of TF32 compute and 60 TLOPS of general purpose FP64 compute. The A100 lacked support for FP8 compute at default whereas the H100 will leverage a transformer engine to switch between FP8 and FP16, depending on the workload.
According to Nvidia, the H100 delivers 9X more throughput in AI training, and 16X to 30X more inference performance. The company also states in HPC application-specific workloads, the H100 is 7X faster. The goal of the H100 was not only to add more transistors and make the H100 faster, but to also offer function-specific optimizations. This is achieved through the transformer engine.
The architecture aims to answer one of the bigger challenges facing superfast compute, which is that moving data into traditional servers overloads the CPU and system memory and becomes bottlenecked by PCI-Express.
By improving the bandwidth issue, Nvidia’s goal is to create more demand for their DGX Pod and SuperPod Systems, which in turn, will create more demand for their software.
The H100 DGX Pod is a 32-node, 256-GPU system. The H100 DGX Pod connects 32 DGX systems using the NVLink Switch System to scale into a super-GPU capable of 768 terabytes per second. To compare, the entire internet requires 100 terabytes per second. This results in 1 exaflop of AI computing.
From there, multiple H100 DGX Pods can connect through the Infiniband Switch to scale DGX Superpods with thousands of H100 GPUs. DGX SuperPods are turnkey systems that power enterprise AI. DGX SuperPods were also available with the A100 yet the H100 will have 6X better performance with 1 exaflop of FP8 AI performance to run trillions of parameters (more on this below).
Spectrum-4 Ethernet Platform
Perhaps one of Nvidia’s most important advancements for the H100 is the ability to attach the network directly to the GPU to avoid bottlenecks at the CPU. This is accomplished by sending data with direct memory access at 50 gigabytes per second. Hopper HGX and DGX are networking and interconnects that facilitate moving data with an advanced networking processor called the CX7. The result is the H100 CNX that avoids bandwidth bottlenecks and frees the CPU and system memory to process other parts of the application.
The Spectrum Ethernet platform, which consists of a Spectrum-4 Switch, CX7 SmartNIC and Bluefield-3 DPU will be used for several of Nvidia’s AI platforms, such as Riva, Merlin and Omniverse. These workloads include natural language processing, recommenders, and digital twins and will be supported by a networking system that helps exchange large databases between nodes. Whereas traditional workloads required many connections exchanging small amounts of data, the workloads of the future will require data to be shared quickly between GPUs and storage. This is accomplished by bypassing the CPU and sending data directly to the GPU while using the network hardware to move the data.
This is ideal for enterprise use cases where people are more likely to use Ethernet while AI and HPC workloads continue to use the Quantum-2 based off Mellanox’s InfiniBand. Quantum-2 allows for in-network computing to do data reductions in the network. It’s also more likely that Ethernet is used for 5G and sensors.
Eos: The First Hopper AI Factory
Nvidia is building AI factories to compete with AI supercomputers, which are blueprints for AI infrastructure that can be adopted by cloud partners and enterprises.
Eos is built with 18 H100 SuperPods, with 576 DGX H100 systems and 360 NVLink Switches. Nvidia states EOS is 1.4X faster than the fastest supercomputer and offers 4X the AI processing of the world’s fastest supercomputer. This will deliver 18 EFLOPS of FP8 AI compute or 9 EFLOPS of FP16 compute.
Previously, FP16 was the standard for AI whereas FP8 is gaining more support to become the industry standard. Depending on what AI compute you use, benchmarks will not be apples-to-apples if FP8 is compared to FP64 performance. Right now, AMD’s Frontier supercomputer is #1 with 1.1 exaflops of FP64 performance compared to the upcoming Venado supercomputer’s 10 exaflops of FP8 performance.
The difference is that the smaller bit size allows for an economical way to achieve more speed when giving up a small amount of accuracy doesn’t make a critical difference. This also helps in the face of a slowing Moore’s Law. FP8 is most commonly used for inference yet may be used for training in the future due to boosting throughput. Following the release of the Hopper H100, Intel released Gaudi2 which supports FP8. Chip makers Graphcore, AMD and Qualcomm have recently pushed for an industry-standard for the low precision floating point format FP8 rather than integer formats.
Here is what Nvidia said in the GTC keynote:
But the trend in AI computing has been toward developing neural nets that lean on the lowest precision that will still yield an accurate result. The smaller formats compute faster and more efficiently, and they require less memory and memory bandwidth. The addition of 8-bit floating-point units in the H100 leads to a significant speedup—double the throughput compared to its 16-bit units”
DPX Instructions (ISA):
The H100 improves dynamic programming with DPX Instructions that will help specific AI Algorithms to perform up to 7X faster than previous GPUs and 40X faster than CPU-based algorithms. As algorithms require more complexity, the new set of DPX instructions will help break the complex problems down into simpler subproblems using GPUs instead of CPUs or FPGAs.
The DPX ISA are expected to be broadly available with the CUDA 12.0 release. Examples of where this will be useful include disease research and drug discovery where the process can be sped up 35X for real-time processing to match the rate of DNA sequencing. Route optimization and finding the shortest distance between destinations for use in factories and autonomous driving systems, or Floyd-Warshall acceleration, is boosted up to 40X compared to CPU-only servers. These instructions will also be used for quantum computing and SQL queries as dynamic programming can help find the optimal order for joining a set of tables.
GPU Confidential Computing:
Data is encrypted at-rest and in-transit yet is often unprotected during use. Meanwhile, the data used to train AI models is worth millions in investments and is trained from domain knowledge and company-proprietary data. The new H100 offers confidential computing whereas previously only CPUs offered the protection of both data and applications during use.
Nvidia is Becoming a Leading AI Software Company
It would be easy to read the information above and to assume Nvidia is improving its hardware. However, the company’s future resides in software which will remove some of the cyclicality of hardware revenue. I believe once Nvidia’s software revenue begins to reveal itself in earnings reports, the market will finally piece together the true potential of this AI powerhouse.
It’s both the hardware and the software stack that led me to say Nvidia will surpass Apple in 5 years. You know this story well: the relationship between a hardware company leveraging their position to capture the lion’s share of the software — because that’s exactly what Apple did.
There are four layers to Nvidia’s full-stack accelerated computing: hardware, system software, platform software and applications. Below, I discuss a few ways that Nvidia is capturing more of the software stack due to vendor lock-in effects from their dominance in hardware.
As stated, in the past, our focus was the GPU-powered data center. This was a four-year thesis from 2018 and we doubled up on the thesis in June of 2020 for the A100 release. I want to make sure and emphasize that Nvidia’s lesser-known catalyst is actually the software.
The H100 is helpful in maintaining a lead in GPUs, which is critical turf to protect with GPUs being the most popular AI accelerator, however — the AI/ML catalyst will be further fueled by the Nvidia’s lead in software. This is why the majority of who will remain the AI leader will be up to developers and not the C-suite partnerships on hardware that characterized Intel’s lead over the past few decades. The developers choose the frameworks, the SDKs, libraries and the other parts of the software stack, and because of this, they also choose the GPUs they build on rather than IT departments.
Transformers
The transformer engine is one of the key aspects of the H100. Transformers are becoming one of the most popular neural-network models by applying self-attention to detect how data elements in a series influence and depend on one another.
Sequential text, images and video data are used for self-supervised learning and pattern recognition, which results in more data being used to create better models. Prior to transformer models, labeled datasets had to be used to train neural networks. Transformer models eliminate this need by finding patterns between elements mathematically, which substantially opens up what datasets can be used and how quickly. Transformers are partial to the parallel processing that GPUs offer.
Google first introduced transformer models in 2017 and transformers are used in Google and Bing Search. Transformers also led to BERT models, which stands for Bidirectional Encoder Representations from Transformers, and is commonly used for text sequences. Transformers are also used in GPT-3 (it’s the T in GPT) which improved from 1.5 billion parameters to 175 billion parameters. GPT-3 has the ability to report on queries it has not been specifically trained on.
Nvidia and Microsoft recently worked on a Mega transformer model with 530 billion parameters and the future for AI engineers is trillion-parameter transformers and applications. The H100 is already prepping for this. According to Nvidia, the training needs for transformer models will increase 275-fold every two years compared to 8-fold for other models. The H100 GPU with its Transformer Engine supports the FP8 format to speed up training to support trillion-parameter models. This leads to transformer models that go from taking 5 days to train to becoming 6X faster to only taking 19 hours to train.
The transformer engine is software combined with the new hardware in the H100’s tensor cores. As discussed, the A100 was designed for floating-point numbers to 16 bits while the H100 is designed for 8 bits. This is helpful because AI models are moving toward neural nets that lean on the lowest precision and yet still yields an accurate result. In this case, 8 bits double the throughput of 16-bit units, compute faster and more efficiently, and they require less memory and memory bandwidth.
The main feature from the Transformer Engine is the ability to choose what precision is needed for each layer in the neural network at each step, transitioning between 8-bits, 16-bits, 32-bits, plus the H100 is able to do matrix math with two forms of 8-bit numbers with either 5-bits as the exponent or 4-bits as the exponent: E5M2 and E4M3. This is important because the E4M3 may be favored for back propagation while E5M2 may be favored for inferencing.
Pictured above: Nvidia is prepped to support model sizes growing up to 275X every two years
Triton Inference Server:
Nvidia offers AI frameworks to reduce time for developers throughout the AI workflow from data processing and ETL to deep learning model training and large-scale inferencing. These libraries include Dali, Rapids, Triton and Magnum I/O. The library supports all popular frameworks and offers pre-trained models and data pipelines.
Triton is open-source inference software that helps developers deploy models across GPUs and CPUs, it supports Tensor Flow and PyTorch, any query type and any model – such as Transformers or CNNs (used for image recognition) and RNNs (used in speech recognition). The inference engine helps developers take AI development from experimentation to production by removing the need for multiple inference servers and simplifying machine learning infrastructure on the backend.
MLOps (machine learning operations) helps developers with less ML expertise to train and deploy models yet there were limited use cases with little help in deploying custom models. Triton offers high performance inference and scalability with Dockers and Kubernetes while serving up to hundreds of models with the model control API. By supporting all popular frameworks, Triton helps developers avoid framework lock-in due to the consistent interface regardless of training framework or hardware.
Nvidia will Power the Lion’s Share of Automotive – and that means software licensing
Nvidia’s lead in automotive across dozens of OEMs requires its own deep dive. The reason I haven’t prioritized this is because Hyperion 8 is shipping in 2024 and Hyperion 9 will ship in 2026. However, as long-term investors, we should touch base now on the long-term vision for yet another large and sweeping revenue segment. In fact, automotive promises to be Nvidia’s largest segment by 2030 – so on that alone, imagine what Nvidia investors have in front of us.
Nvidia’s Orin SoC (system-on-a-chip) is designed for the neural networks that run robots and AVs at the edge. This is the central computer for the car. The Orin SoC is capable of 254 trillion operations per second by combining Nvidia GPUs with Arm CPU cores and TensorRT APIs. The goal is to help OEMs move from Level 2 autonomous systems to the elusive Level 5 and it stiffens the competition with Tesla’s FSD. Notably, at the release two years ago, Tesla pointed towards Orin’s power consumption as a potential issue for EV batteries but this has not stopped many competing EVs from adopting Nvidia’s in-vehicle hardware and DRIVE software stack.
The EV manufacturers that have already moved forward with Nvidia DRIVE Orin include: Nio, Xpeng, BYD, Lucid Group, Mercedes and Land Rover, GM Cruise — you name it, it’s probably in production with Nvidia at this moment. The company’s current automotive pipeline exceeds $11 billion over the next six years – expect this small blip of pipeline to grow exponentially.
Nvidia recently announced an upgrade to Orin called Atlan with 1,000 TOPS on one chip, or more than then Level 5 compute in AVs today. This chip will catapult forward the computing performance of AVs and is expected to be released in 2023.
Nvidia DRIVE is the operating system and software stack for vehicles that offers an execution environment and includes both security and over-the-air updates. DriveWorks is an SDK that enables self-driving applications. Drive AV offers key ingredients to an autonomous system, such as perception, mapping and planning modules. Regarding mapping, Nvidia DRIVE Map is a multi-modal drive engine that can map independently and has two map engines. Drive IX is open-source software that offers vision, voice and graphics for the user experience. (I will do a separate deep dive on Nvidia Automotive in 2023).
The entire autonomous platform is called Hyperion, which includes the compute and sensor toolkit. This includes the hardware plus a 360-degree camera, radar, lidar and ultrasonic sensor suite. As stated, Hyperion 8 ships in 2024 with Hyperion 9 shipping in 2026, which will double the processing speed and offer an increase in sensors. Nvidia offers open-source developer kits to help increase its compatibility across various projects.
Rather than train the vehicles on the road, Nvidia trains in simulation and can create virtual world obstacles for the vehicles to learn from. This is a different approach from companies like Tesla who have millions of cars on the road collecting data which they then augment for unusual events with a photorealistic simulator.
Tesla has the most data of any car manufacturer which helps the company competitively as more data equals better performing models especially in terms of object detection. More data from millions of cars on the roads also helps with prediction as Tesla collects data from incorrect predictions that can be added to the training set. By leveraging a prediction neural network, Tesla does not need to use human labeling or annotation and can instead use what’s called a temporal sequence of events — in other words, Tesla rewinds events and labels objects automatically with the use of a supercomputer.
The advantage here is that training neural networks correlates with the miles (which again, are substantial due to size of fleet on the road compared to competitors) rather than correlating with the need for human labeling. The result of automatic labeling is that Tesla is able to predict rare situations with more accuracy.
Where Nvidia delivers a strong advantage is the company has decades of history with graphics and simulation due to its gaming roots. As stated, Tesla also uses imitation learning and has a photorealistic simulator which uses vector space for labels and functions like a game engine. However, Nvidia has been quietly working on their simulation platform for many years internally despite only recently marketing Omniverse to the public. In this case, Nvidia has such a high-level of confidence in their simulation skills that they forego the real-life fleet to primarily train virtual 3D models. The company is also packaging the simulation platform for many other uses cases, such as AI factories, 5G networks, power plants and climate research. Developers can work with 3D tools through Python-based development.
To keep it simple, Tesla’s primary advantage is the data they have collected as no other EV/AV has collected this level of data from real drivers. To contrast, Nvidia has arguably the best simulation platform due to decades of graphics work. These digital twins are only now being widely marketed despite being in development for over 5 years. The license costs $9,000 and Nvidia has estimated its current addressable market is 20 million engineers. Notably, Nvidia’s Hyperion will also be deployed in millions of vehicles over time, offering similar levels of data as Tesla’s fleet.
The Tesla VS Nvidia debates have not formally begun but they are certainly in our future … so brace yourself. Ultimately, the way Nvidia stands apart is the company does not directly compete on manufacturing vehicles. This is something anyone can agree on. That means many OEMs will use Nvidia’s DRIVE system whereas Tesla is less likely to commercialize their software as they’re viewed as a main competitor.
As long as Nvidia continues to innovate and maintain a lead, the popularity of its DRIVE system is likely to remain due to the company’s strategic advantages in AI and supercomputing. The company did an excellent job of tackling the edge computing use case of autonomous vehicles first.
Hardware is only part of the equation. The long-term plan is for Nvidia to license software for autonomous vehicles, which will create a recurring revenue stream. The licensing fees will go well beyond Omniverse to include the actual owner of the vehicle paying a subscription fee to Nvidia for its software. Tesla does this with their AutoPilot software which has grown from $5,000 to $12,000 per vehicle.
$150 billion from Omniverse software – fees are charged to both users and robots/digital twins
$300 billion from Automotive – primarily software
What Nvidia is communicating is that software revenue will surpass hardware revenue long-term.
Here is what Kress stated: "Our software content per vehicle can be in the thousands of dollars over the lifetime of the vehicle compared to the hundreds of dollars for the hardware. And second, software scales with the installed base of vehicles, not annual production.”
Note on CUDA:
The software discussion on Nvidia is not complete without a mention of CUDA. We called this Nvidia’s moat back in 2018 and we continue to believe it provides an important moat. The CUDA-related libraries include frameworks that span quantum computing, robotics, 5G networks, cybersecurity and drug discovery. The universal skills taught around CUDA and Nvidia’s SDKs help to drive more business for Nvidia’s GPUs.
The primary risk right now is valuation as Nvidia trades 2X higher than its peers on both the top line sales valuations and on the bottom line with earnings and cash-based valuations. There’s probably equal risk in waiting for Nvidia to drop another 50% as there is in buying Nvidia at the 2X valuation. One reason Nvidia may be valued here is because it’s slowly becoming a software company. Regardless, Knox’s technicals help immensely in determining if the market will continue to award Nvidia it’s gold medal valuation or if the market will discount Nvidia based on sentiment-driven headlines. This is a position we plan to keep on building so you can keep an eye out for those trade alerts over the next few years.
Conclusion:
Finding great companies is only half the battle, fighting negative sentiment is the other half – and semis have no shortage of this in any market – hence our beginning quotes from 2019 and also 2021.
Nvidia is the strongest company in terms of product on the market today. That doesn’t mean semis won’t be a roller coaster – we should fully expect that semis will undulate in sentiment and price while we hold our stocks over many years. We can’t change the way Wall Street works — which is a pendulum that swings between value stocks and growth stocks — but we can describe in great detail why concerns around gaming and consumer electronics slowing down is not going to change our position. We do not care to perfectly time entries or to find a perfect bottom – you’ll be hard pressed to find any legendary investor recommend that this be an investor’s goal. What we care about is finding quality companies and building those positions over time. Nvidia fits this description.
