Big Data, Analytics (and ML): Microtrend Deep Dive

Posted on September 29, 2021June 30, 2026 by io-fund

There are three important trends we weave together in this report to draw conclusions around potential winners in big data and analytics. We’ve recently covered MongoDB, Elastic, and we’ve discussed Confluent. What do they all have in common and why are these companies important right now? That’s what we aim to answer in this write-up.

Before we go into where we are with big data and analytics right now, I’ll quickly touch on cloud IaaS and especially why hybrid and multi-cloud are leading this space and why investors should not be concerned with tech giants that offer competing products in the data and analytics space.

When we talk about Big Data, the main driver is machine learning, which is performed through supervised learning with the use of historical data, or unsupervised learning, with clustering models and associations to identify rules. There is also reinforcement learning that is trained from feedback. Here are the three trends we are going to weave together to form a full picture of Big Data and Analytics.

The migration to the cloud — but more specifically multi-cloud and hybrid
Why multi-cloud drives demand for best-of-breed, i.e., generally speaking, we do not need to be overly concerned when tech giants that offer competing products
How Apache Spark helped catalyze the AI/ML market with efficient data processing
How we plan to invest right now given that #2 and #3 are prepping the market for us

Hybrid and Multi-Cloud are Driving the Cloud IaaS Market:

For cloud IaaS, we don’t want to only focus on CAGR but also the budget allocation that cloud IaaS is capturing. According to IDC, the IaaS market will reach $112.9 billion at a CAGR of 11.3% through 2025 and will account for 66.1% of total compute and storage infrastructure spend. Two-thirds of IaaS spend is on the public cloud.

To compare, the on-premise market (i.e., not hybrid) will grow at 0.3% CAGR for a total of $57.9 billion. According to the most recent Denodo survey, hybrid cloud drove 35% of the workloads worldwide. Private cloud expanded from 16.6% of workloads to 24% percent of workloads worldwide. Meanwhile, the public cloud had flat growth.

Hybrid cloud is a mix of public and private clouds or a mix of cloud and on-premise. Enterprise companies that choose hybrid deployments are motivated to not share intellectual property or data with a vendor, known as data residency, plus other security implications that come with storing data on another company’s servers. Other companies find moving to the cloud to be time and resource-intensive and prefer to keep some workloads on the servers they own.

Recently, a report came out that repatriation, or moving some workloads back to on-premise, has resulted in quite a bit of cost savings for companies like Dropbox, Crowdstrike and Zscaler, who use hybrid approaches. The report is quite surprising as the conclusion is that $100 billion to $500 billion in market value is lost on cloud deployments in terms of margins. One use case that is detailed is Dropbox, a company that reported savings of $75 million in two years after repatriation, which in turn, helped the company’s gross margins increase from 33% to 67%. Meanwhile, companies like Asana and Datadog spend about 60% of their revenue towards committed cloud spend. This report, among others, shows why hybrid is likely to be the chosen deployment for many enterprises into the near future.

We had previously formed a Microsoft thesis in 2018 based on the trend towards hybrid cloud and why a focus on a hybrid strategy for governments and enterprises was important to Azure’s growth rate. Microsoft is especially well suited to serve the hybrid market because of the company’s deep roots with on-premise enterprise software. When the I/O Fund first covered hybrid cloud as a major driver of cloud IaaS in 2018, Amazon’s AWS did not even have a publicly available hybrid product. The company later publicly released Outposts in 2019 to compete with Azure. If you want more information about how these two compete on hybrid on-prem deployments specifically, the in-depth analysis I published in the past is found on Seeking Alpha and also Forbes.

Multi-cloud refers to using more than one cloud provider, which is usually done to avoid vendor lock-in and to choose best-of-breed products. It also helps to avoid downtime should one cloud provider go down or become overwhelmed with demand.

Source: Statista

Multi-cloud is the dominant strategy today and is used by 80% to 90% of organizations. In 2019, Gartner stated 81% of respondents were using two or more cloud providers. The top reason was to avoid vendor lock-in by the “megavendors.” Therefore, this is why investors should not be concerned with tech giants offering competing products in the data and analytics space. The far majority of companies are taking strides to avoid vendor lock-in as multi-cloud technically requires more work yet increases agility and flexibility. The end result is these companies will use best-in-breed products.

According to IBM, 98% of companies plan to use multiple hybrid clouds and 85% operate in multi-cloud environment. There is substantial evidence that organizations are preferring a mix of cloud providers. Most importantly to our thesis and this particular analysis, only 40% use management tools and/or have implemented DevOps practices. The migration to the cloud was happening slowly over time and this migration is under-served in terms of management tools, data and analytics. This would be a sufficient tailwind on its own yet we also have the additional tailwind of data-intensive industries that are moving into machine learning.

The motivation behind cloud IaaS growth and especially hybrid and multi-cloud growth is partially driven by the need for analytics, and also newer trends, such as stream processing. Stream processing is a continuous stream of events that is processed in real-time as it’s received. This allows applications to respond to events as they occur. It combines real-time analytics, inferencing and monitoring to achieve things like optimizing transportation routes, understanding traffic patterns, anomaly detection in cyber security, making real-time predictions powered by machine learning, and even location-based advertising.

In terms of architecture, we’ve covered how microservices and containers are also driving the multi-cloud trend as microservices often span multiple clouds. You can find this write-up here on Forbes and also here on Medium, where we discussed a background on Google Cloud and how the company was the first to automate orchestration across containers. This write-up provides a great overview of where the major cloud IaaS providers are today and where they might go next strategically speaking.

Big Data and Analytics will Explode because of AI/ML Applications

There is an oft-quoted statistic that 90% of the world’s data was created in the last two years – and this stat is from 2018. The world produces 44 zetabytes of data across the digital universe as of 2020 and there is expected to be 200+ zetabytes of data in cloud storage by 2025. Each zettabyte has 21 zeroes or is 1,000 bytes to the 7th power. By these estimates, we can expect to see up to 5X growth specifically in data centers. Statista places the number at 181 zetabytes by 2025 up from 64.2 zettabytes in 2020.

In regards to data integration in the cloud, this spans from data lakes, to ETL pipelines, cloud data warehouses and object storage. Data fabrics and data virtualization is key to both hybrid and multi-cloud strategies.

Here's how Datadog’s CEO describes what is going on in terms of big data in the most recent earnings call: “it's almost a given that there will need to be a different way of charging for capturing some of the value provided to customers that can't just be attached to the straight volumes of data that are being exchanged because those volume of data are exploding exponentially while our customers' revenues are not going to explode exponentially.”

Generating the data is not the issue (clearly), and distributed data storage has been largely solved with Hadoop. I think it’s worth going through what Hadoop is and how it came about, and then we can look at how Apache Spark helped accelerate data processing, including for Machine Learning. Notably, most open-source projects are not “easy” and this is why companies do well that simplify how to work with Apache Spark and other frameworks, like Kafka.

Background on Hadoop and Data Storage:

Hadoop became instrumental in helping companies store large amounts of structured data, semi-structured and unstructured data through distributed storage and compute. The result was that data storage became cheap enough to retain any/all data that was generated rather than only the essential data due to its distributed file system. The distributed file system was designed to store and process billions of search engine pages across thousands of nodes. The project was created in 2006 by a team of engineers at Yahoo, who had worked previously on a search engine in the early 2000s with the goal of indexing 1 billion pages.

You can think of search engines as some of the first projects that needed to utilize Big Data. The original search engine project “Nutch was limited to 20-to-40 node clusters, and for this amount of data, more clusters were needed. At Yahoo, the team separated the distributed computing parts from Nutch and renamed the project Hadoop, which successfully worked on thousands of nodes. Parallelism was key for the data processing model as Yahoo’s algorithm would need to be run on multiple nodes at the same time and it had to scale linearly. It was then released in 2008 as an open-source project with up to 4000 nodes with distributed capacity with contributors such as Facebook and LinkedIN.

Distributed systems and parallel computing didn’t begin with Yahoo, of course, it began with Google. The paper “MapReduce: Simplified Data Processing on Large Clusters” is considered a defining moment in how programming models handled large data sets. MapReduce was a key moment because it was specifically designed to handle Big Data in terabytes and petabytes due to its framework for parallel computation using a key-value pair.

By 2012, Hadoop’s clusters were up to 42,000 nodes and the number of contributors had reached nearly 1500. Apache Hive is a ETL and data warehouse tool that uses SQL, but Hadoop can manage and process large volumes of data that are structured, unstructured or semi-structured data depending on the database that is chosen. Therefore, you can use many tools with Hadoop, such as Spark.

Background on Apache Spark and Data Processing for Machine Learning:

In 2014, Apache Spark was released which took over the MapReduce model primarily because of its speed. By working with data in-memory, the parallel processing framework can push queries 100X faster and on-disk queries run 10X faster. After the extract, transform and load the data (ETL) process, with Spark you can run a training algorithm on the same in-memory data. This helps Spark reach peak performance over competitors for ETL and relational queries, but also for machine learning. Spark’s goal was to become (and now remain) the general platform for distributed programmers where many specialized systems have one interface and one system to install and manage. Apache Spark also reduces code volume by using APIs for Scala, Java and Python. The framework offers a unified API for fault-tolerant stream processing, which reduces the number of APIs to learn. Spark ML and SparkML are the two APIs that are offered for machine learning pipelines.

Hadoop helped solve some of the data storage issues and reduced the cost for expensive storage and compute. Therefore, the next issue is who can work with these databases and can this be simplified. Apache Spark simplifies who can work with the framework by supporting libraries, which can be executed to interact with data shared across many libraries. The data processing engine is extremely fast because it processes and keeps the data in-memory without reading or writing to disk. This has resulted in Apache Spark becoming popular for machine learning and AI applications with the support of Apache’s very large community of contributors.

Overview of Public Companies in the Big Data and Analytics Space

Databricks and Snowflake:

I’m starting with Databricks and Snowflake simply because we discussed Apache Spark in this analysis. The founders of Apache Spark are from Berkeley and later went onto become the founders of Databricks. We covered this company in-depth on our Snowflake analysis because we feel this is Snowflake’s strongest competitor (i.e., not traditional SQL warehouses or Big Tech). Databricks is not public right now but plans to go public soon.

Here is a summary of the explanation we published in April as to how these two companies compare:

Snowflake's main value proposition is to reduce the time required to prep and monitor data so that a customer does not need to manage software or hardware. Even if a team has the technical skills, they may not want to spend the time required for Databricks, which is perhaps one reason why Snowflake is reporting decent growth in the Fortune 500 and other key accounts.

The architecture of a data lakehouse allows for business intelligence and machine learning through a more open paradigm. The idea is to combine the best of data warehouses and data lakes to span unstructured and semi-structured data while keeping costs low. By combining both, teams can move faster and without duplicating the data. This is a key benefit to Databricks DeltaLake, and this is especially important for data analytics and machine learning. With that said, Databricks is more advanced and expert-level.

I want to point out that Snowflake is very clear as to why it's done well – which is that it handles migrations to the public cloud from legacy on-premise systems better than the competitors. Snowflake's priority is to compete with other SQL databases right now, although the company will need to eventually compete with Databricks. Management has discussed rolling out support for unstructured data, for instance, but no timeline has been set.

Looking longer-term, what Snowflake needs to answer is how will it compete with Databricks on machine learning? Databricks is superior here for ML as it’s built on top of Apache Spark and supports Spark, Python, Scala and also SQL. This was discussed in the thread on the forum here.

The forum thread points out that Databricks is more complex to upload the data, monitor and manage, but there are benefits to going through this hassle. One of the primary benefits is support for Python and Scala, which are programming languages for machine learning. For now, you must use an outside vendor or tool as connectors or integrations in order to support these programming languages and libraries with Snowflake. It’s also worth mentioning that Databricks is cheaper for processing a lot of data at petabyte scale.

Growth is the great equalizer when comparing products and my preliminary understanding is that Snowflake is growing much faster than Databricks and expects to continue to outpace the competitor. I will need to look into Databrick’s financials and see an earnings report or two to determine more about the competitor’s sustained growth rate.

What I find to be very intriguing is what Snowflake will do to compete on ML. This gap in product capability is not lost on the Snowflake team and management. Being laser-focused on the public cloud/cloud-native lends itself well for Snowflake to compete here theoretically, yet its laser-focus on SQL is getting in the way strategically speaking. The company is aware of this and plans to roll out support for unstructured data.

We have two strong products here yet the valuation on Snowflake is stretched and I imagine Databricks will be, too. It’s rare to see a company sustain higher than a 40 or a 50 forward P/S for an extended period of time. Right now, Snowflake is at a 79 forward P/S.

MongoDB:

Big Data applications require a flexible data model, which NoSQL supports. MongoDB is a database that can handle unstructured and semi-structured data, whereas SQL competitors require data to be structured and stored in tables. The predefined schema of the relational database is correlated due to common characteristics. SQL is well-supported as the original database management type yet NoSQL is also reaching critical mass.

The reason NoSQL has risen in popularity is because as data grows, there are more data types to work with outside of Excel spreadsheets/CSV or tabular structures. MongoDB and its competitors are a good choice for Big Data because NoSQL databases can process unpredictable and unstructured data. The most popular types of NoSQL databases include graph, key-value pairs, columnar and document.

Moving forward, we think NoSQL is going to take more market share, simply because it saves steps when dealing with Big Data as the unstructured data does not need to be converted and this is preferred for some machine learning models. This is why NoSQL is used by companies that generate the most data, like Amazon, Facebook, LinkedIN and Google. The extra bonus is that the JSON documents in NoSQL databases can be prepared for machine learning. Because you do not have to define a schema, this allows data to be directly loaded from any new source without changing lines of code. SQL is used in training machine learning models with most of this data coming from on-premise servers. Therefore, the migration to the cloud and various types of data that are generated is also helpful for companies like MongoDB in growing market share. This is because the cloud produces various forms of data.

MongoDB has a query language and secondary indexes for specific values to filter, sort and aggregate data. The leading NoSQL database also allows for the storage and retrieval of trained models as JSON documents. In this case, you can query MongoDB to pull up a previous model.

In the multi-cloud trend, MongoDB is a leader here as the company was the first cloud database to run applications simultaneously on all major cloud providers. The multi-cloud clusters allow developers to deploy applications across multiple cloud providers without having to manage the complexity. In addition, the technical team at MongoDB maintains that you can forego Hadoop and Spark, which requires complex functions and logic, and instead rely on Tensorflow.js, MongoDB and a browser for the same level of machine learning but with less complexity. In an example, a MongoDB representative was able to write a ML program with 88 lines of code. With that said, NoSQL requires more expertise than the universal language of SQL.

The takeaway is that Big Data companies prefer NoSQL for many reasons, and we think in the era of ML and AI, that more companies will lean towards having similar requirements as Big Data companies. This isn’t to say that SQL isn’t alive and well due to the sheer amount of support for structured data across various database systems. Financial transactions for instance fit well into SQL. This is not a “SQL will die” discussion, instead it’s a “NoSQL may see a bigger market thanks to big data and the sheer amounts of unstructured and semi-structured data that will continue to grow” discussion.

Although the SQL and NoSQL debate has lingered for some time with SQL being the leading database today, requirements may change and we think MDB is positioned well for this shift.

Also, refer to the fact that MongoDB is fifth in terms of database market share yet is tied for first place for most wanted database skills among software developers. Notably, MySQL and Oracle are the top database systems globally yet MySQL is fifth in terms of most wanted database skills. The demand for talent is typically an important indicator of where we are now and where the puck is going.

You can read more about MongoDB here in our deep dive research report including more details on Atlas.

Confluent:

The founding team of Apache Kafka worked at LInkedIN before leaving to start Confluent. Apache Kafka is used by thousands of companies for message streaming, such as LinkedIN, where a publish/subscribe model allows applications to share and create data in a serverless and microservices architecture. What Kafka solved for is the ingestion of events data in real-time and with low latency.

At the time that Kafka was developed, LinkedIN was ingesting 1 billion events a day. The company is now ingesting 1 trillion per day. Kafka does this through a log that writes messages to a topic and is able to retain messages for a long time. Kafka is also used in stream processing by parallelizing the pipelines. Kafka Streams were built to increase simplicity while retaining the same amount of performance as a Spark streaming job.

As with Spark and other open-source projects, there is a marketplace for making the frameworks easier to use. Confluent Kafka opens up the amount of data that can be integrated, for example, to combine transactional data (orders, inventory) with sentiment-driven data (likes, page clicks). This helps with predictive analytics and also machine learning because the “data flow” allows for algorithms to work as they are intended to. This is what is meant by the title slide of the S-1 filing “Set Data in Motion.” In order for data to be in motion, Confluent’s platform connects data from many different sources.

The end result for Confluent is that the company allows large amounts of data to be moved very quickly. This is needed for machine learning algorithms that are very data hungry. Kafka can be paired with Apache Spark and Apache Samza to route data and then load it into ElasticSearch, for instance, so it’s a bridge (or a nervous system according to Confluent’s marketing department).

The goal of Confluent is to reduce operational complexity. In the case of Kafka Streams, this is done by not requiring a cluster to be spun up, offering a single framework for streams of events, and reducing the number of pieces in a stream architecture. Confluent Cloud is growing rapidly at 200% year-over-year, primarily driven by event streaming.

Please note, that Confluent is on a partial lockup schedule. The partial lockup dates are 15% on the day of the IPO (June 24th), 25% on the second day of trading (August 09th) after the Q2 earnings, with the remaining at the earlier of the second day of trading after Q3 earnings and 181 days of the IPO.My note: Already up to 40% of the shares have already been released by the eligible employees. The full lock-up expiry is between November and December..

Elastic:

Elastic is a best-of-breed search company that has other benefits, as well. Elasticsearch is the core product that allows for the searching, storing and analyzing of data. This allows developers to build search features that pair Uber passengers with drivers, recommend grocery items on Instacart based on your history, match online data profiles for Tinder, or log events for Fitbit at a rate of 250,000 logs per second. In addition to searching and storing data, Logstash and Beats are ingestion tools to ingest data from applications and to query external systems. Kibana is an open-source tool for visualizing the data. We’ve covered Elastic Stack in more detail here.

Since 2018, the Elastic License has been free and open source with paid proprietary features. As Bradley detailed in this write-up, Amazon began to profit from Elastic’s open-source software and did not contribute back. According to Elastic, over 90% of new downloads choose Elastic’s License. As of January 2021, the company dual-licensed Elasticsearch and Kibana under SSPL or “Server Side Public License,” which requires Amazon or any others to publish modifications and the entirety of their source code. We think the multi-cloud trend is one reason that Elastic has been able to overcome Amazon as the primary driver is to avoid vendor lock-in. Notably, Elastic is cloud neutral so it does not rely on any specific external services for machine learning like AWS’s OpenSearch. Basically, this goes back to the points we made about multi-cloud earlier in this analysis.

We also discussed Elastic’s move into XDR is important because security is a primary concern for those who are on multi-cloud deployments. The SIEM and XDR space is not without its competitors yet it could be Elastic’s combination of already having ingestion tools for thousands of applications and sensors that lends itself well to monitoring and detection. SIEM is security, information and event management while XDR stands for extended, detection and response (XDR). SIEM was first used as a compliance product and often works alongside endpoint and network security products in order to offer a narrower yet deeper set of activity. This last piece has become critical over time. For Elastic’s product, XDR builds on the SIEM and EDR (endpoints) combination for more accuracy and applies machine learning models to detect anomalies.

Where there is data, there will be new opportunities for growth as the AI/ML landscape goes from nascent to mature (i.e. not all uses cases have arrived for big data and analytics companies). Due to Elastic being essentially a pretrained model for extracting keywords and synonyms and “term co-occurrences”, it lends itself well to natural language processing (NLP). With Elastic, terms can be filtered by significance and offer out-of-the box shortcuts to Python with its REST API. Cognitive search is a new form of search that uses AI to improve search queries and to extract information from multiple data sets. Cognitive search can combine a traditional search engine with NLP to extract more useful information since keyword search is limited in the variety of data that can be searched. Cognitive search uses machine learning algorithms for its greatly improved search results and will be a $6 billion market by 2025. We think it's impressive that Elastic was named a Leader in the Gartner Magic Quadrant for cognitive search in the first year it was added as a new entrant, blowing past Microsoft, AWS and even Google.

Conclusion:

I wanted to cover Big Data and Analytics broadly and horizontally rather than vertically by company because it paints a better picture of what we are positioning for and why. It’s easy to get lost in the jargon when discussing companies individually especially with technical companies like these. But what really separates each of them? We think the side-by-side comparison can be more conducive at times when setting up a microtrend.

We had a few goals with this analysis that I hope we accomplished:

Bring to your attention this trend (and the common thread) and pull-out names from the general “cloud” list to discuss why they may have a unique catalyst. There will be many winners in this space and we are limited in terms of number of positions we can enter. It’s easy to get caught up in “stock picks” yet we also want to offer you microtrends to help inform your individual portfolio decisions.
We think big data and analytics from best-of-breed companies could become a solid post-covid cloud play due to the sheer number of companies that migrated to the cloud yet have multi-cloud and hybrid deployments
Third, I want to make sure and elaborate on where the MongoDB, Confluent and Elastic positions are coming from that the I/O Fund recently entered. We offer deep dives on companies but we also want to anchor our readers with the underlying microtrends that we are investing in. For instance, Snowflake is a great choice, yet the valuation is high and that range above 50 has not treated us well in the past (i.e., personal choice). Perhaps for your investment profile, you prefer Snowflake right now, etc.

This is a big space and it’d be impossible for me to cover everything but we pulled out the critical pieces. We think it’s important to simplify the key drivers of a microtrend and illustrate the ways that specific companies are serving the trend. You can expect to see MongoDB and perhaps Confluent added to the LTBH portfolio as the thesis should take about 3-5 years to fully play out. The main thing to know is this means we will have to remove a name or two from the current LTBH portfolio. We will keep you in the loop as we weigh these decisions.

Snowflake Premium Analysis

Posted on April 5, 2021June 30, 2026 by io-fund

9c7f7950-f8cd-4f16-ac46-5f9dbeeb1aa1_Snowflake+Premium+Analysis+v1.pdf

Snowflake Premium Analysis:

You can find my previous analysis of Snowflake here. previous analysis of Snowflake here.

After being nearly 50% off from all-time highs, Snowflake still has a neck-breaking valuation. In this premium analysis, we spend time digging into the financials and valuation as this is a critical piece for this particular company. We also discuss product, especially as it relates to the upcoming Databricks public offering, and a few reasons why Snowflake outpaces this important competitor in terms of revenue growth. We also discuss areas where Databricks is stronger and what Snowflake must do to maintain its lead in the future.

Best-in-Show Key Metrics:

Snowflake is a leading cloud software company in terms of revenue growth, net retention rate and remaining performance obligations (RPOs). Like all cloud software, Snowflake is declining year-over-year as the base increases.

Snowflake went public with 121% year-over-year growth, which has settled to 117% in the most recent quarter. The outlook for Snowflake is growth of 82% at the midpoint with adjusted operating margins of (19%) up from adjusted operating margins of (38%) last year.

Snowflake's two areas of strength include a net retention rate that improved sequentially from 158% to 168% in the most recent quarter. The reacceleration is important as it shows customer satisfaction and is a number that tends to decelerate.

Remaining performance obligations (RPO) is also very high at 213% year-over-year, with $1.3 billion in the pipeline. This represents the revenue that is contracted but not yet realized. The number is higher than forward revenue guidance because the future period may require more than a year. Regardless, the point is it provides a glimpse as to Snowflake’s strength in the future and 200%+ growth shows customer satisfaction. The topic of Snowflake’s future strength, particularly 3-5 years out, is discussed in the valuation section below.

The company grew Fortune 500 customers by 46% from 127 to 186. These customers tend to remain loyal due to their size. I also pointed out Snowflake's impressive growth in customers with product revenue greater than $1 million in my previous analysis.

At the time of IPO the accounts >$1M had grown 56% compared to 22% in the year-ago period. The most recent growth shows an acceleration at 88%. The company is also a usage-based model rather than subscription. The more the world relies on data, the more revenue Snowflake will make. These larger accounts also matter more in a usage-based model.

It’s important to note that Snowflake is bleeding on the bottom line on a GAAP basis with sales and marketing costs equal to its revenue. Sales and marketing costs were at 111% of revenue in FY2020 and at 81% of revenue FY2021. These costs are high due to growing the sales and marketing departments. The company plans to hire 1,200 new employees this year with an emphasis on these two departments.

The operating expenses are nearly 200% of total revenue after R&D and G&A. The translation is that Snowflake is in the middle of a land grab. I imagine the aggressive sales and marketing budget strategy is that the high switching costs of data warehousing solutions will support the LTV needed. In other words, the strategy of establishing customer relationships at any cost should pay off in the long run.

Frank Slootman, CEO of Snowflake and prior CEO of ServiceNow and Data Domain, is the main reason investors are not concerned about the bottom line. The CEO improved the cash efficiency of Data Domain, a company he took from near bankruptcy to a significant acquisition after "selling more than all its competitors combined.” He also grew Service Now from $75 million to $1.5 billion.

The free cash flow margin is set to improve this upcoming year again to "break-even" with operating losses of adjusted (19%) compared to (38%) in FY2021. The operating losses saw a 5X improvement from two years ago when the adjusted operating margins were (105%) in FY2020.

Ultimately, Snowflake has best-in-show key metrics. There is a lot of supporting evidence that Snowflake is well-loved by its customers and is positioned well for future/sustained growth. Databricks has an upcoming IPO and this is a primary risk to Snowflake as there will be a competitor on the public markets once this happens. I expect Databricks to split investor interest – more on this below.

Product Overview:

Snowflake’s decoupled architecture allows for compute and storage to scale separately with the storage provided from any cloud provider the customer chooses. By processing queries using massively parallel processing (MPP), where each node in the cluster stores a portion of the data set locally, the virtual warehouses can access the storage layer independently so as not to compete for compute power. With the competitors, such as Redshift, where compute and storage are coupled, more time is spent reconfiguring the cluster.

Snowflake calls this offering a virtual data warehouse where workloads share the same data but can run independently. This is crucial because Snowflake’s competitors combine compute and storage and require customers to size and pay based on the largest workload.

Data warehouses are centralized data repositories that collect and store information across many sources that are both internal and external. The raw data is ingested into the data warehouse and processed to answer queries. To ingest data, warehouses follow the ETL process, which is: (1) Extract the data from the internal or external database or file, (2) Transform by cleaning and preparing the data to fit the schema and constraints of the data warehouse and (3) Load into the data warehouse. The ETL method helps to organize the data into a relational format. Notably, Snowflake supports both ETL and ELT, which allows for data transformation during or after loading.

One key product differentiator is that Snowflake is not built on Hadoop, rather the company uses a new SQL database engine with cloud-optimized architecture. Overall, this translates to faster queries and also reduces costs by scaling up or down for both capacity and performance. This also allows the shift to the cloud while still honoring traditional relational database tools. Just like cloud infrastructure does not require you to hold server space for peak times year-round, a cloud data warehouse does not require you to plan, acquire or manage resources for peak data demand (i.e. elasticity).

The need for resources could change by either increasing or decreasing (scaling up or down). Customers that have a need for storage but less of a need for CPU computations do not have to pay up front and can shrink the environment dynamically. Users either pay for terabytes or are billed on a per-second basis for computations. Notably, Snowflake charges by execution-based usage and is not a cloud SaaS-company that charges by subscription.

Snowflake has a multi-cluster architecture which is unique from single cluster databases. The multi-cluster approach allows the clusters to access the same underlying data yet to run independently. This allows for heavy queries and operations to run very quickly and with fewer errors because the queries are not accessing the same data warehouse.

Queries are made with standard SQL, for analytics, and integrates with R and Python programming languages. The company delivers the ability to handle all incongruent data types in a single data warehouse. Because the data is accessible through SQL, there is widespread developer uptake as it’s the most common database language.

Snowflake supports both structured data and semi-structured data. As machine-generated data grows to include applications, sensors and mobile devices, Snowflake allows semi-structure data to be handled without preparation or schema definitions. The result is handling JSON, Avro, ORC, Parquet or XML data as if it were relational and structured.

Snowflake uses a compressed columnar database. Columnar databases are optimized for the fast retrieval of columns of data and is used for analytic data queries. Other features include centralized metadata management that is stored in a single-key value store that allows cloning of tables and databases. Security is baked into the platform to where Snowflake automatically encrypts all data to the point where unencrypted data is not even allowed. There is third-party certification and validation for security standards like HIPAA.

Beyond the value proposition of separating storage from compute for speed, and also scaling up or down to reduce costs, the third takeaway is that Snowflake is also much easier for customers to use as it’s designed to remove the role of a database administrator for monitoring and/or to tune query performance.

The end goal of choosing Snowflake is that you load data, run queries, and do little else – which is an immense value proposition due to the amount of time wasted prepping, balancing, tuning and monitoring traditional data warehouses originally built for on-premise.

Snowflake is capitalizing on the multi-cloud trend and growing rapidly with customers who want a choice in public cloud provider despite the cloud giants having their own data warehouse systems, such as Amazon Redshift, Azure Synapse and Google Big Query.

Generally speaking, Big Query is a closer competitor as Google’s offering also separates storage and compute. The differences between BigQuery and Snowflake include pricing structure where Snowflake is a time-based pricing model where users are charged for execution time and BigQuery is a query-based pricing model, where users are charged for the amount of data returned from the queries. BigQuery has a serverless feature that makes it easier to begin using the data warehouse a the serverless feature removes the need for manual scaling and performance tuning. Dremel is the query engine for BigQuery.

When it comes to deciding between BigQuery and Snowflake, it can come down to what you do with the database due to pricing structure differences. For instance, Snowflake is a better choice for concurrent users and business intelligence. It’s also a great choice for data-as-a-service, where you might give client access to your data in the form of analytics. BigQuery is perhaps a better choice for ad hoc reporting, where you have occasional complex reports on a quarterly basis or recommendation models and machine learning that require high idle time. Again, these examples are mainly due to pricing structure.

Despite BigQuery having a strong following with nearly twice the number of companies as Snowflake and growing around 40%, it tested slower than Snowflake in field tests performed by GigaOm in 2019. Vendor lock-in from BigQuery is also undesirable as companies may prefer AWS or Azure and/or more interoperability or best-in-breed solutions – we can see this in the growing trend of multi-cloud. AWS Redshift has the biggest market presence but growth is nearly flat at 6.5% and AWS is the leading partner for Snowflake.

Here's a great write-up from the Hashmap Engineering and Technology Blog that points out why implementing optimized row columnar (ORC) format data loads is ideal for either Snowflake or Amazon Redshift due to the ORC file format. Again, ultimately the choice in which system you use comes down to the individual needs for implementation although Snowflake is designed to be a competitor in nearly every case.

There’s a great write-up from analyst David Vellante that discusses how Snowflake competes with cloud native database giants. His analysis discusses survey responses from CIOs and IT buyers with Snowflake having a lead over the tech giants in spending intentions. The Enterprise Technology Research study he highlights showed 80% of AWS accounts plan to spend more on Snowflake in 2020 relative to 2019 with 35% adding Snowflake as new compared to 12% adding Redshift as new. In Azure, 78% plan to spend more on Snowflake with 41% adding new. On Google Cloud, 80% plan to increase spending on Snowflake. We can see the people have spoken.

Pricing pressure is the weakness we must monitor with Snowflake as it’s sandwiched heavily between tech giants in an area where these tech giants are very protective of their turf. How cheap will BigQuery become to attract developers to Google Cloud, which Google can then monetize the life span a customer in various ways?

You'll also see below that the tech giants are also private investors in Databricks.

Valuation Update:

Snowflake is the fastest growing cloud stock with 124% revenue growth over the last 12 months and a projected 85% over the next 12 months. The recent sell-off in tech stocks has led to a significant contraction in Snowflake's valuation. Snowflake trades at 56.8x EV/NTM revenue after seeing its valuation peak above 80x forward revenue following its IPO.

Snowflake guided for 82% revenue growth in 2021 at the midpoint of its target. As covered in the financials, the company is also targeting breakeven in Free Cash Flow for the full year in 2021 after a -12% FCF margin in 2020. Snowflake sees Operating Margin improving to -19% in 2021 from -38% in 2020.

SaaS stocks are typically valued based on a multiple of forward revenue. Below, we take a look at the ten highest valuations in the space:

As we can see, Snowflake has the highest projected growth rate over the next 12 months by a wide margin. Only one other SaaS stock in the table is projected to grow over 40% YoY in 2021 (CRWD). Conversely, Snowflake is projected to grow 85% YoY in 2021, 48 percentage points above the median.

Consensus expectations show that Snowflake is projected to grow revenue 65% YoY in 2022 and 57% YoY in 2023. In fact, Credit Suisse’s DCF Model on SNOW calls for its growth rate to exceed 40% YoY in 2024 and 2025, before retreating to 37% in the year ending 2026.

This growth is unparalleled by any other public SaaS stock, as the median NTM growth rate for the 10 highest valuations is currently 37%. This means that analysts are expecting SNOW to grow revenue in 2026 at the same rate that Cloudflare, Zscaler, and ZoomInfo will this year. It becomes even more extraordinary when you consider that Snowflake is projected to achieve this growth rate after posting $3.5B in revenue in 2025.

This revenue number comfortably exceeds the combined annual revenues of Cloudflare, Zscaler, and ZoomInfo. To put Snowflake's growth into perspective, analysts expect a higher growth rate from Snowflake in 2025 than any SaaS stock this year outside of CrowdStrike.

This is why we believe it is essential to look beyond the NTM revenue multiples, as Snowflake will separate itself by continuing to grow faster than any of its peers over the next 5 years. With this level of growth, SNOW will be able to compound its revenues to bigger and bigger totals.

The farther out we look into the future, the more reasonable SNOW's valuation becomes compared to its peers.

Major Competitor: Databricks

In my most recent Motley Fool podcast, I pointed out that Databricks planned to go public, which was a risk to Snowflake's valuation.

We don't have an S-1 filing at this time, but TechCrunch has done some preliminary work based on a $28 billion private valuation. The tech media site estimates that the company grew from a $200 million annual run rate to a $350 million annual run rate between Q3 2019 and Q3 2020.

TechCrunch's current number is a run rate of $425 million to $485.6 million for Q1 2021. CNBC reported, "$425 million in annualized sales," representing 70% growth for last year. This is quite a bit lower than Snowflake's growth last year, which was 123% for full-year growth.

We believe Databricks could open up at a higher valuation than where Snowflake is currently trading. This is because the forward growth is lower, which we estimate to be 60% for FY2021. We are assuming a $50 billion market cap at the opening. The $50 billion represents about 2X the private valuation while Snowflake opened around 3X its last private valuation.

Early backers for Databricks include Microsoft, which was later joined by Amazon Web Services, Google via CapitalG, and Salesforce. Venture firm A16Z and Tiger Global are also among the investors. This strong lineup participated in the latest $1.9 billion round. The articles that reported on the recent round point out the rarity in getting backing from all four major cloud vendors.

What’s the difference between Databricks and Snowflake?

Yesterday on our forum, there was an excellent post by a subscriber around the differences between Snowflake and Databricks.

The major difference between Snowflake and Databricks from a customer standpoint is that Snowflake is laser-focused on the public cloud/cloud native while Databricks is differentiated in that it can build information pipelines across silos, including on-premise and hybrid architectures. Snowflake's main value proposition is to reduce the time required to prep and monitor data so that a customer does not need to manage software or hardware. Even if a team has the technical skills, they may not want to spend the time required for Databricks, which is perhaps one reason why Snowflake is reporting decent growth in the Fortune 500 and other key accounts.

The architecture of a data lakehouse allows for business intelligence and machine learning through a more open paradigm. The idea is to combine best of data warehouses and data lakes to span unstructured and semi-structured data while keeping costs low. By combining both, teams can move faster and without duplicating the data. This is a key benefit to Databricks DeltaLake, and this is especially important for data analytics and machine learning. With that said, Databricks is more advanced and expert-level.

Conclusion:

Snowflake is disrupting the data warehousing market (or cloud data market) with a superior cloud data platform that delivers across key differentiators. Snowflake demonstrates excellent product-market fit, clear competitive advantages, and strong management — the primary ingredients for a great growth stock.

I prefer to not pay over 40 forward P/S and certainly not over 50 forward P/S yet Snowflake's forward growth requires additional consideration as it promises to outpace its peers. In this case, we initiated in the 55x range and will build more into the position either on a pullback or a breakout.