Data Leaders Feel the Pressure to Prioritize Generative AI over Higher Returning Investments, Survey Says 

For the second straight year, we surveyed hundreds of data leaders in Las Vegas at Snowflake Summit 2023 to get their takes on their data cloud journeys. But what happens when the two biggest cloud data summits happen at the same time? 

You do TWO surveys! Check out what data leaders at both Snowflake Summit and Databricks Data + AI Summit 2023 had to say when it came to things like data adoption in the cloud, data trust, and of course, the now omnipresent Generative AI.

Leading up to this year’s Snowflake and Databricks Data + AI Summits, all the talk was about the – perhaps not unintentional – timing of the two massive shows. How could one week possibly fit within it all the happenings of these two cloud data giants? Well, both conferences certainly did not disappoint. 

At the conferences, the expo halls and breakout rooms were abuzz with talk of acquisitions, data sharing, generative AI investments, and much much more. Meanwhile, the Monte Carlo team was also buzzing through the halls capturing data for the 2023 State of Data Management survey, a poll of over 350 senior data practitioners, to get their take on some pressing data questions. 

Here are a few of our findings: 

Data leaders are feeling the pressure from Generative AI

Nearly 50% of respondents are already feeling significant pressure from their CEOs to invest in Generative AI at the expense of potentially higher-returning investments. It has become almost the running joke of these summits that Generative AI is the most ubiquitous word in tech, to the point where Snowflake CEO Frank Slootman suggested that we play a take-a-shot drinking game every time the word AI was brought up in his company’s keynote (that would have been disastrous, trust us). 

As the survey suggests and deeper conversations with many of the survey takers solidifies, generative AI has become a must-check box for data teams despite the strategic ambiguities around it. As Monte Carlo co-founder and CTO Lior Gavish says, not only is everyone just scratching the surface when it comes to GenAI, we’re all scratching the same part of the surface, to the same exact depth. 

Adoption of cloud data warehouses and lakes is rising rapidly

In last year’s version of the survey (which, again, only polled Snowflake Summit attendees), nearly half of respondents reported having 25% or less of their data in the cloud. However, that same survey revealed 73% intended to invest in a data warehouse or lake that same year. So it should come as no shock that those reporting at least half of their data resides in Snowflake, Databricks, or both, more than doubled from last year’s version of this survey. 

This signals the continued growth of data cloud storage and processing, highlighting that while some data remains on prem, transactional databases, or in SaaS tools (in other words, a data silo), companies are quickly realizing this vision.

Self-service analytics and data adoption are the top priority for most leaders

40%+ of respondents cited data adoption and self-service models for data analytics as their top priority for their data organization in 2023, followed by cost optimization (23%) and generative AI and machine learning (16%). This is likely music to both data cloud giants’ ears, highlighting the enthusiasm for features like Snowflake Data Sharing and Databricks Delta Sharing and native data applications, announced in years prior and finally bearing fruit. The focus on cost optimization should also come as no surprise given the current economic climate and the increasing pull from CFOs and other execs to do more with less as well as bring the data team’s function closer to their company’s bottomline.

In this tale of two Summits, a few things have become clear. 

1. Grab your popcorn. Both Snowflake and Databricks are leaning in HEAVY to generative AI, containerization, and programmatic features that continue to blur the line between warehouse and lake, as well as analytics and ML use cases. In our opinion though, this continued innovation is only to the customers’ benefit.

2. Generative AI is queen.  Will generative AI live up to its potential of, as Ali Ghodsi, CEO and co-founder of Databricks suggests, “curing cancer?” Will data teams be able to reap the benefits as quickly as they’re spending to invest in it? Or will bots simply eat us? Time will tell, but what we do know is that in the short term, generative AI can lead to productivity gains and greater democratization of tasks like data querying and discovery – at least until Snowflake and Databricks’ exciting new features are out of beta.  

3. We are absolute suckers for a good summit. Kudos to Snowflake and Databricks for putting on unbelievable shows and continuing to deliver great buzz, conversation, and most importantly, exciting new features to make data management more scalable, reliable, and most importantly, accessible. Can’t wait until next year! 

Stay tuned for the full survey results, below: