This week, Investor Sabrina Wu talks with Fivetran Co-founder and CEO George Fraser. Fivetran is fundamentally a data replication company. That’s how George explains it. But the company actually started out trying to solve a completely different problem when it was founded in 2012 and pivoted to strictly data integration in 2015 after multiple customers started asking for it. But in attacking that problem differently than it had ever been done before — by only focusing on replicating people’s data into the desired destination without getting sucked into any of the workflows that the user intended to do on the other side — they’ve come out as a leader in the space. Fivetran landed a $565 million series D last year and made two acquisitions, and it was named one of our top 40 Intelligent Applications. In this IA40 spotlight episode, Sabrina and George not only dive into the story behind Fivetran and how it has taken time and patience to get to where they are now, but Sabrina also gets some hot takes on the modern data stack, reverse ETL, query federation, and why Fivetran doesn’t open source. George also has some great advice about coming up with a company name, but you’ll have to listen to get it.

This transcript was automatically generated and edited for clarity.

Sabrina: Hi everybody, I’m Sabrina Wu, and I’m an investor at Madrona. I’m excited to be here today with Fivetran CEO and Co-founder of George Fraser. It should be no surprise that Tran was selected as a top 40 Intelligent Application in 2021. Intelligent applications leverage artificial intelligence and machine learning to continuously become better at solving business problems. However, the data ingestion, preparation, and management challenges that enable intelligent applications are extremely difficult to solve, which is why intelligent applications require enabling layers, such as Fivetran. Fivetran has built fully automated data connectors that enable businesses to extract, load, and transform data from different cloud storage and database sources into one central data storage system. I’m excited to dive into more of this with George today.

George, thanks so much for being here with us.

George: Very nice to be with you. That was a great overview. And applications, of course, our original bread and butter — things like Salesforce, NetSuite, stuff like that.

Sabrina: That’s a great transition. Let’s go back to the beginning —how did Fivetran go from being a data analysis tool to becoming the leading provider of automated data integration? Was there some light-bulb moment for you guys that made you want to pivot the business?

George: Well, it took about two years. Basically, we figured out one thing in those two years, which is that we would find product-market fit by cutting out everything except the data integration. So yes, we built a vertically integrated tool that had connectors as part of it, that stored data behind the scenes and Redshift, and that provided you a user interface to interact with that data. And, it was, it was not great. It was a vertically integrated BI tool built by three guys, and we worked hard at it, but it had a long way to go. What happened was we started to get this request from people we were talking to, which was, “Hey, I’ve got my own Redshift cluster that I’ve just set up,” and this was 2014, right? So, Redshift was this phenomenon in data warehousing at the time. It was the first data warehouse that was fast and cheap. It was available in the AWS console. And at the time, it was the fastest-growing product in AWS’s history. So, we were talking to all these people, and more and more started to say, “I’ve got my own Redshift cluster that I’ve just set up. It’s empty, and the problem I have is just getting data into it. And I like the sound of these connectors that you’ve built as part of this system. Can I just get those into my own Redshift cluster? And then I have my own plan of what I wanna do with the data.” And eventually, we said yes to one of them, and then another, and then another, and that became the company.

Sabrina: And was it obvious at the time that these automated connectors would be so valuable to customers? And obviously, the database world has exploded since then. I’m curious if the problem has been much bigger than you had initially anticipated?

George: It has indeed. It’s turned out to be more valuable and more widely popular than I think anyone anticipated. It’s funny because this is not a new problem. Data integration — taking data from a bunch of systems of record, whether they be databases or applications, and putting it into a central data warehouse — has been around since the 1980s, so it’s not a new problem. There have been tools for doing this for years and years. But the tools were not very good at the time — in my humble opinion. They didn’t really solve the problem. They were like a pile of building blocks upon which you would build your own integrations, and you were responsible for the correctness and reliability of those connections. If it broke, that was your fault. You had to figure out why. So, most people were just writing code to do data integration. I had actually done a lot of that at my previous job in a very different context in biotech. But this is a problem that had been around for a long time. It was everywhere. You talk to 10 software engineers, five of them have written some code to do data integration inside their company at some point. There had been companies founded trying to solve this problem. None of them had ever really taken off. They were a fraction of the size of the data warehousing companies, even though logically, both always have to exist in every account. So why was this such small potatoes? Nobody had ever really found a good solution to this problem.

And we completely ignored the way it had been done before. There was a lot of conventional wisdom about the right way to do data integration, and we just totally ignored it. And we said, Okay, what seems like the right way to us? We should just replicate everything. You know, the data starts in the place that it lives. Replicate everything into a normalized schema. The destination is a database, so people can do whatever they want with it from there. As long as you get it all there in a correct replica, you’ve solved a big chunk of the problem. And the nice thing about that is it’s like a bounded problem. You don’t get sucked into all the workflows that the user intends to do on the other side. What is your definition of revenue? You don’t get pulled into that. You’re like, “Look, I’m going to replicate the amount field from the opportunity table in Salesforce, and I’m going to do that correctly — that’s my job. How that maps to your revenue recognition roles, that’s for you to figure out.” So, there was a nice, clear separation of concerns between what our job was and then the problems that remained to be solved by the user. And the part of the problem that we were taking on was a big chunk of the problem. It’s not the whole problem. And that was kind of the conventional wisdom we were walking away from, that you had to tackle those downstream elements. We said, “No, we don’t. We’re going to solve the replication portion of this, which is actually a pretty big chunk of the problem. That’s where we’re going to focus our efforts — correct replication. And the very first connector was for Salesforce, and then we did Zendesk and Stripe, and then we started doing databases and a bunch of other things. But we always took that philosophy like Fivetran is fundamentally a replication tool, and you are going to work out the semantics of that data in the destination after it arrives.

Sabrina: I think one of the reasons customers really love Fivetran is because the product itself is very simple to use, but in reality, you are solving a very complicated and difficult problem — something that is very cumbersome and tedious … an engineer spent a lot of time building this. You started to talk about it, but I’d love to understand Fivetran’s approach to ELT. So, extract, load, transform, which shifts, as you were saying, the transform step to the end of the process. Can you talk a bit about why Fivetran created this new approach and the evolution of ETL — extract, transform, load to ELT — extract, load, transform?

George: It’s interesting. You know, ELT is a little bit of a lie. It’s a lie that reveals an essential truth. But the reality is that Fivetran does a ton of transformation. So, it’s really ETLT. The way the data comes out of the source, people always imagine what the source gives us is like a nice little list of inserts, updates and deletes, and then we just take it over to the destination and load it in. If it worked like that, we’d have lots of good competitors by now, and we don’t. So, the reality is the sources are sort of crazy in what they give you, so we have to develop these extremely convoluted rules for reconstructing the set of what has changed from the source. And it’s different for every source. But, if you make your goal to produce a correct replica, then that problem is the same for every user. So it’s very complicated. There’s a ton of complicated transformation we’re doing under the hood, but it’s always the same, whether it’s your Stripe account or someone else’s Stripe account or whatever the case may be. Stripe is actually not as hard as some of the other ones. That’s not a great example. Stripe gives you the data in a pretty good format, but if it’s your HANA database versus someone else’s HANA database, the rules of stitching together changes are going to be the same. They’re very complicated, but they’re always the same. And so, we do that level of transformation, but then we abdicate this other job, of like, how do you map all this data onto the concepts of your business. There’s another layer of transformation that is different for every customer. And this is the problem we very intentionally chose not to tackle. We said, “You’ve got to solve that after we deliver the data, and there are multiple ways to solve that problem.” But by making this choice, it sort of gives us a problem to solve and the customer a problem to solve. And it’s a very nice, clean handoff. And so you get that very simple user experience. Fivetran is doing very complicated things under the hood, but you don’t need to be involved in that. You don’t need to know how that works because it’s fundamentally the same for every user, so we can just automate it behind the scenes.

Sabrina: That’s a good point. And I think that what you’re talking about here is also Fivetran’s big investments in change data capture, so this whole process of CDC as well as replicating and allowing customers to move really large amounts of volume with very low impact. I know Fivetran recently also acquired HVR to further strengthen this. And so, I’d love to understand why data replication is increasingly important for customers.

George: Well, I don’t know if data replication is increasingly important. It’s just always been very important. Maybe for smaller companies. You know, one thing we have noticed is that historically data warehouses were something that were only really created by the largest companies. And we have an astonishing number of like 50-person and under companies building data warehouses, which historically that was not really a thing. But the problem has become so much easier at every layer of the stack that it’s become feasible for much smaller companies to pull this off. Then they can realize the benefits of knowing what’s going on in your business, which is the primary benefit of a good data warehouse.

Sabrina: When you think about CDC as a whole — why is this problem so difficult to solve?

George: It’s just a maze of incidental complexity. There’s nothing really fundamentally hard about it in the way that building a globally distributed transactional database is just like a very fundamentally hard problem, and people write Ph.D. thesis about it. Nobody writes a Ph.D. thesis about data integration. It’s just a huge forest of incidental complexity, and the way we’ve been able to tackle it is with a lot of grit, a lot of hard work, and a lot of time. If you look at the history of Fivetran, we talked about the original pivot to just data integration in 2015, but then for the rest of 2015 and a lot of 2016, we didn’t grow that fast because it still didn’t work that well. Part of it just took time to figure out all these little rules that create that easy user experience where you just push the button, and your data appears and keeps itself up to date, and the details of how that happened are unknown to you. So, a lot of it has just been time. As the years have gone by, we have studied ourselves, and we have started to learn how to solve some of these problems more systematically and, in fact, automate the process of developing connectors internally at Fivetran. And you will notice if you look at our release notes that the rate of new connectors being released in the last couple of quarters has increased dramatically. Our CTO Meel Velliste and a small group of engineers spent a bunch of time last year really studying basically all of these quirks that we had observed in every connector we had ever built. And they sort of mapped it out and determined that data sources do a lot of crazy things, but there is a finite number of crazy things. You can kind of catalog them all — all the weird stuff that we’ve encountered over the years. And then they developed a new way of building connectors, like a configuration-driven connector where you just have to describe, within this vocabulary, how any particular data source works. And you don’t have to implement all the procedural rules to correctly deal with all of those quirks. And so that’s a pretty cool thing — it took years to figure that out and to do that right. But you’re seeing the benefits of that now with the acceleration of how many new connectors we’re able to build each month.

And we do hope to get to the point in the next couple of years where we have thousands of connectors. Cause that’s what we ultimately need to have. There are so many data sources out there. And it is extremely valuable to customers, especially larger customers, if Fivetran can cover all their data sources — if they can say, “I can adopt Fivetran, and then my data integration problem is solved.” Whatever the source is, Fivetran’s going to have a connector to it.

Sabrina: That’s a really good point. And I’m curious — you also have decided not to open source, to my understanding, any of your connectors.

George: We never seriously considered open sourcing our connectors. The way we looked at it is our users are not software engineers. They’re analysts and data engineers. and most of them aren’t data engineers, by the way. So they don’t really have the ability or the desire to contribute to these connectors. Open source works really well when your users are software engineers who have the ability and the interest in contributing back to things like Postgres, for example, or Linux. For us, that’s mostly not the case. And the other problem is — when people build connectors for themselves, they tend to take shortcuts that work for their particular situation but won’t work for others. They rely on behaviors of how they use the underlying system. Like, oh, you know, when I update rows in this table, I always update this column named “updated at,” so I can use that in order to get changes from that table. Well, that might be true for you. But it’s not true for others. And so that trick is not going to work. And so, for all these reasons, we didn’t really see a community of contributors being a good way to get high-quality connectors. And I think that hypothesis has mostly been born out over the last few years. Fivetran continues to have a big advantage in terms of the quality of our connectors, by which I mean, are they working? And is the data in the destination actually a correct copy of the source? That is really the hardest part.

Sabrina: Got it. In my mind, Fivetran targets a lot of people and adds a lot of value to multiple people in the organization. So it could be the data engineer, the data analyst, the business analyst, right? And so I’m just curious, how are you building the product with these people in mind? You touched on it a little bit there, but I’m just curious if there are certain use cases where it’s better suited for collecting and analyzing data. How do you think about this from a data analysis versus ML workloads?

George: You know, it’s funny, uh, the answer to that is it really doesn’t actually matter to us. This is something that was al always very frustrating to VCs and often to salespeople — this is not really a persona-driven problem or product. We deliver data. The data needs to be correct. It needs to be up to date. And you can do a lot of different things with that data. And our users do a lot of different things with that data, and oftentimes, the proximate user of Fivetran, like the person who we’re talking to at the company, they don’t even know what all people are doing with this data. There’s this great mantra, “Data attracts workloads.” You know, once you have a good central store of all your company’s data, people will just come up with all kinds of stuff to do with it.

Sabrina: That’s funny. I think people are always kind of asking who’s the right persona. Who’s the right user, right? But um, in reality, when you’re thinking about some of the plumbing work that goes in that Fivetran is working on, it really is impactful to the whole organization. Now I’d love to kind of shift gears a little bit and get your thoughts on the modern data stack. In today’s world, customers are really patching together a lot of different best-in-bread tools and solutions to host, store, transform, and eventually analyze and understand the data. That’s the end goal that customers have. And so, as the number of tools has proliferated, there’s been more complexity than ever before. And I’m just curious how does Fivetran continue to differentiate in a very competitive and complicated data stack?

George: Yeah. You know, people make these like market maps, and they put all these logos on them. By the way, if you look closely at those market maps, the same logos will appear multiple times in different categories. So it makes it look a lot more complicated than it is. And the other thing is, you don’t have to use all these tools. I mean, the first few Fivetran customers, most of them, used Redshift, Fivetran, and Looker. Which was at that time a relatively new BI tool, but it had this key strength, which was this modeling layer called LookML, which allowed you to do that post-Fivetran transformation of data that we required you to do because we had abdicated that job. And LookML has largely now been replaced in that role by dbt, which just does that. But the point I’m making is that stack is still available. You can still buy that today. It still works just as well as it ever did. You don’t actually have to adopt all of these other things. I don’t think it really has gotten more complicated. I actually think it’s gotten quite a bit simpler. You know, if you look at people who are running Fivetran, a good cloud data warehouse, like Snowflake, Big Query, Redshift, Databricks, and dbt to manage their transformations, life is pretty good for them. A small number of people can manage a lot of data and a lot of complexity with that set of tools, and it’s a pretty good stack to adopt.

Sabrina: Yeah, I tend to agree with that. Do you think that there’s going to be more consolidation here in the years to come?

George: There’s always consolidation happening. So, the short answer is yes. And how things consolidate. What are the key strategic workflows that they consolidate around is always a big question. I think we’ve seen some of that happen. As you mentioned earlier, Fivetran acquired HVR. And the reason for that was very simple. HVR was really good at replicating the biggest, most difficult enterprise databases at low latency and the highest volumes. And Fivetran was really good at everything else. And so, it was a perfect marriage. And for users, now you have one vendor that can do both. And we’ll probably see other things like that. It’s something we do think about a decent amount at Fivetran — how much are acquisitions going to be a part of our future … probably a medium amount. So yeah, I do think there will be consolidation. And I do think that Fivetran is going to be one of those key poles that consolidation happens around. Because we’re at the top of the funnel, right? We’re getting the data from everywhere. We control the rules of how the schemas work and how the updates happen, and when things happen. And for that reason, there are a lot of other secondary workflows that make sense to pull into that ecosystem.

Sabrina: At what point would you consider acquiring a company versus maybe just developing stronger partnerships with those companies? Integrating more into the workflow?

George: We’ve only done it a couple of times. You know, there was HVR, and there was a much smaller company called Teleport, which we acquired for an algorithm also for replicating databases and very different circumstances. We are mostly a partnership-driven company. Always have been. Our second customer was a partner referral, so most of the time, the answer is partnerships. And you see that right now with our approach to metadata. Fivetran has all this metadata about the data that’s sitting in your data warehouse: where did it come from? When was it last updated? And we are working to expose all that data. The only times when acquisitions have come into the picture is when, you know, it’s something that’s in that core focus that we felt we were not good at and needed to get better at, and it was going take a long time to do it ourselves. But, as I said earlier, we’ve only done that twice. So how that philosophy will evolve in the future, I can’t say.

Sabrina: Well, it might be a good time to be on the offensive and get some acquisitions under your guys’ belt, given a lot of what’s going on in the macro environment today. But I think that your point about partnerships is really, really important. I think Fivetran is one of the companies that has partnerships with almost all of the key players in the modern data stack ecosystem, Snowflake, as you mentioned, dbt, Collibra, Hightouch, and some of the governance players now with the release of Metadata API. And so, I’m just curious, you know, how have you been able to maintain these partnerships? And it seems very core to the Fivetran strategy, and I would just love to touch a little bit more upon this partnership strategy that you guys have built over the years.

George: Yeah, it’s, well, we have a partnerships team that works very hard. Part of it is just intrinsic to what we do. We do something that all the partners need. None of these tools do anything without data in them, and that’s what Fivetran does — it feeds the data. There is an element of like, we’ve become partner driven just because we do something that all these partners really care about and need us to do. We’ve always tried to just be very straightforward in our strategy and in the way we communicate that to partners.

You know, like I said, the core of Fivetran is connectors and always will be— we’re fundamentally a data movement company, and that’s not going to change. So, we’re very predictable in terms of what we’re going to do next. And then in the details of that, we’re very transparent with our partners about like, “Hey, this is what’s happening next quarter. This is what we think is going to happen next year,” and I think that helps.

Sabrina: There have been some of these newer concepts thrown around. As I’m sure you know, reverse ETL emerged a couple of years ago, and so initially, data warehouses were introduced to essentially eliminate the data silos, right? But for many companies, data warehouses have now become a data silo, you could argue. And so, there’s this concept in the modern data stack known as reverse ETL, which aims to remove the barriers between the data warehouse and all the end applications such that teams can operationalize data into the systems themselves. What is your thinking around reverse ETL?

George: Reverse ETL is such a funny name — it’s a good name because you immediately know what it is. But then, after you think about it for 15 seconds, you’re like, wait, this is also ETL. It’s not reversed in any way. It was originally a Fivetran feature request. Our very first customer asked us to do that within weeks of going to production. And they were not the only ones. And the problem was forwards ETL is hard enough. We had to just focus on that. And we talked very openly about it with other players in the ecosystem and with friends, including the founders of Census, actually. I’m not taking credit or anything, but we did tell them about how we kept getting this request before they started Census, and that was probably one of many pieces of evidence that contributed to their decision to attack that problem. So, it’s a need that is out there. It hasn’t gotten as big as we have just yet, and time will tell how big the demand for that actually is. I think that’s the key question, is this idea of using the data warehouse as a data bus that you can then send data onward to other places. It’s definitely a thing people are doing. Is that something everyone is going to do in a couple of years or is that going to be something some people do? Time will tell.

Sabrina: Yeah. Census and Hightouch, right? Those are the two going after it in the reverse ETL world. There are also some newer approaches around query federation that suggest that engineers and developers may no longer need ETL altogether because you can query data regardless of where it lives. So, you’re removing this step of ETL altogether. Do you think ETL will continue in light of the federation trend?

George: What trend? Query federation has been around for decades. And it has been a stupid idea the entire time. Query federation makes for an incredible demo. You have live access to data instantly. You don’t have to wait for historical backfills or anything. And as soon as you go into production, it will fall over because you will discover that the data sources are simply too slow to give you data fast enough to support realistic queries. There are these little optimizations you can do, like predicate push down, that will sometimes speed it up. But as soon as you go to production, you’re going to discover you have a lot of queries that are not subject to those optimizations, and you have to move the data. There is an exception, which is if you’re reading data from object storage like S3, that actually does have enough bandwidth to make query federation work. But other than that, it’s hopeless. You can’t read the data fast enough to support a real data warehouse workload using query federation. It’s like a beautiful dream. And I will say that in some ways, you know what Fivetran did, where we said we’re going to move the data, but we’re going to treat this as a replication problem. In many ways, we are creating the same user experience as query federation. You know, you look at your data warehouse, and you just see exactly the same schema as exists in all the data sources, except the way it is actually implemented is using data movement because it’s impossible to do anything else. So those are my thoughts on query federation.

Sabrina: I love the hot takes. It’s always great to hear, you know, differing perspectives around what’s “in” and what’s not, and I’m curious if there’s any other, you know, approaches that keep you up at night or anything else that has been a concern — something that you are just thinking through as Fivetran. Like, “Hey, there’s all these different approaches now to moving the data to being able to analyze that data.”

George: Yeah, I think that there are two limitations of, let’s call it, the Fivetran-centered data stack. And they’re really limitations of the destinations of the data warehouses we deliver to, and opportunities for those data warehouses to fix these limitations. And they’re working on it. One is latency. It’s pretty hard to do any better than 15-minute latency the way things currently work. And there is a set of use cases where that’s not good enough, but it is really hard right now to do better. So the world needs a low latency data warehouse, and I think that will happen. I think the great data warehouses that exist today are going to make progress on this. They are making progress on this. And we may see another player emerge where the primary value proposition is they just have much lower latency. Then the other problem is, you know, the usage of the data warehouse as a data bus where you centralize all this data in your data warehouse, but then not everyone wants to run SQL queries. Some people want to just pull the data out and send it on to some other system that does something with it, and it’s just very costly to do that by running SQL queries against a data warehouse. And so, we’re seeing a couple of different ideas emerge about how to solve that problem. So, some people try to adopt something like Kafka as a central data bus to solve this. Now you have a whole different set of problems. And I think that’s unworkable for very complicated reasons that have to do with the way the data comes out of the sources. People don’t realize the sources produce very unclean data. And if you’re not sending the data to a relational database that supports updates and things like that, the data you’re going to be looking at is going to be very ugly.

That’s one vector people are pushing on to try to attack this problem. Another vector is like data lakes. The problem with data lakes is that the latency is even worse. Big Query has done something interesting here. They have this thing called the Big Query Storage API, where you can, basically cheaply, directly read data out of the underlying storage layer of Big Query. So, if you want to do something that isn’t a SQL query with the data, you can use Big Query as a data bus. But no one is using this. That’s kind of a big problem that I’m thinking a lot about right now — how do we attack those twin problems of latency, and how do I support workflows other than SQL queries at reasonable cost and performance none of the solutions that exist right now look great.

Sabrina: Yeah, I like how you frame these two problems. I think they’re definitely key pain points that we’re also seeing, and I think as analytics are moving more towards the edge, especially for use cases around things like streaming and data apps, and as existing workloads grow, it’ll be interesting to really see how some of these new technologies play out in the fullness of time.

So just to wrap up here a little bit. It’s really impressive how fast Fivetran has grown over the last two years, and as you said before, it hasn’t always been an easy journey. It took two years of solving a different problem before pivoting, and then it took some time to get some of those first customers. I’d love it if you could share some of your key learnings for founders.

George: It’s funny, once it was passed a hundred people, it looked the same the whole time. It helped that our growth was extremely steady over the last, you know, five years. So, if you’re growing, you know, 100% a year, that’s hard. But it’s really hard if you grow 200% one year and then 20% the next. It helps if things are steady. Again, that’s something that is as much a property of the market as your company, so you don’t exactly have control over that. So, it’s not maybe very actionable advice. But that has definitely helped. Another thing that has helped is a large percentage of the leaders of the company are people who have been here for many years. We’ve had a very stable leadership team. I think a lot of startups will replace their key leaders of all the key functions every time the company gets twice as big. And if you have to do that, you have to do that. But, through a mix of luck, of, I guess, hiring the right people early, and of really giving people a chance to learn and improve themselves, we’ve been able to maintain a more stable leadership team than I think most startups who have gone through such rapid growth. And that has some big benefits in terms of like controlling the amount of drama as you grow, you know, having these people who have been here for a long time, who have seen it grow up, they can manage certain categories of problems that are harder when it’s a revolving door.

Sabrina: Okay, so I wanted to wrap today with a round of lightning questions. So we asked this to all of our guests on the intelligent applications podcast series, and so — aside from your own, what startup company are you most excited about in the intelligent application space and why?

George: Probably Materialize. I should disclose that I am a very small angel investor in Materialize. The reason that they are so exciting is that they have started with the hardest problem — the hardest unsolved problem, let’s say, in database management systems, which is materialized views. And then worked outward from there. Whereas everyone else starts with all the things that you already know how to do, and then they try to do materialized views at the end, and they never are able to make a very good implementation. If you can solve that problem, there are some really dramatic implications.

Sabrina: Outside of enabling and applying artificial intelligence to solve real-world challenges, what do you believe will be the greatest source of technological disruption in innovation over the next five years?

George: I think that artificial intelligence is incredibly significant. I have a car that basically drives itself, a Tesla, and it’s amazing. But I don’t think it’s going to be a horizontal technology that changes every space. I think what people do with data is mostly going to be the same and is not going to be impacted by AI in the context of businesses managing their data. I think, actually, it’s going to have a lot less impact than people think.

Sabrina: And what is the most important lesson, likely from something you wish you did better, that you have learned over your startup journey?

George: When we started Fivetran, we were too early for what we would ultimately do, and we solved that problem by waiting, so timing is incredibly important, and you don’t control it. That is a big challenge for every startup. Are you too early, or are you too late? And if so, what are you going to do about it?

Sabrina: And then last, just fun question, why the company name — Fivetran — maybe it’s a play on words to Fourtran, but just curious why, why the company name?

George: Fivetran is a pun on Fortran. That goes all the way back to the original, original idea for Fivetran, which did not last very long at all, which was to make a vertically integrated data analysis tool for scientists. You know, way back, Fortran was how most scientists would write their data analysis code. So, it was a short-lived idea, but that’s where the name comes from. And, you know, we changed the idea and never changed the name, but it has turned out to be a very good name. It is reasonably memorable, and it is unique. If someone types Fivetran into Google, then they’re looking for us. And that has allowed us to measure the growth and awareness of our product using Google Trends, which has been incredibly helpful. And so, I always recommend it to founders. If you’re doing a B2B company where the name maybe isn’t that important, make sure you have something that registers a zero in Google Trends.

Sabrina: Yeah, it’s incredibly difficult now these days with all the startup companies — people have gotten pretty unique in the company name. And it’s challenging when you pick a name that’s an adjective or noun or something that is already a word. I’ve seen some interesting ones.

George: You want to know something really funny, and not a lot of people know this, but in the old days of Y Combinator, when Paul Graham was still there. Paula Graham was the master of coming up with names. And anytime a company had a name that was a problem, and they couldn’t get the domain they needed, or they discovered it conflicted with someone else’s name, they would send them to him. And then they would sit there in his office and figure out a new name. And he was so good at it. It’s like his secret talent. It was not needed for us, but there were some other companies in our batch that were marched off to go figure out a new name with PG.

Sabrina: That’s awesome. Coming up with that company name is one of the most challenging things. Well, thanks George for being with us here today. It’s been a lot of fun, and I appreciate you joining us and spending some time with us today.

George: Nice to be with you.

Coral: Thank you for listening to this week’s IA40 Spotlight episode of Founded & Funded. To learn more about Fivetran, visit Fivetran.com. That’s F-I-V-E-T-R-A-N.com. Thank you again for listening, and tune in in a couple of weeks for our next episode of Founded & Funded with Snowflake’s Bob Muglia.

In this IA40 spotlight episode, Managing Director Hope Cochran talks with Tesorio Co-founders Carlos Vega and Fabio Fleitas about getting finance teams out of the world of manual, error-prone and inefficient spreadsheets and equipping them with intelligent applications more common in other parts of a company. That’s where Tesorio comes in — they offer automation solutions designed to help companies manage accounts receivable — in other words, they help companies turn their revenue into cash. The platform replaces that tedious and manual collections processes with accurate, real-time predictions, optimized workflows, and actionable insights based on behavioral trends. Tesorio raised a $17M Series B in July to expand its go-to-market efforts and last year landed on Madrona’s list of the country’s top 40 intelligent applications, a list we launched to recognize companies applying machine learning to solve business problems better than ever before. We think these intelligent applications will define the next generation of innovation and shift the SaaS technology landscape into application intelligence.

In this episode, Hope, Carlos, and Fabio dive into where the idea for Tesorio came from, how the company pivoted from the original idea, how these two met, the hard conversations any co-founders should have before deciding to launch a company together, and so much more.

This transcript was automatically generated and edited for clarity.

Hope: Hello, everyone. I am Hope Cochran, I’m a Managing Director at Madrona Venture Group. I am excited today to be with Carlos Vega, the CEO and co-founder of Tesorio and Fabio Fleitas the co-founder and CTO of Tesorio. You know, When I think back to when I met these two fun friends, I was excited by the fact that they were all about solving the problem of forecasting cash flow. I am a recovering CFO for those of you that don’t know me — I have spent many, many hours of my life in spreadsheets trying to forecast cash flow and often getting it wrong, and often being an area where there was a lot of errors. There’s so many pieces that go into forecasting cash flow, whether it be the AR ,the working capital, the AP, that all amounts to what is your ending cash balance going to be. So when I met these two, and that was their vision to solve, I immediately identified with the problem, and we were off to the races. So we’ve been on quite a journey together, and I just welcome you two to this podcast today. Thank you, Carlos and Fabio.

Carlos: Yeah, thank you so much, Hope. I’m really excited to be here.

Fabio: Thank you.

Hope: I know the moment I met you all, and that’s when I got to be a part of your journey, but let’s start with where you all came up with the idea for Tesorio and solving this problem.

Carlos: Yeah, thanks. This is Carlos — thanks so much, Hope. Before beginning to Tesorio, I spent about a decade working in finance. Most recently, I spent a couple of years working at Lizard doing investment banking in Latin America, and while I was there, I actually co-founded a factoring business. Factoring for those who are not aware is purchasing of receivables at a discount so that people can get their cash flow now instead of waiting to be paid. You know, what we were trying to do is say, “Hey, look, let us help you with your cash flow and model it out, and then wean you off a factor and get you a proper line of credit at a bank.” But folks weren’t really biting. The way it ended up feeling is almost like payday lending for business. Where we’re providing those discounts, providing people the money up front, but they weren’t really taking the advantage of a better way of doing things.

When I went to business school, I knew there had to be a better way, and that’s what I was out to solve. So, we started with accounts receivable because, given that experience with factoring and other things along the way, what we realized was what people really needed around cash flow was predictability. And the most volatile part of cash flow that you don’t control is your cash inflows because any money going out the door, you have agency over, right? You can choose not to mail a check. You can choose to slow down your hiring. But you can’t really force someone to pay you. And one of our customers said it best — Steven Odell, who was at Slack at the time, said, “Revenue’s not real until you get paid.” So, when we start thinking about that, it’s like, all right, what does that mean? At the end of the day, it means that you grow the business. You have this revenue, but in order to generate value, you have to have the cash to pay for your expenses, provide returns to your shareholders, and fund future investments. When we started looking at what we could do we said, look, can we provide software that helps folks manage the day-to-day so that they’re not stuck in spreadsheets? And also give them that predictability they need so they can plan for future growth. And at the end of the day, you know, that’s what we have here with Tesorio. It was an important problem for every business to fund their growth with their cash flow. And it was a problem that we discovered a lot of folks had that we could solve.

Hope: I love that concept because, really, it’s so basic and yet, so true that revenue is not real until you get paid. Truly we know that we need the cash in our bank to pay the expenses, the most basic concept out there. And yet, somehow, it’s such a challenge to not only forecast when it’s going to come in, but know it’s going to come in, have that certainty. And so, you all do a fabulous job of looking at patterns and ensuring that we can approach those payees at the right time and ensure those payments and also help with that predictability. So again, “Cash is king” is the mantra here, and how can we help businesses maximize that?

Carlos: Right. And you know, what’s a funny part about it? How many times have you not gone out with some friends and spotted someone a couple of bucks, and you’re like, “You know what, don’t worry about it.” Cause it’s kind of awkward to tell your friend, “Hey, remember I spotted you $20 bucks last time? Can you give me $20 bucks back?” You just forget it. And you go on. It’s an awkward conversation, even personally. And in business, if you think about it, a lot of things happen, and you close a contract, you sign it. That’s perfect. But then going to have that conversation or remind someone, “Hey, remember we closed this contract, and we’ve been doing business for the last three months. You still owe me some money.” It’s not comfortable for any of us. Yet, folks were still doing this in spreadsheets. And what would you do with a BDR that took your entire database and just blasted the same message to everyone? What would you do with a sales team that didn’t even have a CRM if you’re looking at investing at this company? Well, that’s actually what finance teams are living with. A lot of companies we’re finding are sending the same message to everyone. And then, not using that information the right ways, as you were saying, as inputs to forecast cash. Every little bit of information about who is trustworthy when they say “I’m going to pay you by next Friday?” Who says they’re going to do it, but they don’t pay that? Or who consistently pays late or has a certain pattern? All these things can be discovered with algorithms. And then you can use that to make your cashflow more predictable, as you’re saying, not only to anticipate when you’re going to get paid but also to know when you should follow up with someone and do that the right way. It’s kind of interesting, everyone was doing this in spreadsheets when we came along.

Hope: I attest to that. Everyone does this in spreadsheets before Tesorio. I did that myself, and I experienced it many times. I think many people are still amazed by how messy and manual the back office is. You made the comment that the BDRs, the sales team, they have lots of CRM tools or different tools to help them in the sales world as people are focused on getting the revenue in the door. But then the finance team that’s trying to get the cash in the door is really based on messy workflows and complicated spreadsheets that often break. I often bring this to my friends and peers about how messy the back office still is, and they’re always surprised. But I lived it, and I know it, and I have the scars to show for it. And I love the fact that you all are bringing that solution to bear.

One of the things that I want to explore with the two of you, since I’m so fortunate to have both of you on this with me, is when starting a company, finding that partner or co-founder is difficult and challenging and yet so vital. I often say that the co-founder relationship or the founder relationship is harder than marriage. There are so many nuances, your lives are so intertwined. You guys have been at this now for a while, and I’ve really enjoyed working with both of you and your partnership. I’d love to hear how you found each other, what you feel was complementary in skills, etc.

Fabio: So, Carlos and I met actually, while we were both in college. I happened to be an undergrad in computer science at the University of Pennsylvania, and Carlos was in business school over at Wharton. We were both independently involved in tech entrepreneurship in Philly / Penn. I had at the time co-founded a fellowship program that got funded by the city of Philadelphia to get students from all around the world to come work at Philly-based tech startups. And so, I was getting really involved with the tech community there. Carlos was running a group at Penn called Founders Club that was helping other like-minded students that are interested in entrepreneurship or founding a company to work with each other. How we ended up meeting was because of a professor. He was starting a brand new class that was focused on tech entrepreneurship, and he kind of recruited Carlos and I independently to get students to join his class. He wanted both business students and computer science students at the time, which was a very unusual mixture in school usually. And from there, I remember still meeting Carlos in one of the sessions with talking with the professor he had applied to Wharton with an idea of Tesorio. So, we met there and started working together. Carlos is originally from Panama, I’m actually originally from Cuba. We’re both native Spanish speakers — we both immigrated to the U.S. as kids. So, we had that connection between us. And then, before actually making it an official co-founder relationship, Carlos and I both actually worked together for about a year. And then spent another nine months or so before getting into YCombinator. So, for us, it was actually very important to work together. And Hope, as you said, it’s a little bit like marriage. This time that we spent together is basically almost like we were dating before we decided, “Hey, we have a good relationship together, we can work together long term.” And at that point, obviously, it gets to become a very serious thing. And so we had the opportunity to work together before making it official.

Hope: I love that — definitely understanding each other’s working styles, knowing what each other’s strengths and weaknesses are, is so important to going into this. And, just having really honest, and I want to say hard conversations — don’t know if you both did that. But I really encourage my founders to have those: What does your life look like? What are the things that work in your life and don’t work? I’m curious. Did you all explore that together so you would know that you had the same values going into starting a company?

Carlos: Yeah, totally. I would say that you’re spot on, Hope. Making sure that you have the value alignment just in life generally is so critical just because when you are starting a company, your core values are so key to the success of the culture, and culture is everything, especially once you get 50 people or so. And that, I’d say, is one of the key things we aligned on. And then the other one — talking about hard conversations, which we had a couple of times. But figuring out how you are going to split the work is something founders sometimes avoid in the early days, but you have to have that conversation. And then also, “How are we going to split the equity?” That’s a really basic conversation that could lead to a lot of awkward situations down the line if you don’t have it early on. To anyone who is getting into a co-founder relationship, you can’t avoid those.

Fabio: To add to that. I love to say that Carlos and have a really great working relationship. I think we respect one another. We know what we are good at, and what we’re bad at. And with that, we can support one another and go through all of this journey of building a company together. And I think again, having a great co-founder is arguably one of the most important things for the success of a company. And although it is difficult, it is one of the most important early decisions you’ll make. And finding that right person that you can gel with, that you can have the right values with, and work together and see a long-term relationship and success with is really important.

Hope: Yeah. You mentioned knowing your strengths and weaknesses. I often find that we’re attracted to people that are similar to us, and yet that’s not what you want as a co-founder. You want someone who’s good at the things you’re bad at, and that’s a little bit of a stretch — sometimes it shouldn’t be your best friend who is exactly like you. Can you guys talk a little bit about some of the ways you complement each other?

Carlos: That’s a great point. Right? And it was one of the reasons, like Fabio mentioned, this professor was putting together people from the business school and the engineering school and the design school. Because most of what was happening at the time on Penn’s campus was, business school students would go try and start something together and engineers would go and start something together. And people weren’t really mixing, and Fabio and I were very fortunate to meet. From a skill perspective, Fabio is an engineer and I’m not. I can’t do what he does. And I’ve got the finance background, and I love product and sales. Where we overlap is that he likes product, but he can actually build product. Then also, just more broadly, it’s interesting from a personality standpoint, it’s good to have a balance, right? Like I’m more passionate, more excitable, you know, I have more highs and lows, and I live with the passion and energy of everything that’s going on. And Fabio’s just even and steady the whole way. And those types of things are really valuable as well. Because it’s good to have both and have that complementarity. And then, from a pure skills perspective, I really admire Fabio’s ability to just work on a process and be extremely organized. He can’t live without his to-do lists. We all joke at the company that if Todoist were to go down, I think Fabio would stop breathing or something because everything goes on a Todoist.

Fabio: It’s pretty true.

Hope: Well, let’s jump into what Tesorio does. And I always like to look at companies from the lens of the customers. You guys have a tremendous list of customers — really amazing logos and names.o name a few, we’ve got Twilio and Slack and Couchbase, Smartsheet and many, many others. What do your customers give you feedback on what they love about Tesorio?

Carlos: Accounts receivable automation is like the core of how we started. That’s the workflow bit. For those who aren’t as familiar with the back office as you were mentioning, Hope, everyone knows there’s big sales teams out there closing a lot of business, but then in the back office in finance, you’ve got maybe a handful of people. At a company as big as Veeva, you’re talking about 15 people. Slack when they got acquired for $27 billion, only had two or three people in the back office doing this. And what these folks are doing when I say, “doing this,” they’re following up with your customers to make sure you get paid. So, what the customers tell us all the time that they like is, “Hey, your product feels like it was built by a collector — like one of us built it.” And so that’s pretty exciting to see. And then when you look at the stats — our DAU/MAU, the daily active user over monthly active user, is actually almost 20% higher than Slack’s. So, if you use Slack a lot, which we all do every day, or some of us use Teams — imagine something that gets used 20% more by the teams that use it. Like literally doing your job is using this tool. Again, what people say and what they love is like, “Hey, this feels like it was built by one of us,” and then the stats back that up because it’s what they need to do their job.

Hope: One of the things I loved when I met you both was the fact that you talk in DAUs over MAUs, meaning how many daily active users versus monthly active users that is not a common KPI for a SaaS company. And yet it shows the engagement and the amount of time that your users are in your application. I think I’ve also heard you say, Carlos, that you want it to be the coffee mug experience.

Carlos: When we talk about that is, what we want, what we strive for is that this is the first thing you look at in the morning if you’re in finance. And for a lot of our customers, I kid you not, it’s one of their default tabs in Chrome. So, every time they open up Chrome, Tesorio, next to Salesforce, next to their ERP is one of the things that open up. The team started calling it the coffee mug experience because every morning you get your cup of coffee, you sit down and look to see what’s going on with the business. How much cash do I have? What’s going to happen? What’s going to come in? What’s going to go out? And if you can have that in one place, that changes your day, instead of having to look everything up in Excel. And that was really critical. I started earlier by saying that people use a product daily, and they feel it’s created by a collector. But that comes from a lesson we had — we pivoted to what we do today. We knew that there was a problem with cash flow. But we pivoted to what we do today in focusing on accounts receivable because we learned two things. First off, if you want to do something a little bit more strategic, you have to earn the right to do that. And by that, I mean originally, we started focusing on supply chain financing, given my factoring background. But what we realized is that what people really needed was a solution to their hair-on-fire problems. The strategic day two problems of how am I going to finance my problem takes a second seat if you don’t even know when you’re going to get paid, how are you going get paid, and you don’t know how to follow up with your customers. And so, when we were building Tesorio, I literally worked out of Veeva’s office for three months next to their AR team to see how they did their job and what they needed to solve their day one problems. And then, after that, now we’re getting to the place, right Hope, that’s part of the vision, where we can start forecasting your cash flow. And then part of the longer-term vision is then now that you are managing your cash flow day to day with this tool, then you’re forecasting what’s going to happen. Then you can get back to where we started, which is how do I finance it? How do I do things down the road that are going to allow me to grow the business in a longer-term lens? So interesting stuff, a journey that feels like we’ve come full circle sometimes. It’s been exciting

Hope: I love that you put a lot of good points in that. Number one, you guys have been maniacally customer focused, and that shows in your NPS scores, which are extraordinarily high for a SaaS software company, which is very impressive. The other thing is, you mentioned Carlos, how you need to solve the burning issue in order to get in. And then we’re going to move up the chain to the broader strategic issue. And I think that burning issue is also where you’re getting that strong engagement — the DAU vs. MAU — they’re loving your product, and they’re using it every day. One of the things that I observe in both of you is that while you are maniacally customer focused, you’re also really good at stepping back and not just responding to exactly what that customer is telling you they want — different customizations — but you are able to look across all of your customers and decide what would be useful for all of them, so you’re not just focused on one. Those are hard choices. Can you guys talk a little bit about moments in time where you’ve had to really focus your product roadmap on things that apply to all versus maybe responding to one request here and there.

Carlos: Yeah, definitely. Gary Wiessinger, who’s the head of product at NetSuite, once told me, “You can measure a product manager by how many times they say ‘No,’ more than how many times they say ‘Yes.’ And that ratio is really important. And that’s hard as a founder because you’re so passionate about building things, and you want to get your vision out there. And when you see people responding to it, you almost want to say “Yes,” “Yes,” “Yes,” all the time. There there’s been a couple of things. In the early days, one of our very earliest customers was Spotify, and they were very large. And they wanted a bunch of features that would’ve brought us toward almost like building a treasury management system. And one of the key things for us from the beginning was, again, learnings from having pivoted and just along the journey, was we just simplified as “Insights to action.” One of the core things we’re trying to do from an intelligent applications perspective is tie the insights that you get to the actions that you actually take every single day. So, building a TMS or treasury management system would’ve required a whole bunch of work that’s almost like just purely data aggregation. Let me pull data from all of these different bank accounts, put it in one place, and then forecast out my cash flow. But what was really important for us was actually aggregating the transactional data before it happens. So that means going to the finance systems of record, like your ERPs, your CRM, you’re procure-to-pay tools, where you can see cash flow is going to happen — cash is going to flow, but it’s going to be because this customer’s going to pay me or I’m going to pay this bill. And so, for us, building integrations to those systems was more important than building a bunch of integrations to banks, into FX platforms and that sort of a thing. And so that was one where it was really hard, you can imagine, to walk away from our second customer, who we flew out to Sweden to spend time with, and you know, we’re still really close with their whole team to this day and say, “We’re not going to do this,” because customer number two might want this, but customer number one and then 3, 4, 5, 6, 7, 8, 9, 10 don’t, and it doesn’t align with our vision. I still think back to how painful it was. True story — we were curling with the treasurer of Spotify the day they went public in Sweden. Like we were very close with them. But to say no to them was really difficult and it’s hard. But back, similar to your hard conversations with founders, there’s just a lot of hard choices you have to make along the journey. That’s probably one of the toughest — sticking to your vision in the right ways.

Fabio: And to add on to that, another thing that we had to do alongside making those tough product decisions is we still had to build the product in a way that would allow configurability within it. So, one kind of core product philosophy we have is configuration over customization, especially when we’re dealing with the types of large businesses that we’re working with. Being able to support them and their slightly different workflows and slightly different use cases in a way that is not just a bunch of customization has been something that we had to do from very early on. And it allows us to adapt to these types of larger, mid to large-size companies. And I think that was another key thing that we had to make a decision on pretty early on is that we needed to support this type of configuration and continue to expand and support that as we grew.

Hope: Such good examples. And you guys are just reminding me that part of being leaders in a company and founders is just a series of difficult decisions. I do want to come back a little bit to the fact that here we are on the intelligent applications podcast, and what is an intelligent application? It’s something that is enabling a workflow or something using AI or machine learning and really helping make things more efficient in the back office as well as smarter. That’s been something you guys tackle from early on in your solution. And so, I’d love to hear about how machine learning and AI are incorporated in your offering.

Fabio: Definitely As we’ve talked about here, historically, the entire accounts receivable process has been driven by tribal knowledge and spreadsheets within the accounting team. They know based on personal experience that a certain customer always pays 10 days late, five days late, while another customer is often promising to pay by a certain day, but then they can’t be relied on versus other ones can be relied on. Right? All that information is all tribal. It’s all just within the accounting, maybe even just within an individual. So, if someone leaves that company or is on vacation, effectively, that data is lost. And so, what we’re able to do, is take all that data and actually incorporate it into the product, so it’s not just siloed within individual people or individual departments. Our models are able to look across all of the customer’s payment history and determine, without human biases, when will they pay and also what will they do. Our models are trained across over 50 data points of anonymized invoice history. We’re covering billions in transactional volume to further refine and improve that forecasting accuracy. The product is able to learn from all these data inputs, both from a financial perspective and a behavioral perspective. We consolidate that all into our product and tying back to our integration strategy — which is that our system can connect to various different data sources from ERPs to billing systems, CRMs, emails, and everything like that — we’re able to consolidate all of that, bring all of the actionability and insights into one single place, learn from all of that data and then bring it so that users can actually take action. Cause one of the core things about our product is it’s not just providing insights, but it’s always tying it back to action, and what can they do about it? What can they do if their cash flow isn’t the way they expect it to or if the customer’s not paying them the way that they usually tend to pay? What can they do to affect their workflows affect their cash flows? And that’s been something that from pretty early on, we noticed was a problem with our customers, and we went out to solve that.

Hope: I love this phrase that Carlos talked about earlier, which is “Insights to actions.” So often in systems, you’re either a system of record — where it’s the system where you’ve got the data, and you’re doing the thing — and then you’ve got the analytics of that system of record. Very rarely do you find an application or software that allows you actually to do both? Where you can get not only insights but you can actually take the action there as well. And I think that’s really unique to your solution.

Carlos: That’s a good observation. Even databases are either transactional or analytical. And so that’s from a technology standpoint, that’s one of the things that’s been a fun challenge is figuring out how we have both — the analytical aspect tied to the transactional day-to-day aspect from a tech stack perspective.

And then, interestingly, just before we move off the subject, if I were a founder listening here, a little piece of advice I’d share is, when you’re building intelligent applications, there’s no like blanket way to do it. I think it’s really important to have the context of the culture of your customer base that you’re planning to address and the cost of being wrong. So that’s something that’s really interesting, like for us, we’re proud of our analytical skills, right? We’re finance people, Fabio jokes, but every single meeting — I think in 95% of meetings, I’ll open up Excel and start doing something right. And people joke, but it’s the way we like it. So, if we’re building an intelligent application for people who like analytics, you have to take that into account, and you can’t get cash wrong. So, for us, when we think about machine learning, AI, and all of that, we think about it in our world, at least, more as the supporting actor. The lead, the hero, is the user. And so, for us, that leads us to think, okay, more than here is your cash flow forecast, or here is how much we’re going to collect, it’s how do we pre-process the data so that you can go and make a decision. The really key thing is making sure that we expose enough levers for the customer so that they have ownership of the output.

And that’s something we even got wrong at one point. We thought we should forecast out the cash flow using machine learning, but it felt too black box. That was a really key learning for us — you know, having that context, and for the culture of your customer base, the cost of being wrong are going to influence how you apply machine learning and AI tools.

Hope: So, you took the next question right out of which is great. No, that was exactly where I was going to go. Us finance people like to think we’re good at forecasting, and we like to be able to explain our forecasting and explain what the levers are and what’s impacting it. With machine learning, the truth is it’s probably more accurate than us, and yet we don’t really want to admit it, and we want to be able to explain it. We don’t want it to be a black box. So, you do have a hard customer to sell to, to utilize this type of technology. And you and I have had that conversation, which is why we both led to this topic, and you’ve really addressed that well with your customer base by giving them more transparency and allowing them to feel like they are the heroes. I love that phrasing.

Carlos: Yeah. It’s interesting. When you look at cash in the bank, we joke that if it already hit your bank ledger, cash already flowed. So, that’s why we say we have to get to the things before it leaves or comes in the door. And that’s why all those integrations we talked about earlier are so important. And that’s what allows you to give them the ownership, right? If they’re addressing the cash before it goes out the door and they make the decision on how it’s going to go out and how it’s going to come in, then they can own the forecast and have an influence on it.

Hope: I want to turn a little bit to the macro environment. I know that we’re all watching the capital markets and the macro environment and how it impacts us, whether it be the funding environment or how does it impact our revenue line and our customers. And so, as you think about your solution set, has it become more important to your customers? Is it something they’re cutting back on? How do you think this period of time is impacting the demand for what you’re offering?

Carlos: I guess I’d summarize that we’re cautiously optimistic because I think we’re seeing some tailwinds rather than headwinds. Which, if you think about this from a purely analytical perspective, it makes sense. With inflation the way it is, every 90 days, it’s more than 2% of devaluation. And so, where does cash typically get trapped on a balance sheet? As we talked about before, you control, when cash leaves the door, you don’t control when it comes in the door. Receivables is actually one of the biggest places where cash gets trapped.

And so, every 90 days, your receivables are late, that’s cash you didn’t put to work and just devalued by 2% in real terms. And on top of that, it’s kind of a double whammy with the cost of capital going up. So as interest rates are going up and the cost of capital is going up, the inverse of that, as we know, is your hurdle rate, and the rate of return your investors are looking for is going up. So that cash that you’re not putting to work, if you’re not getting it faster in the door and deploying it, the opportunity cost of that is also going up. And then the market just in general, as we all know, is no longer favoring growth at all costs, they want sustainable growth. For a long time there, it was so easy to go get cash from other sources, and now that’s not quite the case. So, what we’re seeing is that this is more of a pain killer than a vitamin in a lot of ways, and folks also need solutions fast. That’s been a really critical thing, at least for us, back to the theme of intelligent applications is how can you get these products deployed quickly? Because an intelligent application that’s going to take a year or two to get deployed might have great ROI someday in some spreadsheets, but it’s not going to help me achieve my goals now. And so, a solution like ours that can get in there quickly on average, 30 days to get live and 60 days to show ROI, that’s something that folks are really valuing and that you’re seeing today — folks need these solutions before their next earnings call, not by next year.

Hope: Yeah, I think we go back to something we said earlier in our conversation, which is cash is king. And when the economy gets tight, and tough cash becomes even more important. And therefore, the solution that you’re offering is paramount. I want to step back a little bit and talk about broader technologies in this space. And as you think about this space, what is a company or a startup or interesting technology out there that you two are both excited about?

Carlos: I’ll share one that changed my day-to-day. I think Clockwise was really cool because you just plug it, and it automatically starts allotting focus time. And you set rules around meetings and time between meetings and that sort of a thing. And it changed my life from just feeling exhausted at the end of the day of going back-to-back-to-back Zoom meetings, because we’re a distributed company, to actually having breathing room in between to get in the work that I need to get done.

Fabio, I know we’ve talked about another one. I’ll let you take the next one because it’s definitely more of a tech nerd out.

Fabio: Yeah. Yeah. Obviously, a lot of the IA companies out there are very CTO friendly, and the one I admire and I think is doing really cool work is Fivetran. Every company, as we already know, has a problem with getting data into a single place — it’s ETL, especially if you don’t have engineering resources. I found Fivetran is a really cool product that allows companies to actually connect all of their data streams and try to consolidate them into one place. And why I also think this company is cool is because it’s a little bit of a reflection of our product as well. We consolidate data from all these different financial, CRM, billing, email data sources for our customers to use, and Fivetran is doing similarly for their customers so that they can get their internal data together. It’s a really cool piece of tech and a pretty exciting company.

Hope: I love it. And then, so those are existing companies, as I think, five years out, what are some technologies that are coming to pass or things down the pike that you really think will disrupt how we do work today?

Fabio: You know, on my end, augmented reality, I think, is one of the biggest sources of the kind of technological disruption and innovation that’s going to come up soon. I’ve always felt that the same way smartphones changed the way we live and work, augmented reality is going to be that next big leap that’s going to really change the way we work and live.

Hope: So, we’ll end on lessons learned. And I know I certainly have a lot of them in my past history. But as we indicated before, a clear observation is that being co-founders, being CEO and leaders of a company is just hard. And I think it requires more persistence than anyone ever realizes — that ability to climb over any wall that’s put in front of you. I’d love if either one of you would be vulnerable enough to share some lessons learned with other founders and words of wisdom.

Carlos: Yeah, I think for me, the most important lesson just looking back is realizing that you’re never done. As a founder, you feel this extreme urgency. In the early days, someone once told me if you’re asking yourself if you’re moving fast enough, you’re not. But you know, at the sacrifice of what, right? Before being founders, we’re people. We’ve got families, or we’re in relationships. And sometimes I think for a very long time, in most of the early parts of the journey, or until recently, I kind of worked like there was no tomorrow, you know, working till every hour at the end of the day and at all times of night and giving up sleep and all of that. And then, eventually, it clicked. You never really de-risk the company, you just attain new levels of relatively higher stakes. So, even though you think you’re going to do this, and then you’re going to be all good, and you’re going to be done, and I have to do this now and forego a lot of things — I’m not going to go to the extreme of saying that you’re ever going to have good work-life balance as a founder. I don’t think that’s part of the job description at all — but realizing that there’s always going to be more to keep doing tomorrow is very critical. So, at the end of the day, I guess what I’m getting at is, to take care of your health, take care of your personal relationships, and remember that because it’s a long journey.

Hope: Yeah, it’s a marathon, not a sprint. and I think people go into startups thinking it’s going to be this sprint-like event. When really, when you look at the companies that have gotten to scale, it takes 10 years. It truly is a long road and a marathon. And you have to manage that in a healthy way. Fabio?

Fabio: The most important lesson I’ve learned throughout this entire startup journey is it, maybe it sounds a bit cheesy, to be honest, but I think it’s very true that — the people you work with matter the most. And I think it obviously starts with your co-founder and the relationship you have, but it quickly grows. It grows to everyone in the company, the board, the VCs you’re working with, everyone. And who you work with, I feel, has one of the biggest impacts on the success and outcomes of your startup. I believe that finding the right set of people, the early employees, your early investors, and your co-founder will have such a large impact on how things will go. And I have truly felt very fortunate to have this opportunity to work with such brilliant people. It’s ultimately what excites me most about the entire startup journey.

Hope: Those are great words to live by and great words to take to heart. Carlos and Fabio, thanks for spending this time with me together today. I always enjoy it when I get to have a conversation with both of you, and this was right in there, so thank you so much.

Fabio: Thank you.

Carlos: Thank you, Hope.

Coral: Thank you for joining us for this IA40 spotlight episode of Founded and Funded. If you’d like to learn more about Tesorio, they can be found at Tesorio.com – that’s T-E-S-O-R-I-O.com To learn more about the IA40, please visit IA40.com. Thanks again for joining us, and tune in in a couple of weeks for our next episode of Founded and Funded with Battlesnake Founder Brad Van Vugt. We’ll be spotlighting another IA40 winner next month.

In this episode of Founded & Funded, Investor Ishani Ummat talks with Zayd Enam, Co-founder and CEO of Cresta AI, one of our 2021 IA40 winners. The two dive into the important topic of how AI can be used to help empower people to be more productive, specifically in the context of call centers — or now more commonly referred to as contact centers — because they include much more than phones.

We hear the story of why Zayd dropped out of his Ph.D. program to pursue launching his own company based on what he explains as the “schlep blindness” of contact centers. He also discusses the unique way he landed his first customer, which happens to be Intuit, one of the largest financial software companies in the country, and the benefits of taking a modular approach versus a full rip and replace of a customer’s entire system.

This episode may make you crave a Costco hot dog, but you’ll have to listen to find out why.

This transcript was automatically generated and edited for clarity.

Ishani: Hi, everyone. I’m delighted to be here with Zayd Enam today, the CEO of Cresta AI. Cresta is a productivity suite for the modern contact center that leverages artificial intelligence to drive a better customer experience in real time. We’ve all struggled with contact center experiences and so we’re really excited to have Zayd here to talk about the unique angle that Cresta takes.

Cresta was selected as a top 40 intelligent application by over 50 judges across 40 venture capital firms in 2021. A quick moment on that. At Madrona, we define intelligent applications as the next generation of applications that harness the power of machine and artificial intelligence to create a continuously improving experience for the end user and solve a business problem better than ever before.

Zayd, we’re super delighted to have you with us today.

Zayd: Awesome. Thanks so much for having me, Ishani.

Ishani: Why don’t we start it back at the beginning. Cresta was formed out of Stanford, but I believe you dropped out of your Ph.D. program. Tell us a little about the work you were doing and how that led to the founding of Cresta.

Zayd: In the Ph.D., I was working on how artificial intelligence can be used to help empower and augment people and make them more effective in their day-to-day work? Originally, I was looking at applications of artificial intelligence for things like helping graders and teachers grade assignments more effectively and be able to give better feedback to students based on the most effective feedback and based on common mistakes. It was really a thread of a lot of the work that was done in the early ’80s at the Stanford AI Lab where this concept of intelligence augmentation, which is pioneered by a lot of the folks that started that lab. And it’s a continued thread of work in terms of understanding how humans can be bicycles of the mind and extensions of bicycles of the mind. What ended up happening is I built software for teachers and grading assistants, and I built software for email and both those directions of the project, ultimately, didn’t lead to success. With email, what happened is at six months after I built it and I got a bunch of people in the Stanford building using it, Google released their smart buy project.

The day Google released the project, like 20 people messaged me that link because they had this big PR announcement. It was clear that Google had a data advantage there in the sense that they have access to billions of emails to train these kinds of models on. So, then I pivoted to graders and teachers, and there, the tricky thing is that a lot of universities and schools aren’t ready to adopt software. It’s just a slower market to try to get traction in. I ended up working with a bunch of offices in the Bay Area. I’d go sit down with someone and just observe the kind of work they do on a day-to-day basis. The goal was to build small tools to help augment or help automate basic things in the workflow. I started working with these sales and support teams, and, just as a grad student, I could sit down with a sales and support team and basically build these systems that would understand the effective way to resolve an inquiry or the effective way to have a conversation and provide these real-time prompts that guides someone through the conversation. At the first company I worked with while I was a grad student, within a few weeks, we were generating $100,000 more of sales per month — that’s more than $1 million per year. It was clear that there was opportunity here from a business perspective. And so, through a whole process, I decided to drop out. Because there’s something core to this, where there’s a lot of value to be delivered, it turned into this overall vision of using artificial intelligence to help empower people to be more productive. And it felt like a great place to start. And that’s ultimately how I got there.

Ishani: So, in many ways you were able to do the super early stages of customer discovery while you were in your Ph.D. program, spending time with different kinds of customers and mitigating a couple of these issues that we see. You know, you and I were talking about GPT-3 earlier — lots of people have good applications of big models like that, that feel like solutions in search of a problem. And the way that you got to spend your time doing this early customer discovery and building smaller tools for them was starting to figure out, okay, where is there a problem where I can really apply a solution that I know how to build, and I can build in a small way and create a wedge. So, a couple of trends there where you need to figure out 1) what’s a real problem to go solve. And then 2) where do I have a unique advantage to go do that.

Zayd: Right. And that’s a philosophy of my co-founder and Ph.D. adviser at Stanford — a gentleman named Sebastian Thrun. He started the Google X project and Udacity and won the DARPA Grand Challenge in 2006 with the first self-driving car. And his philosophy and what he teaches in his labs is basically that in order to go build a self-driving car, you don’t sit in the lab and research the future of computer vision, you go out in the Mojave Desert and figure out how to get those kinds of systems to work and then come back to the lab. And you’ll actually realize that you’ve made some fundamental breakthroughs in the technology and the engineering systems associated with it. That’s the same thing that happened with Cresta in terms of working with companies and identifying how can we make this really work for the customer and then coming back, understanding what are the fundamental technological breakthroughs that happened in terms of making it really work for the customer.

Ishani: It feels related to this concept of customer obsession. Madrona was one of the earliest institutional backers of Amazon. I know you think and talk a lot about customer obsession. Take us along the journey. You leave the lab. You start Cresta. What drove the decision to focus on contact centers? Contact centers are amazing. Notoriously bad experiences, long wait times, low NPS. You’ve shifted from call center to contact center, and that means that you have digital as a new modality and email and all these other components, but still, lots of friction. And everyone interacts with a contact center, right? The vast majority of businesses have them. The vast majority of customers are talking to them and interacting with them in some capacity, but it’s not obvious, necessarily, that augmenting humans who work at contact centers is a business to then go build. So how did you zero in on that as a customer segment? How did you obsess over that? And then talk to us a little bit about that first customer journey you had.

Zayd: I mean, the thing is they’re not particularly sexy, right? So, it’s not like a Stanford Ph.D. says, ” Hey, I’m going to go focus on contact centers and understand how these technologies apply and drive tremendous value or transformation there.” There are a few secular trends happening in general in the last couple of years where folks are rearchitecting their systems from on-premise to cloud-based systems. And you have folks adopting these multi-channel experiences, whether it’s phone, chat, email — it’s a space that’s going through rapid transformation right now. Because NLP and deep learning have transformed so much and are progressing so quickly that what’s possible in software now is just dramatically different than what was possible just a few years ago. So, you have a lot of change happening in the space and you have a fundamentally new solution set with deep learning. And it is just an opportunity there. That’s an analytical answer. I think the more emotional answer actually just was that, so Paul Graham has this essay in Y Combinator about schlep blindness. He talks about when Patrick and John started Stripe, everyone was building these niche social networks and these niche travel websites, and no one was solving the payments problem on the internet because it felt like a lot of schlepping around. To do it, you’d have to go deal with legacy APIs of banks and deal with how to do partnerships with banks and these kinds of things. But every website in the world needs some way to take payments if they’re doing business on the internet, and the experience was terrible. And I felt the same way about contact centers. It felt like a lot of schlepping around. You’ve got to figure out how you integrate into all these systems. It’s not exactly the most exciting part of the business for a lot of people. But the thing is, every company in the world has some way to interact with their customers, and the experiences right now are terrible, right? Where the average employee NPS within a contact center is less than zero, average customer experiences are very poor in terms of the way that they interact with contact centers. So, everyone has something. No, one’s happy with it. And I have something where I could, within a few weeks, have an impact on a team. Even though it wasn’t the dream of the Stanford Ph.D. to go work in the contact center, it felt like a lot of schlep blindness in why people were avoiding the problem. And so, I was really excited to lean in and solve that problem.

Ishani: Yeah. And I mean, by the same token of Paul Graham and the Y Combinator reference, you know, every incubator will tell you to stay away from the enterprise. Right? No one will tell you to go and drop out of your Ph.D., start a company, and then go sell to the big enterprises at the get-go. How did you even begin that early customer journey of getting someone to sign on?

Zayd: Yeah. Another Y Combinator advice, which is, don’t do big deals. But I think one of the most important skill sets is figuring out what advice applies to your situation and what advice does not apply to your situation. So that one, I don’t think applies. For Cresta, the market for artificial intelligence makes a lot of sense to start an enterprise because that’s where you have these teams of contact center agents with more than 250 people. And you’re able to use artificial intelligence in a way that can capture repetitive patterns across the conversations that multiple people have, and that’s where you can provide the most value and the largest segment of the market on which to make an impact.

Usually, it’s hard to go to enterprise because they have a bunch of requirements to go engage in that business and have an impact there. So, people often start mid-market or SMB segments of the market. This is where I was slightly unconventional — I tried cold calling and prospecting in terms of getting our first customer, and it wasn’t getting much traction. But then, through Scott Cook, I cold-emailed him after a presentation, and I got this meeting with the CIO of Intuit. And I went down there to present Cresta and the results that we had in terms of my Ph.D. work and asked him, “Hey, I’m starting a company in this space, and I’d love to work with you as our first client.” He said, “This really ties into the Intuit strategy. This is what we’re trying to do from an AI perspective, and this is a really great project, but we can’t work with you because you’re a one-person company, and we’re Intuit, we’re the nation’s largest financial software company, And we can’t really work with a company like yourself. But if you want to sign up as an intern to my group, you can sign up as an intern for the summer. I took him up on the offer. Once I got in, I got access to their data systems and basically worked with them on the technology and deployed the first software to their group in Tempe, Arizona. And the first person to use it was actually one of the top salespeople there, and he loved it so much that he got the rest of the team to start using it. The whole team’s performance doubled in a few weeks. And so that’s when they were looking at it and saying, okay, now we want to take that team and expand it to this other bigger team in Virginia. That’s when I went back to them and said, “Hey, this is now becoming a really serious project, and you probably want some sort of enterprise agreements around this. We came to a standard SaaS agreement, the only addendum being that this clause hereby terminates Zayd Enam’s internship. Then I had to sign as the CEO of Cresta and then as a former intern at Intuit, which is a fun reminder of that contract every time it comes up again. But yeah, that’s how we got our first customer. And then, once we had that case study, we were able to go to more enterprises with that credibility.

Ishani: That’s an incredible story. And I think YC needs to write that into their playbook around how you go build a first enterprise customer. First of all, most startups don’t go to enterprise, but if you’re going to go to enterprise, follow the playbook that Cresta lays out.

Zayd: Yeah, I think you learn a lot, and I think you do things that don’t scale, right? So, take an internship at a company — that’s something that doesn’t scale.

Ishani: No, definitely not. But I think it really does take customer obsession to a whole other level — around being able to go in and understand and see how these data are collected and see how people are using them. And then how they’re able to use something that you can build in the duration of an internship, even to increase performance and double it in this case, then actually take that out into the real world and say, okay, that’s great validation. Now I have the comfort to go build a company around it. And the proof points, right? And then, by the way, you also start with a great contract.

Zayd: Yeah.

Ishani: One of the things you mentioned is this concept of building on existing infrastructure versus designing an entirely new user interface. And what I mean by that is you talk about integrating with all these systems that already exist in contact centers, right? They have agent-facing software. It seems like you’ve taken the approach of building on top and integrating with that rather than having contact center agents focus on learning something new. From a product perspective and from a utility perspective, tell us about that decision and how it’s worked out so far.

Zayd: Right. Our approach is to peacefully coexist with existing systems and infrastructure in the contact center. And that strategy really is just, there is a lot of different underlying systems and topographies in the contact center, a lot of different C cast platforms, a lot of different CRMs and knowledge bases of these kinds of things. Some folks take the approach of a full rip and replace where they recommend that they come in and replace your entire system. And our belief is that’s likely not the right approach. What we have is a modular approach where we can come in and integrate with your existing systems and take you to the future state of the contact center piece by piece by one module when you adopt another module will get more value from the second module because they amplify each other. But you don’t need to rip and replace your entire system to do it. It’s a lot of pain, a lot of implementations. You have to build more integrations. You have to spend more time on some of these edge cases and these kinds of things. It’s more effective and efficient to start with one piece and see the value from that and then expand over time.

Ishani: Especially when you think about going to the enterprise, right? It’s unlikely that you get a Fortune 100 company that’s going to rip and replace their entire existing system for you if you’re even a 50-person company. I really love how you lay that out and saying, “Hey, actually, this is worth a lot of investment from your end because there’s clear ROI.” And in your case, doubling performance and being able to demonstrate that feels like a very clear path to demonstrate for an enterprise buyer that there’s a specific return for buying Cresta.

You’ve alluded to this a couple of times in terms of different modules of the product, maybe just contextualize. Cresta started as this sort of augmentative tool for contact center agents. Where’s the product today? If I’m an end user at a big enterprise, how have I adopted it? And what exactly does Cresta do for me?

Zayd: Yeah, so we started as a real-time agent coaching and assist for chat — sort of, how do we help chat agents more effectively and more efficiently handle sales and support conversations? Over time, we built a full platform. It Really pieces together all the different components of what an intelligence layer and an intelligence suite on a contact center should do. The way Cresta works is we have an insights product that is understanding your conversations, understanding what makes things effective. Why are your top performers performing better? And what behaviors do they do that make them effective at that? And that’s giving you insights on how that’s changing over time. And then, we are able to take that and have a set of actions through real-time coaching and post-call coaching that help democratize those behaviors across your entire team. So, we figure out what makes your top performers really good. And then, we’re coaching everyone across the team to help them be as good as the best. And then, we take those top performers and we’re able to make them superhuman because we have stuff like automation and efficiencies like summarization and these features that drive these kinds of superhuman efficiencies of the team, because all of a sudden you can summarize a call, or you can automate a call or automate a workflow that takes less than a second when it used to take a long time. That’s the core loop of Cresta. And then, we go back to insights. We identify what’s working, what’s not working, how customer sentiment trends market’s changing. And then go back through the loop again in terms of democratizing the behaviors that lead to better results and driving automation that lead to superhuman efficiencies.

And it’s that virtuous cycle that goes to the contact center and it’s a platform that each piece can be adopted in a modular way. But then, as you adopt each piece, it adds value to the other piece. And when they put all the pieces together, it becomes a powerful engine that drives compounding benefit for the business.

Ishani: Absolutely. We would call that a flywheel effect. That the more you use Cresta, the better it gets. And in many ways, this is perfect for our core concept of what an intelligent application is. The idea that you’re building a core machine learning-based product that gets better the more that you use it. Versus, I think a lot of companies out there today are building AI and ML, but they’re kind of retrofitting to a product and using AI and ML to optimize it overall and over time. Whereas the concept of starting with a core machine learning-based and intelligence product that enables your users to get better, but then also actually becomes better as a product because of that feedback loop and iterative cycle, is really, I think — The next generation of companies are going to have to be built that way.

Zayd: It’s certainly not easy, but it’s something that compounds over time. Then it becomes more and more powerful for a company. And the folks that adopt it end up outcompeting the folks that don’t, so you’ll see teams and individuals that are empowered with these kinds of systems are going to produce dramatically more, and they’re going to outcompete teams that don’t, and over time, we’re just going to see this play out where teams that do this versus teams that don’t do this and it’s a competition that’s already getting started.

Ishani: That’s a good insight actually, Zayd — that not just is the next generation of companies, intelligent applications versus predictable SaaS applications, but rather, also customers that are using intelligent applications have a wedge and a differentiation component over their counterparts that are just using SaaS tools.

Zayd: Yeah, that’s what makes it exciting. I think it’s a fun time for the industry. It’s a fun time for technology. And it’s one of the things that just makes it fun to build at this time.

Ishani: Agreed. And let’s get a little bit more specific than just around how you build this intelligence layer. What’s the approach to data, for example? There are lots of customer conversations available to you, once you’re in a customer and you get maybe a transcript or set of data that they have on existing customer conversations. How do you start? How do you continuously train that model, learn from each of those conversations? And actually make sure you’re delivering an accurate, great result to the end customer and surfacing the right level of insights of what good looks like.

Zayd: One piece to that is basically we get the conversation transcripts, the audio, and then we tie specific outcomes to each conversation — was this a positive outcome in terms of, if it’s a sales use case, was it a close one or what was order value? What was the upsell rate? These kinds of things. And if it’s support, is it a first call resolution? What was the average channel time? Was it high CSAT, high transactional NPS. So, depending on the outcome the business is looking to optimize for, we’re looking at those outcomes and tying it to those transcripts. And then we’re training models. And so that then gets to one of the core differentiabilities of Cresta, which is that we’re building infrastructure to make it possible to train tens of thousands of custom models for many enterprise customers where our vision is Cresta provides the Costco hot dog, which is like a factory to build these custom models for many customers and the infrastructure internally in the tooling, internally around conversation designers and labelers and machine learning engineers that produce high-quality models trained for that customer and keep them up to date regularly. That’s a hard challenge, but that’s something that we are specifically investing in from a tooling perspective. And we’re seeing the results of that payout because we’re able to deliver a model that understands a customer’s conversations. So, we can get to the specificity of why a customer doesn’t or does buy from Verizon in terms of they’re having an effective conversation and up-sell. Versus something for a different customer. They’re able to train a specific model that helps that specific company become more effective in their sales conversations.

Ishani: Super interesting. How do you view the role of these increasingly new and increasingly powerful pre-trained models? When you go to do that. For example, I could imagine you taking a pre-trained model that’s a, maybe more rules-based type of engine when you go into a customer initially. And then, of course, creating all these derivative models on top of that, maybe for specific customers, but also maybe even for specific use cases within customers.

Zayd: So, pre-train models are powerful, and we’ve known that for a long time — more than probably I’d say a decade now. The thing, though, is that when you take a pre-train model and you’ll fine-tune it for a particular application, what ends up mattering more than the amount of data that you fine-tuned on is the quality of that data. So, are you training on high-quality examples? And are you training on high-quality labels that truly get to the root of what you’re trying to train the model on, and that’s where you need the right infrastructure and tooling to label and design effectively. And make sure you’re encoding best practices effectively into the model.

So, pre-train models give a boost across everything really, but by themselves, I think they’re useful as toys in which, and toys soon become very serious business applications, but to really leverage them as business applications, you need to have a set of infrastructure around it to really control and fine-tune it with specifics of what you’re trying to do. And that needs to be high-quality and effective data.

Ishani: Yeah, you’re giving an example to this concept of garbage and garbage out, right? That if you’re training your model on, in this case, Cresta’s model, on bad customer interactions, of course, that’s the guidance that Cresta is going to give you is, is bad. But if you figured out a way to scalably and effectively label customer conversations that are really good and train Cresta’s model on that, then Cresta gives you really good recommendations. So, it’s just a good way to then think about, okay, you have to have a data strategy and the infrastructure and tooling, as you say around it, that is really robust. Even if you already have the machine learning components, the data really does matter as a differentiator here.

Zayd: Yeah. That becomes fundamental to this Costco hot dog approach.

Ishani: I think if I take away nothing else, there will be two analogies here. One is to go intern at a company to get a customer. And two is create a Costco hot dog.

Zayd: Yeah. Our head of engineering — this gentleman named Ping Wu — he’s very passionate about Costco hot dogs and how their inflation uh, resistant. And so, that’s the vision for Cresta.

Ishani: Real-world analogies help a lot.

Zayd: Yes.

Ishani: Tell me on this tooling, an infrastructure component, what’s out there that you’ve been able to leverage in terms of software and structures that exist versus some of the things you’ve had to create that are big hurdles for you to get data to, the right insights for Cresta.

Zayd: I think right now we’re still very nascent in terms of the whole stack for data and machine learning operations. Because there are no best practices really established, and it’s tricky, right? Because it’s not as mature as other industries in terms of like, this is how you build this type of application. For us, we’ve built, I would say, almost everything in-house, just because of the specifics of our application and the relative latency of tooling and infrastructure in the ML space. That makes it an interesting problem. I think over time, it will settle, and the space will mature, but right now, there’s gaps in open-source tooling. I mean, there’s a lot of great stuff happening, but I think it’s just a while before that stuff matures to the level that we can standardize on it.

Ishani: It’s a little bit surprising that you say that. Lots of companies I know are building off some of these open-source projects, but as you say, there’s gaps. And so, is the gap in stitching it all together or is it just that they don’t serve enough function? They’re emerging and nascent and exciting, but they’re features and not platforms.

Zayd: Yeah. Our data model in terms of conversations and outcomes, and these kinds of things, let us build in specific ways. We leverage in our stack, everything from the cloud providers to Kafka and Kubernetes and all these things. But as part of our stack approaches the machine learning aspect to it, and the process of labeling and training and regression testing and automating machine learning delivery, and production, all those things. Those are things that we’ve built in-house. I think over time, we’ll see more and more of that probably get adopted across many companies, but that’s the state of where we haven’t quite found something that truly hits the nail in terms of what we need.

Ishani: That’s awesome. Maybe there’s more opportunity, right? For all the folks listening that work on MLops, that work on building the infrastructure to get to an endpoint iterative machine learning-enabled application, it isn’t quite there yet. The industry hasn’t converted on something, so it feels like a good opportunity to go and understand, and obsess about com companies like Cresta, right? If the next generation of end customers may also be intelligent applications, then we should go be building for those intelligent applications too.

Zayd: Yeah. I think the promise is there, and I think there’s a major opportunity here to standardize and build this part of the stack. I just think it takes a while to have code bases that become mature and solve the problem and the right approach to it. And so, whenever we’ve looked at these things, there’s always something missing. But I think the opportunity is there. There’s definitely a market opportunity there for sure.

Ishani: Well, and in the interim, you’ve been able to build the infrastructure internally and then also create this company that is an intelligent application. But then also part of the core of what you talk about is enabling humans to work alongside technology really well, harnessing the power of automation. It seems like a deliberate choice by Cresta. You know, people talk a lot about AI automating away jobs, but in this case, I think you view it as strongly humans and technology working together side by side. Where does that come from? How much of that is the idea that we’re not just quite there yet on technology and in terms of conversational intelligence side, and it is just not quite good enough. And how much of that is a deliberate design choice working with your end customer and those partners?

Zayd: I think it’s a fundamental principle all the way back to my Stanford thesis, which is the focus on intelligence augmentation and really the concept of the bicycle of the mind. And I think to some extent, there’s a way to look at artificial intelligence and I sometimes characterize it as lazy artificial intelligence, which is that you take an existing process and you say that I’m going to automate this process end to end, and I can use some kind of automation or some kind of AI to go do it. But that kind of overlooks, what’s possible when you approach it more creatively. And in that sense, you kind of look at AI as a building block — how does this capability combine with humans unlock things that just weren’t possible before? And that’s a fundamental approach that we took. It’s been our goal and our direction in terms of that, this is what we believe is the right approach to this and ultimately results in larger upside and larger potential. And especially in our application, there’s, yes, we’re not at human-level AI for conversations, but even then, there’s opportunities for humans to have a continuous and big impact in terms of the companies that they work with.

Ishani: If you play it forward 10 years, how much more or less are humans involved in the contact center process?

Zayd: Our vision is that you get to this point that you have this concept of experts on day one. So, folks come into an environment and within the first day, they gain this expertise of all business information and all the knowledge and subtleties of their particular environment and are able to use a whole set of support and decisioning systems to get to the right decisions and the right effectiveness in their role. And what they’re bringing to the table is creativity in terms of understanding how to approach something that isn’t encoded in the patterns of the data yet. They’re bringing that creativity to the table, and they have a whole support decisioning system that’s helping them be that expert on the first day. And as soon as they’re able to encode and establish a best practice through their creativity, that becomes a part of the system again. And then the human is just focused on the next thing and the next thing. And so, our approach with these kinds of augmentation systems is that we’re constantly figuring out what’s the best practice, what’s really working, what’s effective, building support systems and information systems that can help deliver that scale to many people, and then figure out how could the humans continuously act as in some ways like a mutation process, that’s identifying what’s a new thing and what’s the creative approach to this problem. And then keep doing that and you just build a system that just gets smarter and smarter.

Ishani: A bit of a separate question. If I look at the investors around the table of Cresta, you’ve got folks like Greylock and Tiger Global that more traditional institutional venture investors. All makes good sense. You also have a coalition of investors from kind of legacy industry players. How has that experience played out? You know, having strategic involved can always be a little bit of a double edge sword in terms of having folks around the table. Tell us about your experience.

Zayd: We’re really fortunate to work with great partners from an investor perspective. So, in this last round, Genesys, Five9, and Zoom invested in Cresta. We have great partnerships with those folks in terms of leveraging Cresta on top of the platform to really see a big impact for their businesses. We’re seeing go-to-market acceleration through those partnerships, and what that really has done is mark our leadership in this space in terms of this real-time intelligence for the contact center.

Ishani: Super exciting. Yeah, I think investors can represent impactful connections, powerful networks that enable you to, again, just build more of a flywheel. Right? So, I love that concept of solidifying you as a winner in this space.

We typically end these podcasts with a lightning round of three questions. So, I’m going to shift into that. First, aside from your own, what startup or company are you most excited about in the intelligent application space and why?

Zayd: Oh, that’s a good question. I think Tesla is doing some very interesting things overall in terms of how they’re approaching autopilot systems. I’m not sure if it counts, but I think that vision is something that will take that company far, I believe, in terms of the way they have this loop for model predictions and data collection. I haven’t quite seen other startups operate at that level of data flywheel. And I think that’s the right approach.

Ishani: Love it. Question two, outside of enabling and applying AI to solve real-world challenges, what do you believe will be the greatest source of technological disruption over the next five years?

Zayd: Some of these contact centers are still working on these green screens with these old ’80s-style computers. And it feels surreal, but the cloud hasn’t actually fully happened yet. And a lot of companies are working in all kinds of different environments. I think that just better systems, better cloud systems, better UX, better integrations — that stuff has a big impact. We see with our customers as well that the AI has a lot of value, but they also get a lot of value from just better data integration and better UX to do their day-to-day workflow.

Ishani: Yep. The concept of customer obsession going and being on-site and visiting your customers and being integrated in them, even at the intern level, really gives you a real-world perspective for how true that is and how much the install base of technology takes a while. And a couple of cycles to come up to where we think about it, whether it’s in an academic lab or as a startup CEO, or as an investor,

Zayd: Right. Agreed.

Ishani: Final question. What is the most important lesson — you know, maybe something you wish you did better — that you’ve learned over your startup journey so far?

Zayd: So, I asked Scott Cook this question and I think the biggest one is probably self-awareness. And I think if you unlock self-awareness, then a lot of other things can happen in terms of development as a leader and development just as a company. I think understanding your own weaknesses, understanding what you need to get better at, and then approaching those things with the growth mindset, that becomes really fundamental. It sounds a little fluffy or psychobabbley, but I think it’s true.

Ishani: Awesome. Zayd, thank you for talking about Costco hot dogs, interning at your company, growth mindset, and the next generation of machine learning ops. Super appreciate having you on the podcast and your time today.

Zayd: Awesome. Thanks so much, Ishani.

Coral: Thank you for joining us for this IA40 spotlight episode of Founded and Funded. If you’d like to learn more about Cresta, they can be found at Cresta.com — that is C-R-E-S-T-A.com. To learn more about the IA40, please visit IA40.com. Thanks again for joining us, and tune in in a couple of weeks for our next episode of Founded and Funded with Tesorio Founders Carlos Vega and Fabio Fleitas.

In this episode of Founded and Funded, we spotlight Intelligent Application 40 winner Snorkel AI. Managing Director Tim Porter not only talks with Snorkel Co-founder and CEO Alex Ratner all about data-centric AI and programmatic data labeling and development, but they also dive into the importance of culture — especially now — and how to take advantage of what Alex calls “one of the most historic opportunities for growth in AI.”

This transcript was automatically generated and edited for clarity.

Coral: Welcome to Founded and Funded. This is Coral Garnick Ducken, Digital Editor here at Madrona Venture Group. Today, Managing Director Tim Porter talks to Snorkel Co-founder and CEO Alex Ratner all about data-centric AI and programmatic data labeling — the two core hypotheses Snorkel was founded around. The research behind Snorkel started out as what Alex calls an “afternoon project” in 2015, but it quickly became so much more than that and officially spun out of the lab in 2019. Since then, the company has raised a total of $135 million to continue its focus on easing the burden required to label and manage the data necessary for AI and ML models to work and to extend its Snorkel Flow platform into an entire data-centric programmatic workflow for enterprises. Machine learning models have never been so powerful, automated or accessible as they are today. But we are still in the early innings of what it can do. IA40 companies are solving issues across the AI ML stack, but it all starts with clean data. Snorkel has built an incredible platform that taps into human knowledge and dramatically speeds up the data labeling that is necessary for the rest of the pipeline to work. Alex says that even the largest organizations in the world are blocked from using AI when it takes someone or a team of someone’s months of manual effort to label data every time a model needs to be built or updated. But that’s where Snorkel and its Snorkel Flow platform comes in. It should be no surprise that Snorkel was one of our 2021 intelligent application 40 winners, so I’ll go ahead and hand it over to Tim to dive into all of this and so much more with Alex. Take it away, guys.

Tim: Well, it is a real pleasure to be here with Alex Ratner professor of computer science at the Paul G Allen School of Computer Science and Engineering at the University of Washington, but even more relevantly the Co-founder and CEO of Snorkel AI. Congratulations on being recognized as one of the top 40 most innovative and high potential companies and ML/AI space broadly as voted by a large panel of VCs that are active in the space. None of whom could vote for their own portfolio companies. So, congratulations on that, Alex, and thank you so much for being here today.

Alex: Tim, thank you so much for having me and we’re obviously incredibly excited and more importantly, humbled by the honor. And I’ll note I’m not the professor yet. I’m an assistant professor, so it’s still a ways to go. But obviously, I’m very excited about the work that goes into both that on the academic side and Snorkel the company around what we call data-centric AI.

Tim: A VC once again, slightly over-promoting. I apologize for misspeaking on the title. But being based in Seattle and having a lot of connections with the University of Washington, we’re thrilled that, you know, over time that’ll be a home for you and you’re already doing a lot of things to impact the school there. But let’s talk about Snorkel. It’s been a company that I’ve followed for a long time. You and I have known each other for a number of years, Alex. But maybe we could start out by just telling our audience what exactly is Snorkel AI and what problems are you solving for customers?

Alex: We developed a platform called Snorkel Flow and it’s one of the first — we call it a data-centric and programmatic development platform for AI. I think a lot of people know that AI today involves data, and it has centrally for quite some time. But what we do really is support this new reality that a lot of the success or failure in building and deploying AI applications has to do with the data they learned from. And not just any data, but carefully labeled what’s called training data, to teach them to do something.

So, we work with all kinds of customers — the top five U.S. banks processing everything from loan documents to customer complaints and conversations in their chatbots all the way to medical images, network data, all sorts of stuff where basically users are trying to build machine learning models or AI applications that learn to predict or label something. And they do this by training on or learning from tons of data that’s been labeled with kind of the correct answer. If you look back 5+ years to when we started the project originally at the Stanford AI Lab, if you went onto the field and asked the practitioner, what are you spending your time on? What are you throwing team hours at? It would be all about the machine learning model or the AI application — building some bespoke machine learning model architecture to handle chest x-rays or loan documents or conversational intents. And the data was an afterthought, or it was something that someone else prepared and labeled and did. You downloaded it from something like Kaggle or Image Net, and then you started your machine learning. This is what we now call model-centric development, where the data is something exogenous to the process that happens before from someone else. And you’re just iterating on your model. Fast forward a couple of very exciting years in the ML/AI space — a lot of that model development is now for a staggeringly large range of problems, almost push-button — a couple of lines of code thanks to some of the great companies and vendors and open-source contributions out there. And also, more powerful and more automated than ever before. But there’s always a trade-off. And the trade-off is that these new approaches are much more data hungry. So, the buck has shifted from model development to data labeling and development. And so, what Snorkel does is it tries to automate that data labeling development process- make it more programmatic, like software development — writing code, pushing buttons to label and develop your data — and solve that thing that’s often the bottleneck in AI today. This is complementary to the model development, which is often much more push button or, a line or two of open-source code. And this is based on techniques for this kind of data-centric programmatic development that we’ve developed over the last six and a half, seven years at, places including now UW, but also originally back to the Stanford AI Lab and co-developed and deployed with, lots of different tech companies, government agencies healthcare systems, et cetera.

Tim: This move from model-centric to data-centric — I really like how you frame that. We see that across companies and tying it back even to this notion of intelligent applications that data is really remaking how all applications are made, bringing new data to bear and providing new insights. I think most people also realize that ML is only as good as the training data that you bring to the problem. And this really helps speed and improve that. How the heck do you do it, though? It sounds a little bit like magic, so instead of having a person or a subject matter expert, being able to label this data, you’re able to do it with code. Two pretty hot areas that are talked about in the field — weak supervision and generative models — are two important building blocks on how you make that happen. Maybe you could spend a minute just explaining a little bit about the nuts and bolts and how you do this thing called programmatic labeling.

Alex: Yeah. So, I’ll start with the first thing that you said about magic. That’s one of the things I like to anchor on first in demos and customer presentations is that this is decidedly not aiming to be push-button auto magic. It’s still a human loop process, but it’s one that we aim to make look more like software development than just clicking one data point at a time to label it. Imagine, you’re trying to train a model to triage a customer complaint at a bank and maybe flag it as urgent or not urgent, or maybe, flag it with a specific regulation that it should be reviewed against. The traditional legacy approach that a lot of machine learning progress is based on is you’d have a bunch of people sit down and click through customer complaints, one at a time with the correct label. And that’s what your model would learn from. And in some ways, you know, what we’ve been working on is as much about inefficiencies or gross inefficiencies in that process, as it is about clever, algorithmic, theoretical and systems work that we do.

One way to think of it from that perspective is, if you have a subject matter expert sitting there who knows about all of these regulations and is reading these customer complaints, they probably know certain things they’re looking for. They have a bunch of domain expertise. They’re looking for certain phrases, certain keywords, certain metadata patterns, etc. Why can’t you just have them tell that to the model? In Snorkel Flow, they do exactly that. And some of it’s through no-code UI techniques, some of it’s through very heavily, auto-suggested, auto-generated techniques where they can even explain something they’re looking for and automatically label data that way. So, lots of acceleration automation, but at the core, it’s a domain expert using domain knowledge, heuristics, and programs to label the data, versus clicking one data point at a time.

We have customers who will take six months of manual labeling and collapse it into a couple of hours of this kind of programmatic labeling and development. The magnitude of the problem here is that a lot of these projects just don’t get tackled, especially take that example of customer complaints. You have data that’s very private, data that requires specific expertise, and data that’s always changing as input data changes and regulations change. We had a series of papers with Google and YouTube on how they were throwing away hundreds of thousands of labels a week before they started deploying Snorkel tech because even for a company with those kinds of resources, wasn’t scalable to label and re-label, every time something changed. So, we’re used to a lot of the kind of tip of the iceberg problems where we’re using machine learning to solve very standard problems. Is it a cat or a dog or a stop sign, a pedestrian, a positive or negative restaurant review? But there’s this whole iceberg under the surface of enterprise and organizational problems that are much, much more difficult to label at scale in this old way.

Tim: It’s fascinating, and just to double click on this point that it’s not magic, and you still need humans and subject matter experts, is that you want to start out with a set of ground truth labels that maybe comes from a human, but then you can use your technology to extrapolate from that to a much larger set of data, much, much more rapidly. I think it’s also clever how you use other organizational signals to try to come up with those labels in an accurate way.

Alex: Yeah. I like the parallel to software development. You don’t write down a bunch of zeros and ones every single time you want to compile a new program and you reuse assets, you use a higher level of abstraction, this higher-level knowledge. And similarly, that’s what we’re trying to do here. A lot of times, people try to apply AI in some enterprise setting. And the recommendation they get is — okay, great, you built all these things before, you have knowledge bases, you have legacy rules or heuristics. You have experts internally who have all this rich knowledge. You have models, you have all this other stuff — throw it out and start labeling data from scratch. Every single time you want to train a new model on a new setting. And what we’re saying instead is no use all that information. Use those organizational resources, whether it’s in a subject matter expert’s head, an underwriter, a legal analyst, the government analyst, a network technician, a clinician, and use other models, heuristics, and legacy systems to teach or bootstrap your machine learning model. And the cool thing is you can actually do this without any ground truth labeled data, although it’s often helpful to have a little bit of that.

You’d asked about weak supervision and generative models. A lot of the original theoretical work was diving into the algorithmic and theoretical aspects of this problem of, okay. If we shift the paradigm from labeling data points one by one, and assuming that they’re perfectly accurate, that they’re ground truth, which is, by the way, a very faulty assumption already, but, if you shift it to now having some programs, we call them labeling functions, that are radically more efficient, more auditable, more reusable, more, more adaptable, but also going to be messier and their heuristics. They’re not going to be perfectly accurate. They’re not going to perfectly cover all the diversity of data out there. They might conflict with each other and have all kinds of other messy aspects. How do you, ideally with formal guarantees, clean and de-noise and integrate those into a training set you can use. And that’s in fact, what we’ve spent half a decade working on about theoretically grounded techniques for using generative models and other approaches to figure out which of these labeling functions to up weight, down weight, how to de-noise and clean them. So, you can take this much more efficient, direct, but somewhat messier input and use it to train high-performance machine learning models.

Tim: Are there certain classes of problems or certain types of data or applications that this is best fit for?

Alex: So, we’ve applied it to everything from, self-driving to genomics to machine-reading and beyond, but I think some rules of thumb in terms of our inbound filtering, our outbound targeting is around where we think it will provide the biggest delta above other approaches.

One, first of all, is we handle structured data, especially messy structured data, but we also have a lot of focus on unstructured data. You hear about all these advances in machine learning, AI, all these new deep-learning or representation-learning models that are super powerful, but super data hungry. A lot of them provide the biggest deltas on unstructured data — think text, image, video, network data, data, PDF, website, etc. All this very messy long-tail data. Bigger data, more data-hungry models, and more need for data-centric AI development. That’s often where we sit.

Another rule of thumb is how expensive and difficult it is to actually label and relabel and maintain these big training datasets. If you’re talking about something that can be, a stop sign versus a pedestrian. Stop signs don’t change much, so you just leave it and maybe you can get by with a legacy manual approach there. But if you look at that iceberg under the surface of problems we tackle, think about very private data, very expertise, intensive data- financial insurance, medical government and most industries, honestly, anything with a user data network or technology and also settings where your data is changing, and your objectives are changing. So, you have to be constantly relabeling. So, when you have these aspects, suddenly the cost of just throwing people at the problem to kind of click, click, click, you know, a week or a month or longer at a time per model becomes infeasible literally for the world’s most well-resourced ML teams. And that’s where we like to step in to ease that bottleneck.

Tim: You mentioned moving towards even more data, hungry, deep learning, large models, you know, to extend that thought, these, very large-scale transformer models, foundational models that have gotten a lot of coverage. And then the ability that they provide to maybe do, one-shot or no-shot learning on certain problems or to refine them or train them with a smaller set of data for your specific use case. Do you see that as an overall large trend? And does that create more need for programmatic labeling?

Alex: I think it’s an extremely exciting trend, although it definitely will take a while to percolate into the enterprise for reasons we can go into about, everything from, efficient deployment to governance and auditability. But just talking about the tech trend for a second, it’s something that we’re very excited about. We had a recent webinar and a paper from my co-founder Chris’s lab at Stanford on combining these foundation or large language models with weaker programmatic supervision. And then we had another paper we just posted about using zero-shot learning on top of the large language models to automate some of these data-centric labeling and development techniques. So, we’re very excited about a whole host of complementary intersection points. And we already support basic pre-trained models of this class and these large language models in our deployed platform. In a nutshell, how I paint it is that these foundation models and a lot of us are not calling them foundation models because they serve as a great foundation for building or training or fine-tuning custom models or applications on top of them.

They do still fall into the general body of transfer learning techniques. And I’ll stick to very basic and old intuition that, you get what you train on, right? So, these large language models, let’s say they’re trained on something like web data when you actually want to use them to handle geological mining reports or clinical trial documents, or, loan documents, or, you know, the list goes on, they don’t just magically work out of the box, the zero or a few shot techniques don’t just suddenly solve the problem. You still need to label a bunch of data for the specific data and task at hand to get utility out of them. So, they’re very nicely complementary. But there’s a lot of very cool, but somewhat cherry-picked examples of what they do out there. They still need a lot of additional work to get them to be production level for enterprise use cases. So, there’s a nice complement there we’re very excited about.

Tim: You get what you train on — it continues to be a truism. Hey, you mentioned you know, the Stanford AI lab and your partnership with your Co-founder Chris Ray, maybe take us back to the founding story here, Alex. You put a lot of time into this before launching a company. You know, I feel like over the last few years there’s been this rush to start the company. Venture dollars are plentiful, so get while the getting’s good and you definitely took a longer path to get it to the point of saying, “Hey, we’re ready to commercialize this.”

Alex: Well, it all started with a massive con from my advisor and Co-founder Chris — he suggested this as an “afternoon project.”

Tim: I know there’s some lore about this started as like some math on a whiteboard, right?

Alex: Yeah, it was math on a whiteboard and then there was a Jupiter notebook. and we were, uh, teaching an intro course and we had just refactored it over the summer to center around Jupiter Notebooks, which was a really cool idea that also led to one of the most traumatic office hour sessions I had ever, where everyone in some massive intro course at Stanford came asking how they could install Jupiter Notebooks on every device imaginable. You know, My Tesla screen doesn’t support Jupiter notebooks, please help me ASAP — a five-alarm fire drill. I still remember that. It was a confluence of trends. One trend that was coming in was we had been working on all these systems for things that were more in the model-centric world. So feature engineering, model development, joint inference at scale, all these things that are super cool, but we were seeing this trend, kind of hitting us in the face from our users, who were, you know, biomedical data, scientists, geologists, all kinds of data scientists saying, “Hey, this is all great, but we’re starting to use these deep learning models and these other models and really our pain point is labeling the data. So could you help us with that?” Like everyone else back then — this is 2015, we said,” that’s not our problem, that’s someone else’s problem. We do machine learning. So. We’ll keep helping you with the fancy models.” Eventually, after being smacked in the face of this enough times, we said, “Hey there’s actually something here”. Everyone is getting stuck on the data and the data labeling and the data curation. Maybe we should look at that. So that was step one. Then, as we started thinking about more clever ways and looking at old techniques that did some of this and what our users were doing. Some of our users were getting very creative hacking together, ways to heuristically ad hoc labeled data. We said, okay, first of all, this is so painful that people are doing ungodly contortions just to hack together training sets. So, we said, “Okay, we’ve got to shift to this data-centric, versus model-centric realm. Because there’s something here.”

And then number two, we started asking, okay, how can we support this a bit more? So, we had this idea that we would come up with this kind of Jupiter Notebook where domain experts could quickly dump in some heuristics of how they were labeling data. And we would try to turn that into labeled training data. And that was the “afternoon project” that then spiraled wildly out of control because it just led to all these interesting problems of okay, well, how do you solicit information from the subject matter expert? How do you then clean it? Because even a subject matter expert is still going to give you rules of thumb that are only somewhat accurate. So how do we clean that and model it — that’s this weak supervision idea so that it’s clean enough to train a model? Then how do we build this broader iterative development loop that involves this kind of programming of data and then training models, and then, getting feedback on where to develop and debug next. That was how it all spun up. And then to your question of why were we so slow? Which is a fair one.

Tim: It’s a hard problem.

Alex: Yeah well, and honestly, I mean, we were and are very invested in this problem and in the kind of pathway that we’ve been charting with this you know, data-centric direction. And we’re always anchored on where’s the best place for us specifically to you know, center this effort. And for many years, I’m super biased, but we couldn’t ask for a better place than academia and the purview we had at Stanford. We had office hours weekly. We had everyone from major consulting companies to bioinformaticians to legal scholars coming by and trying to use these techniques. We had purview to work on getting the core theory and ideas. And we started to put some ideas out there, some code out there, we started to get some pull, started to get a bunch of people who were, trying to get me to drop out of the Ph.D. program with a pre-seed or seed round, or I had no idea what the terms meant back then, so it was all nonsense to me. But we thought that we had core problems to work out.

And then, four and a half years in, we started looking at what our users were telling us. And they were telling us things like, “Hey, maybe instead of, working on another theorem, which is cool and all you could help us solve the UI problems or the platform problems or the data management problems or the deployment problems or the feedback and error analysis guidance problems.” When that started happening, you know, we started poking our heads up and, decided, hey, it’s entered the next phase. We’re actually moving to a vehicle where we can put together a different set of people with different sets of skill sets to really build a product and a platform and engage more deeply with customers here. That was the next phase. So that was when we finally spun out.

Tim: I love that you were pulled by customers and customer-centric and making those decisions. It seems like you nailed the timing for when the market was ready and started to need these solutions on a bigger scale. But there’s another piece that you just hit on that I wanted to ask you more about, you know, we’ve talked a lot about the labeling aspect and that’s certainly the core of the solution that you provide. Snorkel Flow is a broader framework. Maybe talk a little bit about how that whole loop is important for Snorkel Flow.

Alex: Kind of our whole point, both research and product has always been that it can’t just be about labeling. When you think of labeling as this separate step in a vacuum, that’s where you get these very unscalable and impractical model-centric only setups.

The idea of data-centric development is that labeling and developing your data — so not just labeling, but sampling, slicing, augmenting all these things that people do as modern data operations. This is your primary development tool — not just to get data ready for models, but to adapt and improve models over time. And so, you have to have that whole loop otherwise you’re flying blind and you’re not really completing this kind of idea of data-centric guided development. So, our platform today starts with looking at your data sampling, labeling more broadly, developing, slicing, augmenting, etc. But then includes a full auto ML suite. Mostly just to give very rapid feedback. Where am I successfully training a model and where do I need to go next to continue this data-centric development? And then, you can export the model from our platform. We actually support broader multimodal applications. You can, if you want, just pull out the training data and train your own external models, we’re very open, but the core workflow has to include a model in the loop. And it’s more feasible than ever before to do that. Given all the kind of great modeling technology that’s out there in the open source these days.

Tim: One interesting observation about this space called MLOps now is I feel like, and sometimes joke, that companies that start out providing one important piece of functionality across this pipeline, for lack of a better term, whether it’s labeling or a feature store or deployment, you know, want to be end-to-end, and I think you just gave some good reasons why in this data-centric world, you need to be able to close the loop from watching how an application or a model is performing and tie that all the way back to iterating, to what’s happening with your label data. So that’s a good reason. But it also seems there’s a little bit of just, you know, startup imperialism that you want to be end-to-end and provide all these pieces.

On the other hand, I think you talk about plugging in other frameworks, other deployment mechanisms, other infrastructure management, it seems like you give customers the choice of like, “Hey, you can use Snorkel end-to-end, or plugin you’re best-of-breed for different pieces.” Is that the way you talk to customers about it? And is there a common way that customers tend to engage or is it really across the board?

Alex: So, and maybe three comments that are Snorkel specific, then I want to go back to that awesome phrase of startup imperialism. So, for us, first of all, there’s just a core definition of what we’ve been working on very publicly for over half a decade is this idea of data-centric development, which involves labeling. And that’s one of the several key interfaces, it’s one of the ways you can program your model. But it’s part of this broader loop that involves a set of, development, activities, and feedback from models, as you said. So that’s part of what we’ve always been, supporting and aiming to support. A second thing that’s specific to us is that we’re often approaching as I was talking about before a lot of zero-to-one type settings where you didn’t have a very sophisticated high therapist modeling stack because you were blocked on the data. You’re not predicting say customer churn where you already have the labels and you’re predicting column 20 from columns 1 through 19, you have just a pile of documents or a pile of chest x-rays or a pile of network flows, and, because it’s the zero to one state, there’s more of a pull often to just actually get to an end-to-end solution when you do go from zero to one.

And then the third point is just to touch on what you’ve mentioned, which is giving customers optionality. Our goal is to support a workflow that we’ve been working to define over the last six or seven years. But how you integrate different pieces into that workflow is something that we’re extremely open to. We have a Python SDK kind of one-to-one map throughout the whole process to make that really easy. And I think that’s critical if you want to play in this space. I think, on the one hand, I’m super biased, I think the most exciting technologies and projects will have an opinion on a workflow that is more expansive than just one little layer. But I think that workflow has to integrate with just the space that’s out there.

It’s an interesting question about startup imperialism and starting off with kind of one slice and then moving toward end-to-end. I think for a lot of folks in the space, there is also just a lot more pull to fill gaps than people may realize. I think if you just skim blog posts and academic papers, you would get a vastly different sense of AI maturity in the enterprise and the market than is actually the case. So, I think, people think we have this very complex, blog-post-defined-stack and every enterprise, but because of these problems of data is one of them and others around deployment and risk management, etc., we’re a lot earlier I think than many others do. And so often companies get pulled because there’s actually a bigger gap to fill them than people realize.

Tim: That’s a great point. And I want to talk more about that. So, you start helping a customer with problem X, maybe it’s, you know, the labeling issue here, and they’re actually asking you like, “Hey, we’re using you for this. We don’t have good solutions for these other pieces. Now help us deploy, now help us monitor. Okay. Now help us close the loop. But that’s a customer pull piece more than it is a high-level architecture strategy decision.

Alex: Yeah. And there’s a lot more of this pull in enterprise AI than people realize because there’s a lot less maturity than people realize just because there’s just so much to do. I think that one of the big challenges from a design perspective is where you draw the line so that you can really focus on what you’re uniquely best at. And we try our best to navigate that. We expand to cover this data-centric loop. We often push customers off and try to help them with reference architectures or connectors for pieces that we don’t think we have a special sauce around or that we shouldn’t spread into.

Tim: So, on this level of enterprise maturity, we have a thesis that we’re really at the beginning of a major wave of ML in production. Over these last several years, we’re kind of coming out of a period of intense experimentation at enterprises. Lots of innovation groups working on ML, working on models, seeing where they can build insights, and trying to get their data pipelines together. Cutting-edge companies certainly have been doing ML for years, but in those sophisticated examples, maybe there’s been an exponential increase in the number of models in production. Not that they are just getting them in production, but kind of the net effect is we’re at the beginnings of a pretty big bow wave of ML in production, both for internal applications, as well as the external applications that might be your company’s products.

So, is that what you’re seeing? Like where are we in terms of the innings here?

Alex: I think we’re early innings and I think it’s exciting because I don’t believe we’re early innings because of the lack of extreme concentrations of talent in the enterprise. There are historic levels of access to a lot of the core machine learning techniques, like the models, out there often in the open source than ever before. And so, you’ve got all the right ingredients, money has been put down, and they’re extremely talented data science and AI/ML engineering teams. You’ve got a flood of open-source tooling, especially around the models in the market, but you still have these significant blockers and headwinds that I think enterprises are really just starting to solve. Obviously, the one that we’re anchored around is the data. So, I think for that reason and everything else, that enterprises are very reasonably and responsibly trying to approach carefully, governance, auditability and interpretability, risk management, and deployment, we’re still in the early innings. And you see this kind of shift from the science project phase to the real production phase happening. And it’s a really exciting time to be in this space.

Tim: What industry verticals are you having the most success or focusing on the most and does that sort of map to this maturity that you’re talking about?

Alex: Our technology and our platform support a very broad set of problems. If you look at our publications and literature doing anything from, you know, self-driving to genomics to machine reading to many other things, but we focus on templatizing around certain core, very horizontal applications. And then today we work with very, highly sophisticated data science teams, and often with the kind of subject matter experts that have the domain knowledge about the problem in large enterprises across all sorts of verticals. We have a lot of customers — top five, top 10 U.S. banks and others in finance, insurance, biotech and pharma, healthcare, telecom, the government side and a range of others. So, it’s really these cross-cutting applications, things like dealing with unstructured data and, classifying, extracting, and performing other modeling tasks over them that we then templatize and target per vertical, where there are these great highly sophisticated data science teams that are blocked on the data.

Tim: I’ve used the term MLOps a few times in this conversation to describe the space. And I noticed you have not. I wonder if you like that categorization. We had another recent podcast in this series recently where Clem from Hugging Face and Luis from Octo ML hypothesize that in a few years, there will be no such thing as MLOps, it’s just DevOps and the problems that you have around machine learning deployment and management will be the same as any other application.

Do you like this category name MLOps, and do you think it has a future as its own thing or does it all converge?

Alex: I would’ve liked to be in that room for that debate. I don’t know if I’ll do justice to what that discussion covered. Okay well, why, why haven’t I used MLOpps? I think it’s growing to become a very expansive term. And so, I don’t have anything against it. I try to keep what we do a little bit more curtailed. I think there are many ways in which MLOps will remain its own thing and should. I mean, there’s a big difference between, code that is directly defined versus essentially code or programs that derive from, large statistical aggregates over, massive data sets that is just, fundamentally different in terms of how you build them, how you audit them, how you govern them, how you think about them.

Even the academic methods are very different — think more like formal analysis versus, something closer to statistical physics types of analysis. So, I think there have to be parts that are different, but I think also at the same time, there has to be many ways in which MLOps becomes closer to traditional DevOps and traditional software development. Obviously, that’s part of what we’re trying to do with data. We’re not going to get rid of all of the messy, unique properties of large data sets, but being able to at least treat the way they’re labeled and managed as more of a code asset and take a more DevOps stance versus this kind of manual activity. So, I guess in summary, I’m a big believer in pushing MLOps closer to DevOps and we’re, in some sense doing that at Snorkel, but I also think there are going to remain some aspects that just have to be unique and different even as they will get more standardized, commoditized and drift closer to DevOps as they have to.

Tim: Great points. Great framing. I completely agree with that. Let me switch gears a little bit. I was rereading your website, Alex, and I’m on the About Us page, you had your obligatory description of what the company does and then a rooted in research, point — we covered that and the cool beginnings from Stanford. And then I was struck, the next big part was about culture. How would you describe the culture at Snorkel? Are you a completely distributed company at this point? How have you continued to build the culture over these last few years, which have been tumultuous to say the least with everything going on in the world?

Alex: So, by culture, do you mean what code lender do we use? What’s our favorite sock emoji pack.

Tim: Um, among other things, yes.

Alex: I’m kidding. It’s obviously one of the, or the most important, questions, even divorced of the very unique situation we’ve been in over the last couple of years. It’ll sound somewhat vague and cheesy but, one of the most important things that starts at how we try to recruit and then goes into what we try to enforce a normalizes, is this, idea that can have extremely kind empathetic, friendly people who are also, very hard-charging and type A and obsessive about what they build and do, and that you don’t need to have one or the other. I think you can find people to work with in any context who are very fun and very kind but maybe won’t push as aggressively as you need to in the startup world. Finding people who can do both is the special thing. We always try to look for that intersection.

Of course, there are other extremely important things about building an inclusive, constructive, and positive environment. A lot of it is, again, back to cheesy comments, but about the balance of trying to always be extremely positive and supportive, but also, normalizing criticism and editorial input as much as possible as a positive, not a negative.

Tim: Are you fully distributed at this point? Is there an office-centric part of this? I’m sure everyone’s hybrid to some degree — how does Snorkel work?

Alex: Yeah. So, we just soft reopened the Redwood City office. So, for parts of our team where there, we have some parts of our go-to-market team in New York and distributed. We’re trying to navigate that in a way that’s responsive to what people want to do. We do plan to have some hybrid component and some in-person component. This is kind of an amateur hypothesis, but just from observations the last couple of years, I think you can do a really good job, and in some ways an even more efficient job of maintaining one-on-one relationships, small pods over virtual. But you face headwinds for cross-functional interactions and the broader social fabric. It’s really hard to schedule a five-minute Zoom meeting on someone’s calendar for like a bump into each other at the water cooler or walk by your office and overhear there’s a good essay that gets passed around for a lot of uh, uh, kind of intro grad students. When you start a Ph.D. program called, “You and your Research.” There was one statement I remember thereof saying that the people who always left their doors closed seemed to be much more efficient, but never really got anything done. So, I think there’s some aspect of that of you can be much more efficient with just everything back-to-back Zoom calls, and we want to keep some aspects of that, but also you lose some of that aspect of creativity, cross-functional interaction, and of course, social interactions. So, we’re going to try our best to navigate a path where we can capture the best of both. And that will be some form of hybrid that we’re still figuring out with our team.

Tim: Makes sense. And by the way, we love cheesy comments. I think some of those that might seem cheesy are the things that stick with people. Is there one ritual that you’ve established over the last few years that just works well for Snorkel that’s worth sharing?

Alex: We started doing these things we call “Whatever you Want”— “WW” at the beginning of all hands. We used to do it more than weekly at the beginning of the pandemic, but we do it weekly now, it’s just a retitling of “Show and Tell,” but it’s a couple of slides about any topic you want. And so, it’s a nice way to get to meet people who you’re not getting to bump into in the hallway and hear a little bit about some aspect of their life — a hobby, where they’re from a recent trip they went on. We did a series on failed past startups. So just little snippets and it adds a little bit more of the other dimensions to people beyond the purely kind of professional interaction. So that’s one thing that we’ve liked.

Tim: In this world of hybrid or remote, using the All Hands effectively, I think becomes really, really important. I did a panel at our CFO conference here with three chief people officers and the chief people officer from SeekOut had a different, but somewhat similar answer to what you just said. They said at their All Hands, they always kick it off with an opportunity for people to celebrate each other, which is something you said was core to your culture too, is to be, celebratory of each other, but still hard-charging. I think those little rituals mean a ton, especially in this world that we’ve been living in.

Alex: Puns are very important also. The things I’m most excited about as I recently had a second child, and I was informed by the team that I’m allowed to make two dad jokes per day now. So that’s been double the fun.

Tim: I have two kids also, and dad jokes and bad puns are right down my alley. So, there’s a lot happening in the technology markets and the public markets have corrected or repriced. You raised this awesome $85 million round last August. That was great timing. I’m sure you have a lot of cash in the bank. Your business also clearly is going well. What is your posture that you and your management team and board are talking about? Is it sort of, let’s keep accelerating here as fast as we can bear? Is there a little bit of a, ‘Hey things are good now, but we’re not sure’ in coming quarters? So maybe we don’t want to hire quite as quickly as we originally planned?” Or what’s your posture between sort of the gas pedal and the brake here as we go into the back half of this year. I know that’s a, no one knows no one has a crystal ball, but that’s a top conversation with all the companies that I’m working with.

Alex: It’s certainly uh, an interesting time. Seeing some of it as a return to sanity is obviously, I think, a positive for the space. Those of us who work in an AI, especially are always wary of over-hype leading to winters. I think for us, in particular, as you mentioned, we had recently raised a round. I think once you raise a bunch of cash in succession, you can either kind of go off the deep end or you can kind of instill good cultural habits and practices and grow up a little bit as a company. So, we were always planning to do the ladder and grow up a little bit. Obviously, the most important thing is being responsive to our customers, and we see just the same level of demand and, even more so for a lot of the projects that we try to anchor around with customers that are about increasing efficiencies and adding massive business value. And so, we’re still charging ahead at full speed. But we do think it’s a good reminder to be mature as a company and value efficiency and have that kind of culture and cadence. And I think it’s also a good reminder for the AI space to really, again, this is a little biased because we’ve been trying to do this from the beginning but focus on the business value rather than the science projects. We spend a lot of effort in our product building our go-to-market motion, trying to align with those teams and projects and budgets that are going to deliver a meaningful impact that’s robust. And so, I think it’s a good validation of that approach.

Tim: Very wise and very consistent with what we’re trying to counsel our company — don’t stop being aggressive, but efficiency ultimately also matters. And really inspect you know, new investments that you’re making because you may want to err on the side of making the runway last, even longer.

Alex: Yeah. I mean, we don’t want to slow down during one of the most historic opportunities for growth in AI, but I think you can keep going aggressively forward while also taking a nice reminder about the importance of building good, scalable practices, culture, etc.

Tim: Here, here! So, I’d be remiss to not ask, is there a company or two that you think are particularly cool or innovative in the field of ML broadly? Whether it’s an enabling company or a, finished application?

Alex: I may not sound too original cause the names already came up, but we’re big fans of Hugging Face and OctoML as representatives of those other areas of the ecosystem that, are very exciting and what they’re doing and just the evolution around models around infrastructure and, the fact that those companies exist and those technologies are at the stage of maturity, they are what makes data-centric AI development such a thing.

Tim: I’m sure we could do a whole separate podcast on learnings and tips and advice, but any tips for, maybe the technical founder, your best piece of advice that you’ve gotten on this journey — anything come to mind that you always think of first.

Alex: This is a little specific to data science and AI/ML but gravitates toward real customer problems and real customer pain. Don’t obsess over fitting into the perfect stack diagram or, you know, matching the perfect paradigm of scalability right away, go to where there are real problems, real data real use cases and learn from that.

Tim: Terrific customer-obsessed — customer-focused — is the most important thing. So, I’ve got to tell you maybe as we wrap up here, I’m sitting here, the audience, can’t see, I have my Snorkel T-shirt on, I’m a little bit of a Snorkel fanboy. A few years ago, some website or magazine interviewed me and said, what is a company you’re not an investor in that you’re most excited about? And I said, Snorkel. And Alex rewarded me with a box of swag, so I have this T-shirt to show for it. But the other piece that you don’t know, Alex, is that there was a pair of socks that you sent me, and you have the very kind of fun snorkel logo. And the socks were kind of too small for me. And my daughter saw them sitting on my desk at my home office, and they became her favorite pair of socks. She plays a lot of basketball. She’s in seventh grade. And I just want you to know that in the seventh-grade girl’s hoop leagues of Seattle, programmatic data labeling is being represented well with some flashy footwear.

So, thanks for that.

Alex: I think that’s going to be one of our biggest growth markets. We’re playing the long game here. So, I’m both incredibly humbled and incredibly appreciative because that’s going to be some great long-term value.

Tim: This is terrific. Thank you so much for your time. Congrats on everything you’re building at Snorkel. Thanks for the insights for other entrepreneurs and customers who are building in this world of machine learning and intelligent applications. And hopefully, we can do this again sometime.

Alex: Tim, thank you so much. And this was awesome.

Coral: Thank you for joining us for this IA40 spotlight episode of Founded and Funded. If you’d like to learn more about Snorkel, they can be found Snorkel.ai. To learn more about IA40, please visit IA40.com. Thanks again for joining us and tune in, in a couple of weeks for our next episode of Founded and Funded with Sila Founder Shamir Karkal.

This week on Founded and Funded, we spotlight our next IA40 winners – Hugging Face and OctoML. Managing Director Matt McIlwain talked to Hugging Face Co-founder and CEO Clem Delangue and OctoML Co-founder and CEO Luis Ceze all about foundation models, diving deep into the importance of detecting biases in the data being used to train models as well as the importance of transparency and the ability for researchers to share their models. They discuss open source, business models, the role of cloud providers and debate DevOps versus MLOps, something that Luis feels particularly passionate about. Clem even explains how large models are to machine learning like what Formula 1 is to the car industry.

This transcript was automatically generated and edited for clarity.

Coral: Welcome to Founded and Funded. This is Coral Garnick Ducken, Digital Editor here at Madrona Venture Group. And this week we’re spotlighting two 2021 IA40 winners. Today Madrona Managing Director Matt McIlwain is talking with Clem Delangue Co-founder and CEO of Hugging Face and Luis Ceze Co-founder and CEO of OctoML. Both of these companies were selected as a top 40 intelligent application by over 50 judges across 40 venture capital firms. Intelligent applications require enabling layers, and we’re delighted to have Clem and Luis on today to talk more about the enabling companies they co-founded, which can work in tandem and are both rooted in open source.

Hugging Face is an AI community and platform for ML models and datasets that was founded in 2016 and has raised $65 million, and OctoML is an ML model deployment platform that automatically optimizes and deploys models into production on any cloud or edge hardware. OctoML spun out of the University of Washington and is one of Madonna’s portfolio companies. Founded in 2019, Octo has raised $133 million to date.

I’ll hand it over to Matt to dive into foundation models, the importance of detecting biases in data being used to train models, as well as the importance of transparency and the ability for researchers to share their models. And of course, how large models are to machine learning like what Formula 1 is to the car industry. But I’ll let Clem explain that one. So, I’ll hand it over to Matt.

Matt: Hello, this is Matt McIlwain. I’m one of the Managing Directors at Madrona Venture Group. So, let’s dive in with these two amazing founders and CEOs, and I want to start with a topic that’s important not only historically in software, but certainly relevant in some new and different ways in the context of intelligent applications and that is open source. Luis, I know your company, OctoML plays on top of your open-source work that you and your team, built with TVM, how do you think about that distinction between the OctoML role versus TVM.

Luis: Just to be clear, the OctoML platform is really an automation platform that takes machine learning models to production. That involves automating the engineering required to get your model and tune for the right hardware, the right choices, reasons, rights, other pieces of the ecosystem, and then wrapping it up into a stable interface that it can go and deploy in the cloud and in the edge.

And TVM is a piece of that, but TVM is a very sophisticated tool that is usable by, I would say machine learning engineers in general. So, the platform automates that and makes it accessible to a much broader set of skill sets, a much broader set of users, and then also pairs TVM with other components of the ecosystem. For example, when should you use a certain hardware-specific library is something that we automate as well. What we want here in the end, is to enable folks deploying machine learning models and teams deploying machine learning models to treat ML models as if they were any other piece of software. Okay, so you don’t have to worry about how you’re going to go and tune and package it to a specific deployment scenario. You have to think about that very carefully today with ML deployment. We want to automate that away and make that be fully transparent and automatic.

So why do we make Apache TVM open source? One of the things that TVM solves — we call the matrix from hell. And if you have a bunch of models and a bunch of hardware targets, and you are mapping any model on any hardware, this requires a lot of diversity, right? What better way to deal with diversity of these combinations of models to hardware than actually having a community that is incentivized to do that. For model creators and framework developers, by using TVM, they have more reach to hardware. So, creating this incentive and folks participating and putting all hands on deck and creating this diverse infrastructure is a perfect match for an open source. So TVM is, and will always be, open source and very grateful to that.

Matt: Clem, frame a little bit for us, how you thought about open source and how you’ve thought about it in the context of your marketplace.

Clem: Basically, at Hugging Face, we believe that machine learning is like the technology trend of the decade, that it’s becoming the default way of building technology. If you look at it like that, you realize that it’s not going to be the product of one single company, it’s really going to take collaboration of hundreds of different companies to achieve that. So that’s why we’ve always taken a very open source, collaborative, platform approach to machine learning.

And a little bit, like what GitHub did for software, meaning becoming this repository of code, this place where software engineers collaborate, version their code, and share their code to the world. We’ve seen that there was value, thanks to the usage of our platform, in doing something similar, but for machine learning artifacts — so for models and data sets. So, what we’ve seen is that by building a platform, by being community first, we’ve unlocked, for now 10,000 companies using us, the ability to build machine learning better than what they were doing before.

Matt: So, Clem, that’s really interesting. Maybe just to build on that last point. When people are trying to use these models, there is often some kind of underlying software that’s involved with the building, the training, the leveraging of the model. There’s also datasets — some that are open public data sets, some that are not. So, in that context, how do you all work with both the software and the data set elements that are more or less open in terms of leveraging your platform?

Clem: Yeah. So, something that we were pretty convinced about since we started working on this platform three years ago, is that for it to work and really empower companies to really build the machine learning, it had to be extensible, modular, and open. We don’t believe in this idea of providing an off-the-shelf API for machine learning — like having one company doing machine learning and then the rest of the world won’t be doing machine learning. It can be useful for a subset of companies, but the truth is at the end of the day, most companies out there will want to build machine learning. So, you need to give them tools that fits their use cases that fit their existing infrastructure that can be integrated with, parts of the stack that they already have.

So, for example, for private-public, what we’re seeing is that by giving the choice to the companies to pick which part they want to be private, which part they want to be public — what’s interesting is that it usually evolves over time in the machine learning life cycle. If you think of like the beginning of a machine learning project, what you want to do is maybe train a new model on public data sets because it’s already available, it’s already formatted the right way for you task. That gets you to a minimal viable product model really fast. Then once you’ve validated that it could be include into your product, then you can maybe switch to private data sources and then train a model that you’re going to only keep for your company and keep public. Maybe you’d use that for one year, two years, and then you’re like, okay, now I’ve used it a lot and we’re comfortable sharing that with the world, and then you’re going to move your model into the public domain just to contribute to the whole field. It’s really interesting to see the timeline on these things and how the lines between public and privates are probably much more blurrier than we can think looking at it from the outside.

Matt: That’s super interesting. At one level delineates between the public data sources that presumably people are free to use and the private data sources, which might have some proprietary usage, rights, and permissions. Maybe one other level in there is kind of the — I want to know what data was used in my model. So, kind of this data lineage piece, and how do you help people with that topic.

Clem: So, we have a bunch of tools. We have a tool that is called the data measurement tool that is very important and useful to try to detect biases in your data, which is a very important topic for us.

We have someone called Dr. Margaret Mitchell, who co-leads and co-created, the machine learning ethics team at Google in the past, and who created something called Model Cards that are now adapted to data, too, which are a way to bring more transparency into the data. Which for me, is incredibly important most actually on the data side than the model side, because if you look today at a lot of the NLP models, for example, if you look at BERTs, it’s incredibly biased, right? If you take like a simple example, like you ask the model to predict the word, when you say, “Clem’s work,” “Clem’s job is” or “Sofia’s work is.” You’ll see that the word that is predicted is very different if the first name is a male or if it’s female. You’ll even get on the woman’s side, the fist prediction of a BERT model is “prostitute,” which is incredibly offensive and incredibly biased. So, it’s really important I feel like today in our field that we just acknowledge that. That we don’t try to put that under the rug and build transparency tools, bias mitigation tools, for us to be able to take that into account and make sure we use this technology the right way.

Matt: Yeah, that’s incredibly powerful and helps illustrate beyond the sort of the first set of challenges of building machine learning models that there are these second- and third-order derivative challenges that are going be hard to tackle for a long time to come but are important as you point out to put on the table and acknowledge and work on.

Luis, I’m curious, you referenced this data engineer as your initial customer. Can you tell us a little bit of what you’re learning about the state of these customers and who this data engineer is? Who else might be key decision-makers and using, let’s even put aside like paying for your stuff, just wanting to use it?

Luis: I wouldn’t call them necessarily data engineer. It’s more like ML engineer or ML infra-engineer. So those are folks that think about how to deploy machine learning models today. But what we want here is to have any software developer to be able to deploy the machine learning models and use their existing DevOps infrastructure and existing DevOps people. Right? We are learning a bunch of things from them. First is that it’s just incredibly manual. There’s something that we call the handoff problem from, a model created by a data scientists or folks that create that model to something that’s deployable today involves many steps that are done by humans.

For example, turning a model into code is one step that’s done by hand. Then after that, just figuring out how you’re going to run it. Where are you going to run? It is something that requires a lot of experience with system software tools. If you’re going to deploy on Nvidia, you have to use a certain set of tools. You’re going to deploy an Intel, CPU’s are going to have to use a set of tools.

That’s done by different companies and different customers that have different names for this. Some of those are sophisticated DevOps engineers. Some companies call those machine learning infrastructure engineers, and as the maturity of ML deployment increases in these companies. I’m sure there will be a common name across them, but honestly, if you talk to 10 customers, you’re going to have more than 10 ways of calling those people.

Matt: Is this the same entry point for you, Clem?

Clem: Yeah. What’s interesting to me, the other day I was thinking about, if we want to make like machine learning, the default way of building technology — like software 2.0, in a way. It’s interesting to look at how software became kind of like democratized. If you think about software, like maybe 15, 20 years ago, and who was building software. You realize that maybe, obviously software got adopted really fast, but if there was one thing that was limiting is how to train a software engineer. Because it’s hard, to take maybe someone who was a consultant before, or like was working on finance and then train them to become a software engineer is hard work. It’s not something that they’re going to do really fast. What’s beautiful with machine learning is that, this wave of education of software engineers almost kind created the foundations to go much faster on a machine learning because turning a software engineer into someone who can do machine learning is much faster. For example, with the Hugging Face course, which takes a few hours to take, we see software engineers starting this course and at the end of the course, being able to start building machine learning products, which is pretty amazing. So when you think about the future of machine learning and the rates of adoption, one of the reasons why I’m super optimistic is that I think it’s not crazy to think that, maybe in four or five years, we might have more people able to build machine learning than software engineers today. I don’t really know how we’re going to close them. Maybe they’re still going to be called software engineers. Maybe they’re going to be called machine learning engineers? Maybe they’d have another name.

Luis: Maybe just application engineers because applications have any intelligent components, it should just be application engineers, right?

So, Matt I have a bunch of questions for Clem too. So let me know when we can ask questions to each other here.

Matt: Let me ask one question of you and then you can go. You’ve shared with me a few times that you think this whole construct of MLOps, which I guess arguably today is the cousin of DevOps is just going to go away and maybe this gets back to this, what are we going to call the people? It doesn’t really matter, maybe they’re all application engineers over time. Do you see MLOps and DevOps merging or is MLOps just automated away? What’s your vision around that Luis?

Luis: To be very clear for the rest of the audience here. So creating models or arriving at a model that does something useful for you, it’s very distinct to how we’ve been writing software so far. I know to Clem’s point, he put it very well. That part, I don’t know what name that has. I do not include that in MLOps. But MLOps, I mean, like, once you have a model, how do you put it in operation and manage it? That’s the part that whenever I look at it super closely today, it involves turning a machine learning model into deployment artifact, integrating the machine learning model process and deployment with the regular application life cycle deployments, like CICD and so on. And even monitoring a machine learning model once it’s in deployment. So, all of that, the people call MLOps. If we did it right and enabled a machine learning model to be treated like any other piece of software module today, you should use the existing CICD infrastructure. You should use the existing DevOps people. You should even use your existing ways of collecting data for things in deployment, like what Datadog does, and then put views and interpretation on top of that.

So, our view here is that if we do all of this we should be able to, once you have a model, you turn that into an artifact that you can use the existing DevOps infrastructure to deal with. So, in that view, I would say that MLOps shouldn’t be called anything else other than DevOps. Because you have a model that you can treat as if it were any other piece of software. So that’s our vision.

Matt: Clem do you agree with this vision?

Clem: Yeah, yeah — I think it is very accurate.

Matt: Good. Luis, what were you going to ask Clem?

Luis: First, what makes some models wildly popular? Out of these tens of thousands of models I’m sure there’s a very bi-modal distribution there. Do you see any patterns of what makes models, especially popular with the general audience?

Clem: It’s a tough question. I think it varies wildly based on where the company is in terms of like their machine learning life cycle. Like when they start with machine learning, they’re going to tend to use the most popular, more generic kind of models. They’re going to start with BERTs, with DistilBERT, for example for NLP. And then move towards kind of like more sophisticated, sometimes more specialized models for their use cases. And sometimes even training their own models. So, it’s very much kind of like a mix of what problem it solves, how easy it solves the problem, how big the model is. Obviously like a big chunk of your work at OctoML is, you know, to make the scaling of these models cheaper for companies to run billions of inferences. It’s all that plus I think one layer that we really created that wasn’t there before is the sort of social or peer validation.

And that’s what you find on GitHub. It’s hard to assess the quality of a repository if you didn’t have things to like numbers of stars, numbers of forks, numbers of contributors to the model. So that’s what we provide also at Hugging Face for models and data sets where you can start to see oh, is this model has been liked a lot. Who’s contributing to this model? Is it evolving and things like that? That also, I think provides like a critical way to peak models, right? Based on what your peers and what the community has been using.

Luis: Yeah, that makes sense. Peer validation is incredibly powerful. I want to touch on another topic quickly and then I’ll pass the token back, you mentioned public data versus private data. There was a really interesting discussion that I think parallels really well with the trends in foundational models. Where you can actually train a giant foundational model on public data and go and refine it with private data. Of course, there’s some risk of bias and we need to manage that. But I’d love to hear your thoughts and where you see the trends of, making the creation of foundational models or even the access to foundational models be something that’s wide enough to have many users refining upon that. We keep hearing about some of these models costing a crazy amount of money to train. Of course, folks are going to want to see a return on that.

Clem: Yeah. I mean, for us, transparency and the ability for researchers to share their work is incredibly important for these researchers, but also for the field in general. I think that’s what powered the progress of the machine learning field in the past five years. And you’re starting to see today some organizations deciding not to release models, which to me is something negative happening in our field, and something we should try to mitigate because we do believe that some of these models are so powerful that they shouldn’t be left only in the hands of the couple of very large organizations.

In the science field there’s always been this trend and this ability to release research for the whole field to have access to them and be able to, for example, mitigate biases, create counter powers, to mitigate like the negative effects that it can have. To me, it’s incredibly important that researchers are still able to share their models, share the data sets publicly for the whole field to really benefit from them. Maybe just to, to complement on that we’ve led with Hugging Face, an initiative called BigScience, which is gathering almost a thousand researchers all over the world. Some from some of the biggest organizations, some more academic — from more than 250 institutions to train ethically and publicly the largest language model out there. It’s really exciting because you can really follow the training in the open.

Luis: I’ve been seeing that’s fantastic to see that.

Clem: I like to joke sometimes that very large models are to machine learning, what Formula 1 is to the car industry. In the sense that the two main things that they do is first they’re good branding. They’re good PR, they’re good marketing — the same way Formula 1 is. And second, they are pushing the limits of what you’re able to do to have some learning. The truth is you and I, when we are going to work, we’re not going to use like Formula 1, because it’s not practical. It’s too expensive. And so that’s not what we’re going to be using. And not all like car manufacturers need to get into Formula 1 — like Tesla is not doing Formula 1.

Matt: I’m going to have to ask you about Charles Leclerc then. Because I have a feeling you might be a big fan.

Clem: Yeah, absolutely. But so, if you think about large language models, that way. And if you realize that the biggest thing is the learning that you get by pushing everything to the extremes, then it creates even more value in doing it in the open. And that’s basically what, BigScience it is kind of like doing this whole process of training a very large language model in the open so that everyone can take advantage of the learning of it. So, if you go on the website, if you check on GitHub, all the learning in terms of oh, it failed because of that, it worked because of that. We tweak that and completely change the learning rate and things like that. So that’s super exciting about that in the sense that it’s building some sort of an artifact for the whole science community, for the whole machine learning community to learn from and get better at doing these things.

Luis: I like the parallel a lot. One of the parallels that I like to think as well as the training these giant models should be equivalent to building a large scientific instrument, say the Hubble Telescope. We spent, a few billion dollars to put it in space and a lot of people can use it. On the commercial side you build a giant machine that you give people some time on to go and do things. I see the parallel, like as any huge engineering effort that’s done upfront to enable future uses. I think that’s the computational equivalent of that, where you have a giant amount of computation whose result is an asset that should be shared. So, in a way that makes sense.

Matt: What I’m trying to get my head around, not to extend this analogy too much is, every team has to build their car. And they don’t tell you everything that they’re doing to make it the fastest car on the track. So, what’s the right layer or layers of abstraction here. Open AI with GPT-3, there’s some things that you can work with and play with, and you can do prompt engineering and all, but there’s some things that are let’s call them in more of the black box, what has been additive about OpenAI’s efforts? And maybe touch a little bit on, with projects like BigScience, what are different and also needed to put it that way.

Clem: I think different layers of abstraction or needed by different kinds of companies and are solving different use cases. Providing an off-the-shelf API for machine learning is needed for companies that are not really able to do machine learning — who just need to call an API to get the prediction. It’s almost the equivalent of a Wix or a Squarespace for technology, right? People were not able to build software to write codes, they’re going to use kind of like a no-code interface to build the websites. And that’s the same thing here, I think. Some use cases are better served with providing an off-the-shelf API and not doing any machine learning yourself. Some others you need to be able to see the layers of the model and be able to train things, to understand things for it to work. So, I think it really depends on the use case, the type of company that you’re talking to. So, for example, the largest open-source language models are on Hugging Face. So, it’s like the models from Editor AI, it’s like the biggest T five models. And they have some usage, but it’s not massive to be honest. Even if they’re like a fraction of the size of the ones that are not public. So at the end of the day, again, it’s Formula 1 there are a couple of cars that a couple of drivers are building, but most of the things that are happening today are actually happening in much smaller models. From what I see, I don’t know if Luis is seeing the same thing. Even like Codex for example — the one that is actually used in production is much, much smaller than what the, like the big number it’s claims in terms of size of the models. I don’t know. Luis, are you seeing the same thing?

Luis: Yeah, similar thing even private companies, right? So, they develope their large models in private, and they go and specialize it — they have their own foundational models and specialized specific use case and deploy that to typically much smaller and much more appropriate for the broader, deployment. I think it’d be interesting to see in the spirit of building communities around it and having people refine on top of large-scale models, is creating broader incentives for folks actually go and pay the high computational costs of training these models. But once they make available, is there a way for them to share some of the upside that people get by refining those models specific use cases. Again, like how I repeat what I said before. I see this giant piles of computation involved in training these models as producing an asset, so that can be used in a number of ways.

Matt: That’s actually a great segue into business models. So, I take a pre-chain model that’s in the Hugging Face market, and I decide to use it and adapt it for my own purposes. How does that work from a business model perspective?

Clem: So, I think the business model of open source and platforms are always similar in terms of high level, in the sense that they like some sort of a premium model, where like most of the companies that are using your product, are not paying most of the time and it creates your top of the funnel? For us, it’s 10,000 companies using us for free. Then a smaller percentage of the companies that are using your platform are paying for additional premium features or capabilities. What we’ve seen is that there was definitely some companies that were obviously very willing to pay because they had specific constraints. When you think about enterprise, especially in like regulated industries. If you think about banking, if you think about healthcare. Obviously, they specific constraints, that make them willing to pay for help on these countries. So that’s one way that we monetize today. The other way is around infrastructure because obviously infrastructure is important for machine learning. And what we’re saying at Hugging Face is that we almost becoming some sort of a gateway for it in the sense that because companies are starting from the model hub, taking their models and then making decisions from them. We can act somehow as a gateway for compute for infrastructure. It is definitely like very much early days, right? As most of our focus has really been on adoption, which I think is what’s making us unique. But I think there is a growing consensus that as machine learning is becoming key for so many companies that machine learning tools, providers, are going to be able to build these big businesses — especially if they have a lot of usage.

Matt: And Luis, similarly, you’ve got a lot of demand and interest for your SaaS offering, as you call it. Maybe tell us a little bit more about that and what you’re seeing in terms of early usage and thoughts about business model.

Luis: Yeah, absolutely. We call it the OctoML platform. So, it’s model in, deployable container out. It’s a simple model people pay to use it. And then the pricing is a function of the number of model hard repairs and also the size of deployment. And what customers are paying for there really is first for automation, right? So often when you’re replacing what humans are doing when taking models to deployment. It’s turning to either using our web interface or an API call. Imagine instead of actually having an engineering team where data scientists say here’s a model and then the deployment folks like, oh, give me the container to deploy it. We put an API on that and run it automatically. It’s a different motion than what Clem just described because the open-source users of TVM — and these are folks that are more sophisticated, they’re using TVM directly. Some of them want to use a platform because they want more automation. For example, they don’t want to go and have to set up a fleet of devices to do tuning on. They don’t have to go and collect the data sets to feed TVM for it to do it’s machine learning, information learning things — all of that is just turn key. And we have, what I call altar loop automation, where you could give a set of models, get a set of harder targets and we solve the matrix from hell for them automatically. Given that there’s a huge difference between using TVM directly or the experience of the platform provides that in that case, it’s very clear. And the platform is a commercial product folks have to pay to use.

Clem: I’d be interested Luis, to hear you about how you see your relationship with the cloud providers is that mostly as, you know, potential customers, partners, competitors. How do you see them?

Luis: Oh, great question. And it’s a good segue here too. I see them as potential customers and partners. Less so as a competitor, and I’ll elaborate. Even though there is some specific points that might seem contradictory to them saying. First of all, so some cloud providers happen to have popular applications that they run on their own cloud and these applications use machine learning in that case, customers — I call that “sell to.”

But the bigger opportunity that I see here is “sell with.” And, from all cloud vendors, what they care about is driving usage in their clouds. So, the way you drive usage in their cloud is to make it very easy for users to get machine learning models, use a lot of computation, and make it really easy to get them on their cloud. So, in whether a service provides us turning models into highly optimized containers that can be moved around in different instances and the cloud vendors like that because it drives up utilization in their cloud.

So, in that case, we’re not seeing resistance. In fact, we’re seeing a lot of encouragement in working with cloud vendors as partners. So talking about selling to and selling with — now, of course, one of these cloud vendors have a service that also builds on TVM — Amazon has something called SageMaker Neo, which is an early offering of using TVM to compile models to run on Amazon cloud. We see our services differentiated in a number of ways. First, there’s some technical differentiation of how we do the tuning of the model to make the most out of the hardware target by using our machine learning for machine learning magic. But more broadly, I would say that the key thing that there’s no competition here is because we support all cloud vendors. And if there’s that one cloud vendor where they can’t be is to be the other cloud vendor at the same time. So, the fact that we sit on top of all these cloud vendors is a huge selling point that I feel likes makes the competition not be relevant. be

Matt: What I think is really interesting here is it’s like what are going to be the right abstraction layers to deliver value in the future? What are the kinds of application areas that are most exciting to you all for the future?

Clem: What I’m super excited obviously is that transformers are starting to make their way from NLP from texts to all the other machine learning domains. If you’re starting to look at computer vision, you’re starting to see vision transformers, if you’re starting to look at speech, you’re seeing like a WAV to VIC, you’re starting to see things in a time series. Uber announced that they’re using transformers now to do a time series for their ETA right? You starting to see biology and chemistry basically taking over all the science benchmarks. So it’s really exciting. Not so much because I feel like the other fields are going to get accelerated as fast as the NLP field did, but also because I think you’re going to start to be able to build much greater bridges between all these domains, which is going to be extremely impactful for final use cases. Let’s say, for example, you think about fraud detection, which is a very important topic for a lot of companies, especially financial companies. Because before, like the domains were very siloed and separated, you were doing it mostly with a time series, right? So, prediction on events and things like that. But now if you’re seeing that everything is powered by transformers, you can actually do a little bit of time series, but also NLP. Because obviously fraud is also predicted by the kind of texts that someone trying to fraud or like a system trying to fraud is sending you. And so you’re starting to see these frontiers between domains getting blurrier and blurrier. In fact, I’m not even sure that these different domains will really exist in a few years. If it’s not going to be all machine learning, all transformers just with different input, right? Like a text input or audio input, image, input, video input, numbers input. And that’s probably like the most exciting thing that I’ve seen in the past few months on Hugging Face. Now we’re seeing it out of adoption for computer vision models, for speech models, time series models, recommender systems. So, I’m super excited about that and the kind of like use cases that it is going to unlock.

Luis: I feel like it’s pretty clear today that almost every single interesting application has multiple machine learning models in them and as an integral part of that. And they’re naturally multimodal as well. There’s language models with computer vision models, with time series models. I think the right abstraction here would be you declare it, that you know, where your ensemble of models are and you should give it to the infrastructure. And infrastructure automatically decides where and what should run, that includes mobile and cloud, right?

So almost every single application has something that’s closer to an end-user and cloud counterparts and even knowing what should run on the edge, what should run in the cloud, that should be automatically done by the infrastructure. So, for us to get there, it requires a level of automation that is not quite there yet. Even like when you give a set of models and deciding maybe a given model should be split into two, where part of it runs in the cloud and part of it runs on the edge. So that’s where I think the abstraction should be. You should not worry about where things are running and how. That should be fully automatic.

Now on the — what is an exciting application? This is going to be more personal and Matt, that’s probably not going to be a surprise to you. I think there’s so many exciting applications in life sciences. It’s inherently multimodal — from using commodity sensors in smartphones to make diagnostic decisions. There is a lot of interesting progress there using microphones to measure lung capacity, for example, for using cameras to make skin cancer early diagnosis and things like that. All the way to, you know, much larger scale computations and everything that’s going on in deep genomics in applying modern machine learning models into giant genomic datasets, is something that I find extremely exciting and not surprisingly a lot of those use transformers as well. So what I’m seeing actually, I’m also very excited about what, Clem said. It’s fantastic to see what Hugging Face has been doing and showing the diversity of use cases, transformer models apply to. Just like, bring it a little bit closer in terms of the actual application, I feel like life science is the one that inherently puts everything together into a very high value and meaningful application of human health.

Clem: And something I wanted to add because it’s easy to miss it if you not following closely, but already today, if you think about your day, most of it is spent in machine learning. And that is something new you have to realize because maybe two, three years ago, there was some like over-hype about AI, right? Everyone was talking about AI, but there was not really a lot of final use cases. Today, it’s not the case anymore. If you think about you day you can, do a Google search — it’s machine learning-powered. You’re going to write an email — autocomplete its machine learning-powered. And you’re going to order Uber, your ETA is machine learning-powered. You’re going to go on zoom or this podcast, noise-canceling and background removal is machine learning. Going to go on social network, your feed is machine learning-powered. So already today you’re spending most of your day in machine learning, which obviously is extremely exciting.

Matt: Yeah, it kind of leads to a question what’s the technology, that’s the greatest source of disruption and innovation that you see in the next five to 10 years.

Clem: So, for me, it might not be a technology in itself, but I’m really excited about everything decentralized. And not just in the crypto blockchain kind of sense. So, for example, Hugging Face, we’re trying to build a very decentralized organization in the sense that decision making is done kind of like everywhere in the organization in a very bottom-up way rather than top-down. And I’m really excited about applying this notion of decentralization. I think it’s going to fundamentally change the way that we build technology.

Luis: For me, it is impacted by AI too, but it’s molecular level manipulation. It’s just everywhere. You saw Nvidia’s announcement of 4-nanometer transistor technologists, soon we’re going to see 2 nanometers — we’re closely getting to the molecular scale there. So, this is applied to manufacturing electronics, but then, going back to life sciences, our ability to design, synthesize and read things at the molecular scale is something that’s there today already. So just think about DNA sequencing. You can read individual pieces of DNA with extreme accuracy, in large part because of AI algorithms that decode very noisy data, but our ability to read individual molecules is there and the ability to synthesize them.

So, I hope I’m not being confusing putting these two things together. I think in the end, being able to manipulate things at the molecular scale has a deep impact on how we build computers, because computers are in the end dependent on how you put the right molecules together, and same thing applies to living systems. So in the end, we’re all composed of molecules and being able to engineer synthesize the right ones has profound impacts on life. So that’s my favorite one, yeah.

Matt: I don’t know how I can bring us back down after that. Basically, to synthesize it, the journey from atoms and physics to bits and computing, to bases and biology, you know, and the intersections of those worlds. And what’s going to happen in the future as a result.

I know you and I are both passionate about that and no doubt from what Clem is saying, too, and bringing in this point about decentralization as well. And how does that change the way that we can work and learn and discover together. Very exciting. Hey, is there a company, in this intelligent application world, maybe more up at the application level, as opposed to the enabling level where both of your companies are playing today, that you’re, you really just admire and think a lot of maybe it’s because of some of these cultural attributes about the centralization clam, or maybe cause of the problem that they’re trying to solve that you’d say, wow, that’s one of the coolest, private, innovative, intelligent application companies?

Clem: I recently talked to a Patricia from a Private AI, which to me is doing something really exciting because initially it sounds like a boring topic in a way, which is a PII detection, like detecting personal information in for example, your data or your sets. But I think it’s incredibly important to understand better what’s in your data, what’s in your model, in terms of problems, right?

Like is there personal information that you don’t want to share? Are there biases? I think of being much more like valuing forums and kind of like building technology with values rather than thinking that you’re just a tool, that doesn’t have value and kind of like the harm comes from people using your tool. I think it’s a very big technology switch that we’re seeing happening now with companies and organizations having to be very intentional about the product decisions that they take, to make sure that you reflect their values and the values that they want to kind of like broadcast.

Luis: One company that I think is doing really cool, intelligent applications is a company called RunwayML. That’s the ability of manipulating media in a very easy way using machine learning, really cool. Like for example, how you can very easily edit videos in a pretty profound way, that had been incredibly manual and hard in the past. Now turning that into something that’s point and click it’s pretty exciting. Also comes from the ability of training, large you know, models to generate visual content. So that’s one of them.

Matt: Let me bring us to kind of a wrap up with a question around your own entrepreneurial journeys. We have a lot of folks that are listening that are starting or thinking about starting companies. And if you could share with us, one or perhaps two, the most important lessons, things that you’ve learned, wished you knew better going into the entrepreneurship journey that might be helpful for others. I think that would be tremendously valuable to our listeners.

Clem: It’s a tough question because I think the beauty of entrepreneurship is that you can really own your uniqueness and really build a company that plays on your strengths and doesn’t care about your weaknesses. So, I think there are as many journeys as they are startups. Right? But if I had to kind keep it very general. I would say for me, like the biggest learning was to take steps, just one at a time. You don’t really know what’s going to happen in five years in three years. So just like deal with the now, take time to enjoy your journey and enjoy where you are now because I don’t know if Luis it’s the same, but you obviously look back at the first few years, and at the time you felt like you were struggling, but at the end of the day it was fun. Then, yeah, obviously to trust yourself as a founder, you know, like you’ll get millions of advices, usually conflicting. For me it’s been a good learning just to learn, to trust myself, go with my gut and usually it pays off.

Luis: It’s hard to top that, but I will say, for me, personally coming from academia, it’s been fantastic to see a different form of impact because as a professor, you can have impact by writing papers that people read and then can change fields or training students that go and do their own thing and become professors and so on. But then I see building a company out of research that started in universities and all the ways of impact that actually putting products in people’s hands. Some of the lessons that I’ve learned as you know, Matt, there’s massive survivor bias here, but you know, just picking people that you generally like to work with is incredibly important. People that are supported, they can count on people around you and feel like there is a very trusting relationship with the folks that you work closely with. It’s just something that is true in building a company. I’m sure it’s true in many other things in life as well, but I’m extremely grateful to be surrounded by people that I deeply trust. I have no worries about showing weaknesses and having to be always right. No, I think it’s great when you say you know what I did wrong, I’m going to fix it. It’s much better to admit if you’re wrong and fix it quickly than trying to insist that being right is important. But a funny thing that I’ve learned like yet again, is that we overestimate what we can do in the short term, but we underestimate them, what we can do in the long run. When putting plans together, we all have this ambitious things, we’re going to get this into the next two months. And you almost always get that wrong because you overestimate that. But then when you think about a plan that is a few years, like a couple of years out, you almost always, undershoot, right?

So, when I keep seeing this time, and again, and this is something that I think affects how you think about building your company, putting plans together, especially when things are moving fast. It matters a lot. So put a lot of thoughts into plans, writing things down a lot.

Matt: Well, you’ve heard this from me before Luis, but Clem, I love what you said too, because it is, the customer and the founder are almost always right. And the VC is often wrong. So, they’re trying hard. We try hard! Well, gosh, I’ve just so enjoyed getting a chance to listen to both of you and asking a few questions and, you know, excited to see where this world of enabling technologies like Hugging Face and Octo ML and the underlying capabilities around that go in the future. What that portends for the future of intelligent applications that are brought together and really can, I think, transform the world where I think you’re probably both right. That in the future, we’re not going to think about DevOps and MLOps, we’re not going to think about apps and other apps. We’re just going to have this kind of notion of application engineering. But there’s lots of problems to solve along that journey. So thank you so much for spending time with us. Congratulations again on being winners in the intelligent application inaugural class.

And we look forward to seeing all the progress in the future for both your companies.

Clem: Thanks so much.

Coral: Thank you for joining us for this IA40 spotlight episode of Founded and Funded. If you’d like to learn more about Hugging Face, they can be found at HuggingFace.co to learn more about OctoML, visit OctoML.ai. And, of course, to learn more about IA40, please visit IA40.com. Thanks again for joining us, and tune in in a couple of weeks for Founded and Funded’s next spotlight episode on another IA40 winner.

This week on Founded and Funded, we spotlight our next IA40 winner – Starburst Data. Managing Director Matt McIlwain talks to co-founder and CEO Justin Borgman about how launching his first company was like getting a Ph.D. in entrepreneurship, and then they dive into the customer problem Justin saw that made him believe the time was right to launch his second — Starburst. The two discuss open-source alignment, why making use of cloud partnerships early, especially cloud marketplaces, can be so beneficial for startups, why Starburst had to change the name of its query engine from Presto to Trino, and Justin’s guidance for creating a future-proof architecture.

This transcript was automatically generated and edited for clarity.

Coral: Welcome to Founded and Funded. This is Coral Garnick Ducken, Digital Editor here at Madrona Venture Group. And this week we’re spotlighting another 2021 IA40 winner. Today Madrona Managing Director Matt McIlwain is talking with Justin Borgman, founder and CEO of Starburst Data, which was selected as a Top 40 intelligent application by over 50 judges, across 40 venture capital firms. We define intelligent applications as the next generation of applications that harness the power of machine intelligence to create a continuously improving experience for the end-user and solve a business problem better than ever before.

These applications require enabling layers. And we’re delighted to have Justin on today to talk more about the enabling company he co-founded in 2017. Justin walks us through how launching his first company – Hadapt – was basically like getting a Ph.D. in entrepreneurship and then through the customer problem he saw that led to the launch of his second company – Starburst. Matt and Justin discussed why making use of cloud marketplaces early can be so beneficial for startups. Why Starburst had to change the name of its query engine from Presto to Trino, and Justin’s guidance for creating a future-proof architecture. But I don’t want to give it all away. So, with that, I’ll hand it over to Matt and Justin.

Matt: Well, hello everybody. I’m Matt McIlwain, I’m a Managing Director here at Madrona Venture Group, and I’m just delighted to welcome Justin Borgman, Founder and CEO of Starburst Data. Starburst is really behind the popular Presto-based open-source project called Trino that helps customers carry out complex analytics on disparate distributed data sources. We’re going to talk all about that here with Justin and, you know, Starburst was selected as one of the top 40 intelligent applications, as an enabling application. And as you’ll see, Starburst is very much the kind of the core of that. And one of the things we’re going to dig into today a bit is at what layer of abstraction this next generation of data enablers actually lives. But before we get into all of that, Justin welcome.

Justin: Thank you, Matt. You know, we’re honored to be selected, and it’s a pleasure to be here with you today.

Matt: I think it would be just great because prior to Starburst, you’ve done some really amazing things, and I think they kind of inform ultimately how you got energized and excited to create Starburst. Can you, for our audience, just walk us through the time before Starburst?

Justin: Yeah, sure. My journey, at least in big data and analytics, really begins back in 2010. So, 12 years ago with the founding of my first company, which was called Hadapt. And that business was really based on some research by the folks who became my co-founders in that company, Daniel Abadi and Kamil Bajda-Pawlikowski who were a professor and Ph.D. student at Yale University and co-wrote a paper called HadoopDB. And the basic idea of back in 2010 that they had, and really were pioneers with this paper — was could we turn Hadoop, which was becoming the data lake. In fact, the term data lake was really created in the context of Hadoop back then — could we turn that into a data warehouse? Could you actually run SQL analytics on data in Hadoop? Could you connect BI tools? Could you use this effectively as an open-source data warehouse? And I was in business school at the time. I had a computer science degree previous to that. I was a software engineer for the first few years of my career before going to business school. I read this paper, and I was like, this is the coolest thing ever. I walked over to the computer science department and talk to those guys into starting Hadapt with me, which was really the commercialization of that research.

Ultimately, we built that business over four years and learned a tremendous amount in that process, both in terms of the market but also as an entrepreneur, as a first-time CEO. Even though I was in business school, maybe my Ph.D. I guess you could say was going through that first startup. Cause there’s so much that you learn through experience that you really can’t read about and almost can’t be taught without going through it. And some of the lessons of that startup that we saw, and this was particularly evident to me when the company was acquired by Teradata in 2014. So, I became a VP and GM at Teradata. And one of the things that became very clear to me at Teradata, which is by the way, like the pioneer of the enterprise data warehouse, right. They’ve been around 40 years and they kind of created this concept of a single source of truth, get all of your data into one place. And what I found was that despite their success none of their customers had gotten all of their data into one place. And that was a really eye-opening moment to me that centralization might not be possible. If the leading company for 40 years couldn’t do it, why should we expect we can do it now? That got me thinking about the future of data warehousing in a more decentralized fashion. And that coincided with me meeting the creators of an open-source project at Facebook called Presto at the time. And we began to collaborate — Teradata and Facebook — which may seem like an unlikely pair. We started working on how we could make Presto an enterprise-grade solution, to really allow you to query data anywhere. And that was what excited me about the technology. It was a query engine for anything.

Matt: Wow. Can’t wait to dive more into that. It’s interesting, your observation about Teradata, which really was a pioneer in data warehouses and sort of this point of how hard it is almost more from a sociological perspective to get all the data into one centralized place. Was there also, as you learned more about Teradata, a technological constraint? And what did you find what’s? I mean, congratulations. I mean, it was incredible to build Hadapt and to be acquired by one of the really, truly great technology companies. But what was the constraints there, too?

Justin: Those are great questions. By the way, I want to put an exclamation point on the sociological piece. I think as technologists, we naturally think that – it was a great engineer and leader who gave me the advice maybe 10 or 12 years ago. He said, “There are no technical problems, only people problems.” And that has stuck with me because I think as technologists, we often underestimate that. But to your point on the technical side, and I would say this is maybe just part of a function of the business model of the day, Teredata sold their product as an appliance. And an appliance for anyone listening, who doesn’t know what an appliance is. It’s just hardware and software combined.

And the goal of an appliance is well-intentioned — it’s to provide simplicity to the customer. You just plug it in and go. But it also makes it very inflexible to the world that’s evolving around you. So, I think that was one of the challenges you were buying basically high performance, almost like a supercomputer database, and you were paying a lot for that as a result. So, you really couldn’t take advantage of increasingly low-cost commodity hardware, and then even more so, you couldn’t take advantage of the elasticity and the separation of storage and compute that the cloud provides. Incidentally, that was, I think, what really helped give rise to one of your portfolio companies, which is Snowflake, right?

Which really was the first to take advantage of that storage compute separation.

Matt: Yes. And then to effectively say, well, I’m going to let the cloud be the kind of underlying resource around which I can build an abstraction layer on top of that, which in that case was a cloud-native data warehouse. But you have, in a sense, taking a different approach, complementary but different. Bringing us back to the story of the founding of Starburst — tell us a little bit about the Presto team, maybe build on the beginnings of that story of that collaboration and how that led ultimately to the formation of Starburst.

Justin: Absolutely. Presto was first created by Martin, Dane, David and Eric. They all are here at Starburst of course today, but they created it in 2012 at Facebook and then open-sourced it in 2013. And it was really, one of the goals for them was to provide a much faster interactive query engine compared to Hive, which was the previous generation also created at Facebook by the way. So, Facebook was very much pioneers in open sort of data lake, data warehousing analytics. But Hive was not fast enough. Presto was designed to be much faster, and it had this really interesting abstraction where it was truly disconnected from storage, meaning that they were agnostic to data source. So it wasn’t just a SQL engine for Hadoop. It was a SQL engine for anything. You could query my SQL, you could query Postgres, you could query Kafka, you could query Teradata, you could query anything. That was what attracted me to it and began the collaboration. And you’re absolutely right, I think this is one of the hidden secrets of the Presto/Trino history. Teradata played a really important role in those early days in terms of making it by companies outside of Silicon Valley — companies who need access controls and security enterprise features.

Matt: Enterprise abilities and your insight to listen to the customer and understand that those abilities were going to be needed, especially when you’re talking about data and accessing data, you know, it’s a little before your time. One of the very first companies that I became familiar with at Madrona, and it was an investment we’d already made when I joined in 2000 was a company called Nimble Technologies. And this was a precursor, and it didn’t work to be candid. And part of it was the sociological reasons, you know, who moves my cheese, who moved my data. It was trying to do it in a way that was distributed like Presto and Starburst do, but there was so much concern about the abilities – the securability, the reliability, the availability that at that point in time, I don’t think the technologies were ready either, created the challenges. What were the early use cases that you were seeing? I mean, I’m sure there were some inspired by Facebook that were just so much: is such a problem, I’m willing to go take the risk on this new open-source project in this company, building a hardened layer on top of it.

Justin: Well, there are really two categories of use cases. I think, where the Silicon Valley internet companies at the time were using the technology and still do today, the Airbnb, Netflix, Lyft, LinkedIn, Twitter, Uber, Dropbox were effectively using this as a data warehouse alternative. Those companies deal with such a volume of data they just couldn’t possibly fathom buying expensive appliances, let’s say, to store all of this data and analyze it. And so this became the way that they ran all of their analytics. So that was one category— essentially, I have my data in a data lake. In the early days that was Hadoop. In the more recent years, that’s probably S3 on Amazon or Azure data lake storage or Google Cloud Storage. So, you know, I’ve got really cheap storage. I can store my data and open data format so I can use different tools to interact with it. I can train a machine learning model using Spark, and I can query it with Starburst or at the time Presto, which later became known as Trino. And the reality is, that has been a very core bread and butter use case. Some call that use case now a Lakehouse, basically doing a data warehouse in a data lake.

The other category though, which I think you’ll find interesting Matt, and was a big reason why we built a business around this. We were seeing that fortunate 1000 global customers had a slightly different need that I think actually we could only uniquely solve, which was the fact that they had data silos. They had data in a variety of different systems. So, if you’re a big bank, a big retailer, big healthcare company, particularly regulated industries, you have decentralization, and that’s never going to change. It’s just impossible, truly for those types of enterprises. So, what we were able to do is essentially join tables in different systems and give you fast results.

So maybe you’ve got product data or customer behavior data in a data lake, and you’ve got billing data or finance data in a data warehouse. And you want to be able to join these two together to understand how the customer behavior is driving profitability or revenue, or what have you. So those are classically living in maybe different data sources, and we can execute those queries in effectively real time or at query time and give you fast results. And some people will say, well, that sounds a lot like data virtualization of 10 or 15 years ago. The big difference here is that Trino and Starbursts are actually an MPP execution engine. MPP just means massively parallel processing. So, it’s running on a parallel cluster, not just one machine. And because of that, you can get performance and scale that you could never get with those previous generations.

Matt: And I think that was the technological limitation back in the day is that you didn’t have this MPP capability that has subsequently come along. And for that matter networks so that you could do that in a distributed way.

Justin: That’s exactly right. People ask me, “well, what’s different now.” It is those two points. It’s MPP and its network bandwidth. You’re a hundred percent spot on.

Matt: And so what’s interesting, there is that enables these big institutions to create their own intelligent applications effectively, or their own intelligent analytics platform. They may not turn it into an application. They made us choose to use it for some in-house continuous insights. Is that where you have found more of those types of use cases in contrast to somebody using Trino and Starburst as a platform to build an intelligent application as a service?

Justin: So, in house, I would say was definitely where the business started. And, really began with power users who really understand the data that exists in the organization and just don’t have the ability to access it or query it. It really started with like doing exploratory analysis. I’ve got an idea and I want to go test my hypothesis. I need to run some ad hoc queries and get results. And my goodness is going to take me weeks if I have to go to the data engineering team to create pipelines and move data and get it into our data warehouse. And I need to iterate at a much faster speed. So that time to insight was a real driver of early use cases. The other driver was a need for accuracy or freshness I guess I will say because we allow you to effectively skip ETL and we try not to be too dogmatic about this. We’re not saying that ETL is dead or we’re getting rid of ETL. It’s just that we make it optional. And there are going to be cases where it may be advantageous to just connect to your data source and query it rather than moving it. And that gives you some really interesting optionality as you’re doing your analysis.

Matt: And ETL, of course, meaning extract, transform, and load the data. It’s a set of preparations that make the data more queryable and more usable.

Justin: Exactly. So, with the classic data warehousing model pioneered by Teradata and of course Oracle and IBM, it was all about extracting, that’s the E of ETL, your data from the different data sources you have, doing some kind of transformation to normalize it or get it prepared and then loading it into this new enterprise data warehouse.

And that process, that ETL process, ends up taking a tremendous amount of time, particularly human time in terms of creating those pipelines and maintaining those pipelines. Cause you might add a new field in a source database, and now you need to go add that field in your data warehouse, and you’ve got to keep these in sync and so forth. That’s part of the disruption, I guess you could say that we’re offering the market – the ability to skip that process where it makes sense and just query the data where it lives directly.

Matt: Say more about that? Cause I do think that’s one of the transformative capabilities of Starburst. I mean, how do you do that?

Justin: At a technological level, the easiest way to think about our architecture is that we’re a database without storage. That’s the way I explain it to people. For database geeks, they’ll understand the full stack, you know, there’s this SQL parser, cost-based optimizer, query engine and execution engine, and often a storage engine where you’re storing the data. It’s the storage engine piece that we intentionally don’t have. And that’s what gives us a different perspective on really how we design and build the system where we are intentionally reliant on the storage systems that you connect to. And so, we connect to a catalog that you have either a universal catalog — some companies have all their data in one central catalog, and we partner with Alation and Collibra and Glue Catalog on AWS and so forth. Or you’re connecting to the catalog of the individual source systems — Teradata, Oracle, Hive Metastore and Hadoop — and that is effectively how we know where the data lives. And then our engine is going to execute that query, push the query processing down to where the data lives as much as possible to minimize traffic over the network and then pull back what’s necessary to complete the query, execute the join in memory. And back to that point about MPP — that parallel processing is what’s able to give it the performance and scale. Often I have these conversations with customers who maybe are hearing this the first time and they say, “This sounds too good to be true. How can you possibly do this?” It is that MPP aspect that makes this possible in a performant way.

Matt: And in that sense how should I think about where the quote “file system” lives or the data and metadata system that even if I’m not having to deal with the underlying storage, I still need to know the metadata about all the data that I’m trying to access, so I can do a query.

Justin: Different customers have slightly different approaches here. Some leverage a third-party tool, you know, like Alation or a Collibra, which might be a solution. Others maybe are just joining between data lakes and might be leveraging the Hive Metastore. To me, the lasting legacy of Hadoop is really the Hive Metastore. That seems to continue to persist even in the cloud age, if you will. Or, if they’re in an AWS stack, Glue Catalog is a great way of keeping all of your metadata across a variety of Amazon products in one place, we can leverage that we can collect statistics. Collecting statistics is really important because it allows us to optimize the way we execute the queries when we know how the data is laid out and where it lives.

Matt: That’s great. Maybe also so that people that are not familiar with these things, is this a read-only capability or is there a write-back capability? So, I do a query. I can do some analytics. I want to write something back to those underlying distributed data stores. Tell us about that.

Justin: That’s a really important question. And for anyone in the audience — the reason that question is so important is that historically, if we go back to my first startup, in the land of Hadoop, if you will — the early data lakes, you really couldn’t write data effectively. You couldn’t do updates and deletes. It was really designed to be an append-only system. You just keep adding more data to it, but you couldn’t modify the data that existed. And that was a real limiting factor for a lot of use cases. For example, one of the most popular examples is probably GDPR or other data privacy rules that say, look, Matt wants himself out of our database. He doesn’t want us to keep sending him emails. You have to go in and then remove Matt from the database. And that was very challenging to do in a data lake world. And, and that was one of the reasons, quite frankly, that necessitated that you still had to have a data warehouse in your ecosystem. You couldn’t just do everything in a data lake. Now that has changed in the last few years in a very important way on two levels. On both the query and the storage level. And I’ll explain what I mean by that.

So, first of all, on the storage level. There have been new table formats that allow you now in a data lake to make updates and deletes. And they’re really three that are important today. There’s one called Delta, which was created by Databricks. And then later open-sourced. There’s one called Iceberg, which is definitely a fast-mover. And, I would say keep an eye on Iceberg. That was built at Netflix and is used by many of the internet companies today. And then there’s a third one called Hudi, which came out of Uber. And all three of these approaches effectively allow you to do updates and delete. So no longer is this a limitation of a data lake model or a lake house model.

The other piece is on the query engine side, where over the last year or two we’ve added that on the query side. So now you can write data back. You can do updates and deletes in a data lake. You can even create tables in other data sources. We have some customers that use us as part of a cloud migration, where they’re taking data out of a traditional on-prem data warehouse and moving it into a cloud data warehouse and are able to do that through a SQL query engine effectively.

Matt: I’m going to pop this back up for a second to the open-source history here. So it starts out and you’ve got Presto and then I’m curious how it became Trino and then how the Starburst complements and works with the Trino ecosystem. And what are the types of things you’ve built for the commercial product that are complementary to the open source?

Justin: First of all, I’ll just say for me, as I was thinking about starting my second company, open source was an important criteria of the type of business that I wanted to build, because I think there are some really inherent advantages both for the company and customers. The first is, you get the benefit of contributions from a wide audience. I think that really enriches the technology and allows it to grow and evolve at a faster rate than perhaps a single vendor pushing it forward. And what I mean by that is, for example, in the early days the geospatial functions were created by the ride-sharing companies. We didn’t build those. I mean, maybe we would’ve gotten to it eventually. I don’t know, but they built that. So as a result, pretty much every single ride-sharing company in the world now uses this technology. The other benefit is it gives you very broad distribution. It is open source and therefore it is free. Let’s not mistake the fact that it is free. And like anything that’s free, people are going to download it and start using it and use it on a global basis. So, we’ve had customers in Asia Pacific, Europe, Africa, you know, everywhere from the earliest days of the business because of that distribution.

That was one of the lessons, painful lessons for me, actually, I learned in my first business, Hadapt. Although it ran on top of Hadoop, we were selling proprietary software and when Cloudera introduced Impala and that was free and open-source, included with the distribution. So, you know, that was really hard for us because we weren’t getting the same number of looks or evaluations if you will. The last piece I’ll mention on why open source is, I think for customers, it brings the benefit of not feeling locked in to a specific vendor. And I think at least in the data world that has been a historical pain point – where the Oracles and even Teradatas of the world effectively increased prices became very, very expensive and customers fell kind of captive by their vendors. The notion of an open-source project offers customers the freedom to potentially say, you know what, this vendor isn’t adding the value that I want, but I want to continue to use the technology. They have that flexibility. And this is another reason why I think open data formats are really good for customers because then your data is not locked into a proprietary format either.

So that’s a little bit about the kind of why open source. Then you asked the question about sort of Trino and Presto and how we interact with the community today. So, the original Presto was created at Facebook, as I mentioned by my co-founders and the creators effectively left Facebook, joined us and, in the process, created Presto SQL. And so, you actually had two Prestos — a lot of people didn’t know this, but there was Presto DB and Presto SQL. Unless you were really involved in the space, you know, potato/patato, I guess, for, for a lot of folks back then.

Matt: Yeah.

Justin: But, the reality was that the community effectively moved with Presto SQL. That’s where we were investing. That’s where LinkedIn and the other large community players were investing. The name change was more recent. That was a little over a year ago, and that was driven by a trademark issue because PrestoDB was, was the first name. It was created at Facebook, even though it was created by the folks here. It was created while they were employed there. And the way trademark law works, of course, is your employer owns the IP that you create when you’re employed. And so, basically, PrestoSQL had to change its name. So Presto SQL became Trino a little over a year.

Matt: Got it. That’s super helpful. And I think also helpful for the audience. So now we have you know, this open-source Trino and maybe connect the dots between the underlying open-source capabilities and what Starburst is building on top of that.

Justin: First of all, I will say that the open-source aspect of this is still very core to what we do. And my co-founders are deeply involved in the open-source community. And there is a real, I would say philosophical aspect to wanting to make the open-source project a hundred-year project. I think we look at Postgres maybe as a good example of a database created many, many years ago that is still super relevant today. And in order to do that, you have to really have a vibrant community and you have to be making sure that you’re continuously improving it in a meaningful way.

So, the majority of the performance improvements, scalability improvement — those go right into the engine. The engine remains 100% open source. We build our product off of that open source. We do not have our own proprietary fork. some open-source companies do things that way, we don’t. We build directly off of the open source. And what that means is that effectively, when somebody adds a new feature or capability to the open source, our customers are able to pick it up right away because we’re building off of that. But it also means that we’re continuously invested in the success of the open-source project, because the stability of the underlying technology impacts our own stability for our own customers.

So, we invest a lot of time and energy in that and continue to do so both in terms of code quality and testing and code reviews and so forth.

Matt: And that’s a great mindset to have for both the longevity of the underlying Trino open-source movement, and I think it also serves your customers very well. I know this is a simplification — When I think about another company — Databricks is to Spark as Starburst is to Trino, right? And so, in the case of Databricks, they have done some things to supercharge performance to create a managed service and then create a lot of integrations that make it easier to move things in and out of its managed service in the cloud. And then there’s some of these abilities, these commercial abilities that we’ve talked about that kind of wrap around all of that, that seemed to be some of the core things that you get in Databricks that you wouldn’t get naturally, in this underlying Spark open source. Are those the kinds of things that you all differentiate Starburst from Trino on or complement Trino on? Or how do you think about that?

Justin: In many ways. Yes. I think there are probably a little bit of subtle differences to the philosophy. My co-founders are very adamant that we not have different engines, like core elements of the engine. We just don’t do that in a way that Databricks, I think, does in a few areas. So, you’re getting the same core engine on the open source and Starburst. So, that’s maybe one difference. But I think there are a lot of common themes there. I mean, I think really what we’re trying to do is make the technology accessible, useful, and valuable to customers both in terms of the enterprise features and capabilities they need around security or access controls or connectivity to various different data sources — performance as well. We have this notion of materialized views, which is pretty cool, as well as making it just easier to deploy.

We started with a product called Starburst enterprise that is self-managed, meaning customers have to run it and manage it. That’s been very successful, but we just introduced Starburst Galaxy, which is intended to be super easy. And the beauty here of two products, we debated this a lot. Like, are we just pivoting this? Or is this two products? What does this mean? And it is intentionally two products with different criteria. And what I mean by that is Starburst Enterprise is an always will be intended to be maximally flexible to deploy in your environment, whatever you have. So you’re a big bank. You’ve got Kerberos, you’ve got LDAP, you’ve got, Oracle and Db2, and you’ve got all these different things. We’re going to make sure that enterprise works for you within your environment. Galaxy is optimized for ease of use and time to value. It’s kind of the difference between like Linux and your apple iPhone, right? Like iPhone is meant to be useful to even your grandmother, hopefully. That even she can get value out of it. Linux, of course infinitely flexible. And The way we’ve kind of approached those.

Matt: Just to make sure that I and our audiences are understanding Galexy, how similar is the analogy to kind of Mango Classic and Mongo Atlas, where Atlas is the cloud version — it’s a managed service it’s ease of use kind of dimensions to it. Is that a good analogy or not?

Justin: It is. I think it’s probably one of the best analogies. I would say Mongo and, and maybe Confluent are probably our top-two role models in terms of balancing self-managed enterprise product and a cloud product that are similar and different in important ways. To the point about Mongo and that being a great role model for us, we’re lucky enough to have, Carlos Delatorre the former CRO of Mongo as an angel investor very early on. I’ve learned a lot from him over the years. And then, we just hired as our CRO a guy named Javier Molina, who ran sales for that Atlas product specifically. And one of the reasons we were so attracted to him was because he understands that go-to-market motion, and we think that’s going to be really big for us in terms of the market. Today we do very well in the large enterprise. We think that this technology could be applicable to thousands of customers. Not, not just hundreds of customers.

Matt: That is a great hire because that Atlas product from a sort of a standing start four years ago now represents more than half of all of Mongo sales. It’s just incredible to see the team at Mongo in that way. But maybe take us a little bit into the decision to and then launch Galaxy and how that’s additive to both your existing customers and how it opens the door to some new customers.

Justin: I will preface by saying, and some of the audience may know this, we started Starburst as a bootstrap business. We didn’t actually raise venture right away. And that’s important context because, while I loved that part of the company’s history, and I recommend that to any founder who’s able to get a business off the ground that way initially. The one drawback, of course, is you don’t have the capital to go make huge technology bets necessarily. Right? We were funded by revenue. We were a profitable cash flow, positive business. So the moment that we did raise venture, a couple of years into it, that’s when we said, “Okay, we’re going to build this SaaS solution.” So, one part was like, it takes capital to build a SaaS solution, and that was an important trigger. The other motivator though, which kind of gave us confidence that this would work out, is that we were very early and making our self-managed product available on AWS Marketplace. And the reason I mentioned AWS Marketplace is that was a self-service way of buying and consuming our product.

Now it’s not a SaaS solution per se, but it is a self-service way of transacting, deploying via a CFT, and using our technology. What was very interesting to us, is we launched that when we were, I don’t know, 20 people, bootstrap, tiny little company, nobody had ever heard of us. And we did it mostly just because we thought the marketplace was interesting. It wasn’t necessarily any genius idea. Although, it looks maybe genius in retrospect. But what we saw with that was an organic adoption. We didn’t market our marketplace offering. We didn’t push our marketplace offering. We weren’t doing any outbound back then and we saw more and more people start to use it. What was really interesting about that was not only was it growing on its own without us really doing anything to it, also it was a very long tail of customers. And that was what kind of told us. Okay, we’re obviously having a lot of success with Fortune 1000, but there are companies using our stuff that I’ve never heard of before. And that’s super exciting.

Matt: Yeah, that’s awesome.

Justin: And so, for us, that was the signal that there was a market beyond what we were seeing at that point in time.

Matt: I would imagine that is, especially since it was a self-service offering, so, you know, somebody had to have some degree of technical acumen to kind of stand it up. And run it. Were they most often then running it in the cloud, I guess in theory, I could buy it in the marketplace and then operate it on my own desktop, too.

Justin: I think that’s true in theory, but, but you’re right, that it required some heavy lifting on their part. It was a real effort A) to find us and B) to deploy this, to stand it up and manage it all on their own. To us, it was kind of like, imagine how many people might use it if we could make this easy. And that was the motivation for Galaxy.

Matt: Say a little bit more about how it’s been working with, you know, the big cloud service providers to go to market with Galaxy.

Justin: It is actually available on all three major public clouds. And we designed it that way from the start. But, they’re great partners. And look, I’ll preface by saying of course there’s going to be some coopetition and overlap because every cloud provider has an enormous portfolio of products. So there are overlapping points. But at the end of the day, the field organizations, the sellers, just care about driving consumption of those clouds.

And that’s what we do. You know, the more queries you run on Starburst, the more AWS compute or Azure compute or Google compute, you’re consuming. So, they’ve been great to partner with that way. And the marketplaces, going back to that point, turn out to be a great transaction vehicle. I can’t stress this enough for any aspiring entrepreneur. Get your Ph.D. in marketplaces. And by the way, there are a lot of ecosystem partners now that help you with that, like Tackle for example.

Matt: Are you finding, I mean, I’m sure there are differences. Is there naturally better alignment with your products and the kind of customers you’re trying to reach, between the different cloud service providers or is it too early to tell?

Justin: Well, I think we partner with all of them. We enjoy working with all of them. If I was going to maybe single one out just a little bit, I would say that I think Google’s philosophy or approach to the market is interesting to me and well aligned to some of our own fundamental beliefs.

And what I mean by that is I think Google, as the challenger in the market, acknowledges, understands, and embraces that they’re never going to own all of the data in the world. And that’s important at least important, I think for me, and important for customers, because they’re willing to approach the market from the standpoint of not necessarily saying everything has to be in Google or, creating more freedom for customers to basically do different things in different clouds. They’re much more, I guess I would just say, open to the fact that it’s a heterogeneous world, which is a very core aspect of what we believe.

Matt: And so, to that end, do you find that whether it’s in Google or otherwise, that when I deploy Galaxy in somebody’s cloud, and I’m running it in the cloud, that I’m querying data sources that are back on-premise as part of the queries that I do. Or is it strictly the data that’s living in different data repositories or in a data lake in the cloud?

Justin: It can be either one. And that’s part of the power I think for customers is that flexibility, that optionality, that ability to modernize their architecture before they migrate. We’re not saying don’t migrate, but we’re saying we can give you access to everything you want today. And then you can migrate at your own pace, which I think is very powerful. And just to close on the Google point. We just announced a partnership that allows Google customers to leverage big query, to access data in different clouds, different data sources on-prem, etc., effectively extend beyond Google. And I think that’s an important thing to note as well.

Matt: I do think that this whole thing about data and really workload migrations, you referenced it a couple of times. You know, you and I have lived in the cloud and data world for decades now, and it seems like it’s still relatively early innings, but what are you seeing from a customer perspective, especially the enterprise customer, on their, kind of cloud migration journey?

Justin: I will preface by saying it varies. Some are further along in that journey. Some are just getting started. I think one of the biggest things that I find interesting and really try to drill into when I’m talking to customers is to what degree they think they are going to consolidate all of their data into one place. Because what I have seen, and I think this is a risk, so if there are any potential customers listening to this, keep this in mind. Customers have a fantasy, and I can understand why you would like this fantasy of saying, “Oh cool, we’re going to turn off all of these different databases that we have, this total mess that we have on-prem, and we’re going to just get it all into one cloud data warehouse.” And I’m not picking a Snowflake. I’ve heard the same story repeated with every one of the cloud data warehouses out there. My word of caution would be, we’ve seen that movie before over 30 or 40 years, and to the greater extent that you do do that, the more you’re beholden to a particular vendor, which is going to get expensive for you. What I like to remind people is, all these new companies are very charming and attractive today, but Larry Ellison was charming in 1979, and how many of you are still charmed by him today would be my question.

Right? So just be careful in that. Think from a long-term perspective. Create a future-proofed architecture — those would be just some of our pieces of advice.

Matt: That’s good advice. It might be one thing to say I’m going to retire your old employer, you know, Teradata data warehouse in favor of a more modern cloud-based data warehouse. But I do think it’s highly unlikely and ill-advised to think that you’d ever have all your data in one data store to rule them all as it were for all kinds of reasons. But that I think brings us to this data cloud alliance. I note that Google is a part of that, Databricks, Confluence, several others. What was the genesis? What are you trying to accomplish there in service of your customers?

Justin: It’s around trying to create openness, freedom for customers to be able to work in an interoperable fashion across the different clouds that they may participate in. This is another maybe fantasy that I’ll mention. A lot of companies, I think particularly those early in their journey, will say, no, no, no, we’re just doing one cloud. We’re not doing multicloud. We’re just doing one cloud. It’s all going in cloud X. And, the reality is that changes very quickly. One of the fastest ways that that changes is when you make an acquisition. You just bought a new company, and they’re cloud Y, so now your multicloud, whether you want it to be or not. We have a vested interest in trying to give customers choice and the freedom to operate across these different clouds. And I think Google is very forward-thinking in embracing that as well.

Matt: That leads to an interesting question. I mean, I like to think that, infrastructure as a service or kind of the core elements of cloud service providers, was an abstraction layer effectively on top of hardware. To kind of oversimplify it. But is there a new abstraction layer emerging that maybe we could think of as data lakes, data lake houses, cloud-native data warehouses, or how do you think about that layer of abstraction relative to infrastructure, and then relative on top of it to applications?

Justin: Abstraction is such a powerful vehicle I think for application developers, anyone building an architecture. Abstraction gives you a lot of freedom to change the components of course, underneath. For us, what we’re obviously most interested in is being that abstraction layer for SQL-based access to all of the different data sources that you have, so that you have the freedom to change those pieces. Maybe it’s Hadoop and Teradata today and tomorrow it’s S3 and Snowflake — great — so long as your applications, your BI tools, everything that speaks SQL are pointing to Starburst. And then you have the ability to make those changes underneath, around storage and effectively commoditize storage, which is also very powerful for customers. And there is an emerging name, or a category, if you will, that we’re pretty excited about, which is this notion of a Data Mesh, which is really sort of speaking to this idea of decentralized data and creating a mesh that, that sort of works across that. Now that is back to one of the first things you said on this podcast — there’s a sociological component to it. In fact, the creator of this concept is a woman named Zhamak Dehghani. And if anyone’s interested, I encourage you to buy her book. Actually, we’re giving it away for free on our website. But she describes it as a socio-technical sort of movement, if you will. Which is to say it is people, process, and technology altogether. But we think we can be the technology to enable that. The people and process side is very interesting because part of what that enables is the opportunity to decentralize not just access to data, but a decentralized sort of decision-making and ownership of the data. So, this is kind of like a way of putting more power in the hands of the data producers — the ones who are responsible for that data and know the data the best to also participate in the creation of data as a product that can be shared and consumed by others in the organization. So, it’s a really interesting philosophy one that we see certainly gaining a lot of attention, and I think be gaining more and more momentum over time.

Matt: We touched on some of the technological reasons around the why now. Is there evidence of the, why now on sort of these more sociological dimensions and how much has the fact that we all had to live in a digital-only world for a while? And we now believe, I think we all do, that we’re going to be living in a hybrid working world — has that been part of the why now that sociologically people are saying, “Hey, we just gotta change so we can do more of a decentralized approach,” or am I just kind of speculating here?

Justin: I think that’s right. I think the things driving that in my view are, are first of all, just complexity of data sources. We’ve got more data. Everything is collecting data, right? As we’ve digitally transformed, and the pandemic has only accelerated this, we have now more opportunities to analyze and understand and make data-driven decisions. But to do that, it’s just not scalable for everything to always run through one team, one person, one brain. And that’s where I think decentralization is a great way of giving you velocity by delegating and putting more power in the hands of individuals. And I think consistent with that, we operate in an ever more competitive world and companies have to adapt quickly. The speed of adaptation genuinely impacts your top line and your bottom line. So, I think these are some of the things that are driving serious thought around it.

Matt: That’s well said. I have just a couple of fun questions as we wrap up here, but I just wanted to see if there’s anything else that we didn’t cover. That’s important about what Starburst is trying to accomplish.

Justin: I would just say, you know, at the end of the day, what we’re trying to do, and I hope this doesn’t sound cheesy, but we want to do the right thing for our customers. We want to be on the right side of history. And that was one of the things that motivated me to found Starbursts in the first place was that my time in the database industry, up to that point, I met a lot of customers who just felt very trapped, locked in, they weren’t choosing their technology choices. Those choices had already been made and they were stuck with them. They were living with them. Philosophically this notion of freedom is just core to what we’re trying to do. I think you’ll continuously see that in all of our design decisions. We want to be able to support multiple data sources, multiple data formats, be able to operate anywhere. We want customers to be in control, and we think that’s a slightly different perspective than many in the database world at least have historically had.

Matt: I think one other thing that I was curious about is use cases around taking that freedom and distributed, decentralized approach, and then using some of those data sources to help train models from a machine learning perspective. And are you seeing kind of a growth in those kinds of use cases that Starbursts could help unlock?

Justin: Yeah, absolutely. And I always try to be clear that obviously, we don’t do machine learning. We don’t train machine learning models, but I think we’re a very important partner to that process because you need the data to train the model and the more access to data, the better your model is going to be. And so, getting data is the first step to ML and AI. And we think we’re an important part of that.

Matt: We agree. And that’s why we were delighted that, I mean, it was a very strong endorsement of you all being in this enabler bucket for the Intelligent Application 40, and we certainly see and know about those kinds of use cases. A fun question is outside of your company what’s a startup that you’re most excited about that’s related to this broader world of intelligent applications.

Justin: That is a great question. I think Clari is a really interesting example of this. Clari is really the interface that I’m using to understand my business because it ties in all the important aspects of what we’re doing and provides not only a great summarized view, but also predictive analytics about where we’re going to end up. And particularly as you scale, being able to forecast is so critical, especially in the path to an IPO, which we hope will be able to achieve in the next two to three years.

Matt: So. You’ve now been a successful founder, built two companies, Starburst is still a work in process, but you’re doing incredible things. What’s a lesson or two for those in the audience that are either on their own startup journey or considering the startup journey that had been really valuable to you, whether they’re kind of from your first-hand experience or advice from others or a combination.

Justin: Oh man. There’s a lot. I can say there. I think first of all, the advice that I give to any entrepreneur at any stage in the journey, particularly those that are just thinking about maybe being an entrepreneur. I think the single most important attribute is strictly perseverance. You have to have a high pain threshold and a willingness to push through that pain because is not for the faint of heart. It is not easy. I think just some people are built for that. They have the stubbornness, the drive, to push through that, and others get overwhelmed by it and bogged down. So, that’s kind of like a look inside yourself type of thing to evaluate and consider. The piece of advice I will give that I heard myself. I actually asked a now public company CEO founder, “Does this ever get easier?” Because as you’re building, you always think like, okay, at some point, like, I’m just, it’s just going to get easy, right? Like I’m going to be relaxing on the beach, this thing’s going to run itself. And he said, “No, it’s just different kinds of hard.” And that stuck with me because particularly as you scale, every new chapter has been a new challenge and in a totally different way. That’s part of what’s amazing about startups, I think, just from like a personal growth perspective. You are always having to improve yourself, scale to the next level. And so, that really stuck with me. It never gets easier, just different kinds of hard.

Matt: Different kinds of hard. I love that. I don’t know if I’ve heard it phrased that way. So, I really appreciate you sharing that with us, Justin, and yes, you’re always building these new skills for the next phase of the journey, too. And having to let go of things that you did more of so that you can empower others and scale the organization. It has been an absolute pleasure, Justin, visiting with you and incredible what Starburst has accomplished and your role as an enabler of all kinds of data analytics, including those things that go into building machine learning models and intelligent applications. So, thank you very much for taking time with us today and look forward to seeing the continued success of Starburst.

Justin: Thank you, Matt. I sincerely appreciate it. It’s really been my pleasure.

Coral: Thank you for joining us for this IA40 spotlight episode of Founded and Funded. If you’d like to learn more about Starburst, they can be found at Starburst.io. To learn more about IA40, please visit IA40.com. Thanks again for joining us and tune in, in a couple of weeks for Founded and Funded’s next spotlight episode on another IA40 winner.

In this episode of Founded and Funded, Madrona is launching a special series to highlight some of its IA40 winners, starting with RunwayML, which offers web-based video editing tools that utilize machine learning to automate what used to take video editors hours if not days to accomplish. Madrona Investor Ishani Ummat speaks with Co-founder and CEO Cristobal Valenzuela all about where the idea came from, how he decided to launch a company instead of joining Adobe – and even how TikTok fits into all of this. Listen now to hear all about it.

This transcript was automatically generated and edited for clarity.

Coral: Welcome to founded and funded. This is Coral Garnick Ducken and this week we are launching a special series to spotlight some of last year’s IA40 winners. Today, Madrona investor Ishani Ummat is talking to Cristobal Valenzuela about the web-based video editing tool RunwayML. It all started as a research project inside NYU using an algorithm to stylize and colorize images in Photoshop, but Cristobal now sees Runway as an opportunity to not simply improve how things have commonly been done, but rather leapfrog an entire industry. And the company secured a $35 million Series B in December to work toward that goal. With that, I’m going to just hand it over to Ishani and Cristobal to dive into it.

Ishani: Hi everyone. My name is Ishani and I’m delighted to be here today with Cristobal Valenzuela. The CEO of RunwayML. RunwayML is building a web-based real-time video editing tool with machine learning and last year RunwayML was selected as a top 40 intelligent application by over 50 judges across 40 venture capital firms. We define intelligent applications as the next generation of applications that harness the power of machine intelligence to create a continuously improving experience for the end user and solve a business problem better than ever before. Runway is a story I love — re-imagining creativity with machine learning. And I can’t think of a more interesting conversation to kick off our IA40 spotlight.

Cris, thank you for joining us today.

Cris: Thank you for the invitation. I’m super happy to be here.

Ishani: I’d love to start off with your thesis project actually at NYU. That’s sort of the basis for this company. Take us back to that time. What led you to this idea? Why did you start working on it? And did you know you wanted to start a company?

Cris: So, the short story about Runway is — I’m from Chile, and I moved to New York five years ago. And the reason I moved was at the time, I was just fascinated with things that were coming up in the computer vision world. I’m coming from an econ background and had no experience building deep learning models before, but the things I was seeing specifically around computer vision generative models like five, six years ago, it just blew my mind, and it blew it so much that I just decided to move to study this on a full-time basis at NYU.

So at NYU, I basically spent two years just doing a deep dive into how to really take what was happening, specifically after I would say ImageNET and AlexNET a bunch of really impactful and big milestones in the computer vision world started to emerge, and apply them inside creative and art domains. And the reason was , I think we’re just touching the surface of what it would really actually mean to deploy algorithms inside the creative practice. The reason I wanted to explore those was just, I knew something was happening. I knew something was about to happen, but yet no one was doing it.

So why not just do it yourself? Um, no, I didn’t know if I wanted to start a company, but by the time I was building the thesis, it was more of an organic direction that we took that I realized that my research was way more impactful than I originally thought of. Specifically, when you’re doing research in an academic situation, you’re always constrained, and the bubble is always perfect. You have all the perfect conditions. But when I started applying some of the things I was doing inside school to the outside world, I immediately realized that industry experts, VFX people, film, creators, artists, designers were like, “Hey, I’m interested in this. I want to use it.” And so that kind of sparked the conversation of — “Oh, maybe we should think about this as a company.” And then yeah, it started from there.

Ishani: Was there an aha moment, in that journey as you’re talking to people and they say — “Oh yeah, interesting research, but I don’t actually know how to apply it.” Was there one moment that you can take us back to that said, “Oh, wow. This is actually so significantly bigger and it’s a company, not just a project.”

Cris: I mean, we started the first research projects in school, there were more about taking image segmentation or image understanding models and video understanding models and applying them with creative domains. So how do you take like someone who’s working in Photoshop and help them understand how the software could basically be a bit smarter in terms of understanding what the person is actually trying to do? What the intent of editing and image is and see if you can have an algorithm or a system that assists you on that editing. So, we built a bunch of experiments and integrations in Photoshop and Premiere. And the ideas were very simple. Like, let’s see, for instance, if I can help you just stylize or colorize or edit an image faster by using some very simple algorithms. And again, it was more of let’s see if this is interesting for these creators. And when I realized there was something definitely here, is the reaction when I remember a few tweets around like, here’s a prototype, anyone interested in trying this? And I remember the amount of inbound interest I got from professional photographers, people working in film people working in ad agencies, very organically being just basically, “Hey, I’ve been struggling with this for years can you just help me cut something that took me weeks of work to 10 minutes. I want to learn more.” That’s when we were like, okay, there’s something definitely happening within the scope of creative domains, and so we should go deeper.

I guess there was one moment in particular where I really thought I should try to do it myself. And, so when I was presenting Runway at my thesis at NYU, someone from Adobe was in the panel. And two weeks after my presentation, they basically offered me to join Adobe, to build all the things that we were building at Runway as part of it their new AI team. I was two years into New York as an immigrant, with the perfect dream company offering you the dream job with a visa and the perfect salary – it is just the dream. When I thought about it at that time, I remember my mom was visiting me and she was asking me, “What else do you want? It’s perfect – everything makes sense, rationally. Why would you not take that? Everything you want is there.” But I couldn’t say yes, my gut, my intuition was like, I can’t do it. If I am doing this, if I’m going to build this thing, I need to do it. And I want to have control of how it’s built. And so, the decision of having the offer and having a capacity of jumping in and being like, “Hey, I’m going to take this. This is a safe solution.” Versus, no, I would really want to try and build it on my own even if I fail, I fail, but at least I tried.

So for me, that was the moment where I was like, OK, something happened — either I go and build inside a company or I try to build on my own because I haven’t raised any capital. I’ll try to see if I can sustain living in New York with no money for a couple of months until I figure this out. I think that motivation of like, okay, I’m going to try to prove that I can make the right decision of not taking it, not going to Adobe, was something that I guess motivated us to do it.

Ishani: That’s an incredible story. Can you talk to us a little bit about this technology that underpins Runway? You know, many of the models that you reference and leverage weren’t even around five to seven years ago. We’ve all spent time editing, whether it’s home videos or in Final Cut Pro and the range in between of getting that mug out of the background or even being able to remove the background from an image was such a huge feature in Microsoft PowerPoint that for everyone out there who makes slides on a daily basis and translating that to video seems like an order of magnitude more difficult. Tell us a little bit about the step change in technology that really enabled the core product of Runway to exist.

Cris: Totally. I think there are a bunch of megatrends on which Runway sits today. We’re seeing an emergence of new video content platforms emerging of the last couple of years. And so, the need to create more video has become more obvious for creators, for ad agencies, but also for companies in general. Every company is becoming some sort of media company. They’re creating content all the time. Everyone’s producing their own podcast, their own YouTube shows. The way that software to create content has evolved and has been developed over the last 10, 20 years is, I would say, still based on an old paradigm of how media works. Like, if you open Premiere, if you open Final Cut, those were software made to make ads for TV. And so the limitations and the constraints and the configurations are all set up for like 10 years ago, right? But if you speak with anyone creating content today for YouTube, for TikTok, for Instagram, the volume and the quantity and the type of content is very different. And so that’s, the first megatrend: How do you think about new tools for the next generation of creators. so within that where ML kind of like really come in and where the differentiator of Runaway is that we see a few things that are happening first, the emergence of the web, like the web as a creative medium. I think Figma and Canva have proven this.

The web is such a collaborative space that you need to just be able to build things on the web. If you want to collaborate with more people, if you want to move really fast, if you want to just not be constrained to any limitations from hardware and desktop. I guess, to your question of ML in particular, we build it so in a way that the video platform, the video rendering, the video encoding itself is entirely ML driven. By that, we mean that every single process in that media pipeline that is either tedious, time-consuming or very expensive to do. We can automate via this kind of like pipeline of algorithms. And so, things like you were saying, like removing an object from a background, has been a very tedious process to do historically in video making. It’s a process known as rotoscoping. And it’s been in film and in video for like as early as video was there. Yet, it’s extremely expensive. So, we thought about it. If that’s a primitive principle, for instance, of video-making, how do you make it so it’s accessible? It’s extremely fast. It’s on the web and the way you do it, it’s not a manual, tedious process. It’s automatic – as fast as possible. So, we’ve built it taking those principles of what folks really want in video, simplifying to the core components, using these human-in-the-loop algorithms and then basically helping you make video faster and better. And there’s a lot of other kinds of components of video that we’re automating as well that basically help drive that motion forward to create more video as fast as possible.

Ishani: I love that you frame the company as being built off of megatrends but then focus on the specific use cases. But then, there’s a broad range of use cases here that I hear you talk about. Across whether it’s an individual creator or, you know, a professional photographer. And so it seems quite widely applicable. When you think about some of the research work that you’re doing and the capabilities of making machine learning more accessible to each of those range of end users How do you actually go about picking and choosing the sort of machine learning models that drive it?

Cris: I would say that. Going back to 5, 6, 7 years ago, a lot of the computer vision and ML models started to become more relevant and commonplace. A bunch of things were also built around that time, like the infrastructure to deploy models. And we’ve seen the emergence of ML ops community in general, like tools and systems the monitor, your training process, tools to deploy models to production tools to optimize models to different devices. There’s a lot of things that happen to basically help drive these models into production. And we’ve seen that in like robotics and self-driving cars. Like those algorithms are becoming more predominant than ever before. Basically, because we’ve invested as a community of ML, folds or ML companies on that infrastructure. And so, for us is the realization that we don’t have to build infrastructure ourselves. Like, you can take off-the-shelf solutions to help you deploy the models into production environments, with millions of users in real time, for instance. The core component, I would say it’s not like spending too much time on that infrastructure, given that it’s already been built. It’s more like what’s the unique problem that you’re trying to solve here? If we think about that, there are two ways you can take that approach one is just looking at open source.

The ML community in general has been built a lot on top of open source. And so there’s a lot of ideas that are really interesting. You can borrow them, you can build on top, and you can contribute as well. We do it a lot. We publish. But when it comes to production — like getting things and putting them at the level of perfection that your customers really want — it is a whole other beast. That requires a different mindset. For instance, going back to the rotoscoping example. Video segmentation is a task that has been approached in very different ways on the research side. But when you speak with someone doing video, even if it’s a professional VFX and filmmaker or some casual creator, the way you think about it is completely different. At the end of the day, as a creator, you don’t really care what model goes behind the scenes. I think a lot of people might want to overemphasize the need of showing you how the algorithm works and demonstrating its capabilities. But if you just focus on the customer itself, people just really want to remove the objects from their backgrounds. And so with that in mind, there’s a lot of that comes from like automation from how do you build a robust segmentation model? How do you build it so it works really well? It all has all of these kinds of constraints, but at the same time, how do you involve the user input in that process? So half of it is research on ML and the other is a lot of just user research. How are you doing this today? How are you actually doing a background removal process? Some people might use Photoshop or some very complicated to use tools. Some other people may use some sort of automation by building their own tools, and you’re trying to really understand what that actually means. So you build a solution that specifically within creative domains is never fully automated.

Cris: I’m a big believer that you’re never going to find a tool in the creative space that does everything for you. That’s just a dream. That’s a Utopia. Nothing in the creative world works like that. So every solution that’s just input here, do nothing because the machine will do it for you.

It’s just a complete mistake and totally would not work. So, for us, it is more about, you have a problem, you have an insight, you need something to be done. Here’s a system that we build on research that helps you, but we also understand what you require, how you work with the device and how you work with that loop that we call.

Ishani: And, you know, you could argue that if a machine was doing all of it, isn’t really creative inherently. Do you lose that aspect? That sort of intangible aspect of creativity? So, much to unpack here. So, you talked about the infrastructure layer. We call those enablers in intelligent applications where there’s this whole system of, the Databricks of the world, but that DataRobots and all these other companies that are out there that are Grafana, Monte Carlo, that sit at the enabler level that create the ability for folks like you, RunwayML, to build endpoint applications much faster and better than before. Some of that’s in the open-source community, as you say. And some of that is actually, company-based but it removes the infrastructure layer from every intelligent application that has to be built. And, being able to capitalize on that, I think has made a huge impact on the endpoint applications like Runway. And then you think about bringing that to the product. So much of what you talk about is around accessibility. You know, new technology adoption – so much of it is related to how accessible that technology has become, and so in the academic sense, this machine learning models and development and rendering and all these sorts of technical terms, don’t feel very accessible to creators and particularly the demographic that you’re targeting. But building it into a, low-code/no-code, video editing tool, it really does.

So, the classic question is browser versus application, and you talked a little bit about why you’re in the browser and how it’s become so much more of a collaborative and creative space. What are the other decisions you’ve made along the way to make the Runway experience — specifically being able to get machine learning into the hands of creators at a product level, more accessible for new users, borrowing from things like the workflow of Final Cut Pro or some of the other tools that are out there. Tell us about those decisions that you’ve made along the way.

Cris: There are a lot of things that come into this conversation. The first one is, we’re always thinking in terms of the company, like the build versus buy. If I want to build and deploy models to millions of users, I don’t have to build a whole backend infrastructure and don’t have to own the instances. You just plug into the whole infrastructure that has already been built. And that’s so good because you can focus on the key differentiators of your company. What are the things that are unique as a product that will help your customers do more?

So, for our customers, what they want is just to create more video faster. And so, for that, we basically take existing primitives from the video space. And so, we’re really close to like professional software for people working in the industry for years to try to understand what are you trying to actually do in your workflow and how could something like an automated system help you, but also open the doors for other folks who would have never of being able to do that thing before, do it as well? And when you think about that, you think about, OK, we need to build on top of the infrastructure. We need to allow the new generation of creators to tap into what making video is. The web becomes such an important aspect of that. Mostly because it democratizes access to complicated and sophisticated tools like professional video in a way that I don’t think we’ve seen before.

There are a few things that are really important. The first one is the need for hardware gets reduced to zero. Like a lot of our users are on Chromebooks, on Windows laptops on iPad. It’s really hard to edit video in any of those devices if you don’t have a powerful or deep-feed, GPU machine. So, for a lot of people, that’s not a limitation if you have that capacity to compute. But if you’re a small shop or a small business, or if you’re a small ad agency or even a big ad agency, you still have that limitation on hardware. The web just like completely reduced it to zero. Basically, you’re connected to our cloud. You have that endpoint. And since we already have that GPU cluster running the models, you’re basically able to access not just one GPU machine, you’re able to access a lot. And so if you want to export hundreds of versions of your video, that’s possible. And I think that the second one really important aspect of the web, and why we decided to build in the web again, building on the accessibility point is collaboration.

When you think about video creation today, you can think about people editing video, like video creators, themselves, video editors. But video encompasses more than just people doing the actual editing. It involves the managers. It involves the viewers and the designers. If you’re building a brand, and you have design assets and files, and someone is building in a video, how you share those assets with that person, or with that team really matters. So video becomes like a central hub of collaboration as well. And the web facilitates that at a rate that’s impossible to do in any kind of environment. And so, for us, it’s considering those aspects as well when deciding how and when to build a platform. And aiming and investing in the web for us has been a long-term goal. A lot of the things we’re doing right now in the video space, on the web hasn’t been done, so we’re working with the Chrome team with the Google team, really closely to work on some of the new standards that they’re developing to make sure editing 4k footage with 10 layers at the same time feels as native as possible. And I think Figma has already proven this in the vector UI design. You can run things natively or even more better than native on the web. And now we’re actually starting to see these in video as well, which is a bit more complex in terms of latency and interactions, but we’re definitely getting there.

Ishani: That’s awesome. You talk about cloud computing as a big enabler again and this collaboration concept. Multiplayer in the web is this next generation of collaboration and you’re right, Figma, Coda, Notion, Canva have made collaboration and multiplayer inherent and I think a lot of the applications that don’t have that multiplayer component are proving to be much more difficult to use, especially within teams and within a remote and hybrid kind of world that we’re entering. Figma and Canva — you mentioned them. They really, to me, started to pave the way to this multiplayer concept — web-based — but also this concept of low-code/no-code and being able to set the precedent for using machine learning, using technology in a much more accessible way for a non-technical user.

Do you think of that as one of the big trends that’s enabled and paved the way for you and Runway.

Cris: So, when we think about it, I guess no code for us on the ML side of things, we actually think a lot about how we take these models, these very complex pieces of software with hundreds of thousands of connections and systems to make them work really well and robust, into really consumable and easy to digest and simple solutions as an interface. Making sure that you build interfaces that are programmable or accessible and customizable. I think in a way, it becomes a commodity like it’s a system that you build, it’s proprietary, you develop it, but your customers are less concerned about the internal aspects of how it works and are more concerned about the output, right? And so, when I think about Webflow for instance, and I think about web designing in general, like Squarespace or those kinds of companies, would build like democratizing, no-code solutions for building websites, you really care about your customers just building really good websites. Right? How CSS and the JavaScript endpoints work on the backend are not really useful for them, unless you’re helping them solve a business use case. And so, you don’t really expose those kinds of things.

Ishani: That’s great, framing it as exposure. I hadn’t quite thought of it that way before, but it does make sense. You’re masking sort of the code and you can expose to components of it where it matters and where it’s a variable that people want to influence. But where It’s not. And you learn a lot of this through user testing, but where it’s not you can mask it. Tell us a little bit about the process for that user testing. I mean, so much of what you’re talking about is really driven by your end user. And it seems like you’re really in touch with who that is and how you learned a lot from them. What does the process for that look like? I think it’s so important as you iterate on early product and early build. And when you launch a new feature, you know, in your case Green Screen, for example, what’s the process you go through for a user iteration and feedback.

Cris: I love that question. I think a few things are important. The first one is a lot of times your users don’t actually know what they want.

Ishani: They just know They have a problem, but they don’t know to solve it.

Cris: Exactly. So, if you ask them the answer to what they want, that will not necessarily be the best solution. That’s the realm of knowledge they have today. In a way, no one was ever asking for an automated rotoscoping solution because no one thought that was possible. When you start doing and developing technologies or start delving deeper into things that haven’t been done before, it’s really hard to do comparisons to like, how has these been working before? Because no one has done it before, so it’s really hard to have a benchmark.

And so, when you ask people, what’s a pain for you in the video space, a lot of people will tell you like, Hey, rotoscoping, extremely painful. So, what do you want? Well, I want a better brush so I can do my mask five times faster. And so, I could be like, great, I’ve listened to you. I’ve built this thing, now you’re working two times faster. Do you like it? And it’s great. I like it. But the moment you mentioned like, “Hey, I can actually automate the whole thing for you. Just literally type a word.” And this is true. We have this as a beta that we are going to release really soon, where you can type. Let’s say you have a shot of a car and a tree. You can type “car.” Then we have a model that understands the object in that video, understands the car, creates the masks for you and extracts the mask immediately. And so, you’re not editing anymore with frames, your editing with words, right?

It’s really hard for our customer to tell you that — “Hey, I want to like this thing.” But the moment you show them to them, they’re like, “Oh, it’s insane. Like I want this. It is not only helping me move twice as fast. It’s helping me move a hundred times faster.” So, a lot of the user research in a way is like listening to your customers and listening to your users, but actually trying to really listen or hear their pain. Okay, what are you actually trying to say when you’re saying these things, this is actually the tool itself is a problem, or it’s more of like, the process is broken. If you have the process that’s broken and you as a product person know that technology, know the skills of your team and what’s possible today, how do you build quick prototypes and solutions that can help you actually figure out if that’s actually something worth investing and building?

Cris: So, we do a lot of that. We listen a lot. We understand our customers. We understand either people who have never used the Runway before. We interview them a lot and we try to distill, okay, what’s the fundamental things that are happening here. And how would we build them with a set of technologies that we’ve been developing over the last couple of years?

Ishani: Right — and from the end-user standpoint. It’s just not in the realm of possibility to augment their workflow so much with automation, you know, maybe incremental baby steps. But as you say, the 100X just doesn’t fall within the imagination of someone using a video tool to take it all the way to, for example, text-based video editing. That’s in the realm of researchers at OpenAI, doing GPT3 work and DALL-E, and all those image processing things. So being able to really distill down a pain point, but then you use your imagination to go from up with a solution.

Cris: And that’s a lot of prototyping as well. Basically, coming up with ideas and you just test those ideas with your customers as quickly as possible before building really robust and technically complex solutions. So, I guess to your point of for instance more on the generative side, something we’ve been spending a lot of time on generative models, deficient models, transform, applies to computer vision. The thesis there is that we’re probably going to start seeing more video content being entirely generated. So, think about stock footage or stock video, right? It was the case before you had to either shoot something or buy that footage from like a Getty Image platform, and that’s a really expensive process, both because the acid itself is super expensive to buy, but also because the asset might never actually be the perfect asset that you want. There’s some things that you want to change the color isn’t right. I want that person, but in a different position. It’s so complicated. And so, we’re approaching the space where you’re actually going to be able to generate those things, generate that stock footage, that footage in general. So, when you ask people, how do you want to create or work with assets, with templates, with custom content, they might ask you like, “Hey, I want a better search for my stock footage library.” But the moment you have Dall-E or other models that are able to generate realistic content, the conversation completely changes. You’re not marginally improving a process. You’re leapfrogging a whole industry. You’re like, okay, this was the way people used to operate.

Now, this technology is enabling you to think in just a completely different way. The questions you’re asking yourself are so different. And so having that is something we’ve always had in mind. And we’re also betting on that, on the long term as well.

Ishani: Incredible. Yeah. That leap from video editing to transformer model augmented video editing is massive, right? Transformative from technology perspective, but massive from it. Just how do I make that leap and requiring the technology, the examples to saying, oh, I can use transformer models in this process. We can talk about transformer models forever. Maybe take us to the moment where that started to make sense for you as a business.

Cris: I think the moment we started seeing this as an interesting research technique was the moment people understood that you can apply it not just to tokens, but to like pixels themselves. We use some of these techniques for our models behind the scenes, but in general, I’m less of a fan of a specific technique because techniques tend to move really fast, and something else will happen. And so, I think it’s important always to like — when you see those trends coming up, see how they can adjust to your product or your needs. But at the same time, don’t fixate too much on specific technique because a new technique might come up that might be better. And the ability to switch and learn from what’s better, I think we’ll always pay off versus like, if you’ve spent too much time developing something and then a new approach comes and you’re unable to adjust. then it’s going to be hard. I mean, the space moves so fast. The ML space is moving so fast that something that just published four months ago has already been changed, so keeping track of that, I think it’s the most impactful thing. I guess on the research side of Runway, we do a lot of different approaches from transformers to more generative stuff from our traditional computer vision as well. Again, always in the aim of like, how do we help you make video faster?

Ishani: That’s a really great insight to be nimble across, you know, a rapidly evolving technology field. And the conversation, even if you just zoom in on transformer models on how large these models have been and how many parameters they’ve been trained on, even over the course of the last 12 months, the chart is absurd, right? And that point of you building a business on top of some of these platform technologies, or what will evolve to be platform technologies, being nimble across the methodology is so, key.

Cris: One hundred percent. Because at the same time, there are a lot of things that are happening specifically where you mentioned if like those hose models themselves that our greater research insights, but try to, productionalize a model that has 2 billion parameters for like a million users. You either have a budget of a million, a million dollars a second, or it’s impossible to do it. Right? So, it’s great. Like fundamentally. It’s moving the field in such an interesting way. There’s new techniques. But again, if you’re thinking about how to put it into a product, that’s a whole different conversation.

Cris: So always trying to balance those things for us is really important.

Ishani: So how do you straddle then the business side of Runway and the research side of Runway.

Cris: We don’t see them as different worlds. It’s part of the same. So, research at Runway, is just applied research to product? As a researcher at Runway, you work really closely with the design team and with the engineering team and with everyone to really figure out if there’s something we can do. There’s a cost, like a literal cost, that needs to be considered, and compute, to have in mind when you are developing that. There’s the feasibility approach — is there something we can actually build in a reasonable amount of time? There’s a performance trade-off and all of these things that you have in mind when you’re thinking about applying those into a product.

Perhaps if you’re a more formal academic context, and you’re just doing research. You’re not constrained by those things. I mean, when OpenAI was building GPT-3 they were not thinking about deploying this for video domains with millions of visitors, they were thinking — this is an idea, let’s see if it works. And then people start building on top of that. Now there’s a lot of like pruning and ideas that can come to make it more efficient, more fast. But it’s still, if you look at OpenAIs for like pricing model for, GPT-3 today’s it is still very expensive to use it. And it’s a language model. So video is way more expensive. And so, we’re less concerned about, how do we push like a field so far where it’s opened all these doors for positive expressions. And we are more of let’s be more pragmatic and like research is a product. It’s the same thing. It’s — just make sure that it works inside our environment where users can actually get value out of it. So that’s how, I guess how we want to think about it.

Ishani: I love that researcher as product. Let’s zoom out a little bit. When we look at the rhetoric around RunwayML, you talked a little bit about this confluence of code and art. And it’s not often that we see companies at this intersection. Talk a little bit about that, conceptually, what it means to you and your customers. One question I’m curious about is, you know, did it make it harder to raise venture funding back in the 2019-2020 timeframe because you are sitting at that intersection and that framing,

Cris: One thing I will clarify is, when I came to school, at NYU I went to art school. I spent two years in an art school, which is working and taking classes in computer science. It’s a unique kind of like arts program inside NYU call ITP. It’s our program that’s been running for 40 years. And it sits at the intersection of like technology arts and design. You can think about as a hacky, hacker space. You can just be there working on whatever you want, any kind of topic that involves technology and art and design and take classes from any department in NYU. And so you’re surrounded by really smart people from all sorts of backgrounds and ideas, and skills and you’re building interesting creative projects- just to just building things because you’re interested in exploring things. When we started the company, we started doing the research inside this program. It was a way for us to just have fun. We just enjoyed doing this. Experimenting with this technology, building our projects and then like showing them in galleries or in spaces or in online places. Seeing what was coming out of it, that’s what drove us. When we started, like seeing that the interest was more than just artists, but like companies and filmmakers and creators, and it was like, Hey, we should actually take this outside of an art experimental approach and productionalize it to make sure that we can deliver on the promise of transforming how content is created.

Was it challenging to raise capital at that time with that kind of like art experimental narrative? I don’t know. It’s difficult for me to benchmark because again, I was like two years in New York coming from a totally different country, culture. So, I didn’t really know at that time what raising actually meant. I was more of like, Hey, we just need to start this company. A bunch of VCs and investors had already started to reach out. So, we built a process — it actually took us like four weeks to raise. It was really fast, I think. I was thinking about the time that I never had a deck. We just showed a demo, and everyone immediately understood how it worked. I guess the advice for me from that time would be definitely to just build demos. Build things more than just decks.

Cris: Now that I look at it, and I started raising a few more rounds after that, it was interesting to see that we’re coming from a background on skills that are not common I would say most like venture-funded companies. Most of the members on our team do have an art practice or a creative background. They are artists themselves our engineers and have studied art as their primary study, and then they became engineers after. And I think that drives a few things. First of all, culture. The culture of Runway is very creative-driven, very altruistic driven, and that sits perfectly with the product we are building. Like we’re really thinking about creativity, thinking about content, thinking and about creative tools. And when you’re an artist yourself, you’re building a way for you. You know, you understand this type of user.

Ishani: You frame it as the intersection of art and technology, art and code. There’s so much opportunity as you’re articulating for the intersection of, you know, technology and X. I think that’s where we’re super excited about the next generation of applications that maybe we haven’t all thought about yet. So, we’re excited to see the success that you’ve had and all the continued progress. You know, building a culture of creativity in a technology company is inherently both easy and difficult.

And so being able to do that and then continuing to scale, it is so exciting for us to see.

Cris: Yeah. And it’s been a great way of attracting really great talent. The intersection of art and technology is something that has grown a lot over the last couple of years. And there’s a lot of interesting and talented engineers and designers and people, in general, sitting at that intersection wanting to really think about how to apply these technologies for art making, creative making. So, Runway has become like that spot where you can just come and help us and build the kind of like reality in a way. And yeah, I’m really excited to continue doing that.

Ishani: Chris, thanks so much for walking us through the business. We’re going to end the series of podcasts with three lightning round questions that have a little bit less to do with your business specifically but more about where you sit in the ecosystem. So aside from your own, what startup or company are you most excited about in the intelligent application space and why?

Cris: That’s a good question. I’m really excited about companies who are verticalizing ML, in kind of like niche domains. Uh, we started using this company called SeekOut for recruiting a couple of months ago. And it’s been so transformative for us, specifically for finding talent. I’m excited about companies like Weights and Biases as well — in terms of like research, how do you make sure that within our problem, you can help your team just move faster by identifying what needs to be done and how you can run experiments just faster. So, any company who is just seeking to like, just think about long-tailed use cases and think about optimizations so you can run with some of these algorithms or these platforms are the companies that I’m excited about.

Ishani: Incredible. And what a great segue to the fact that SeekOut is going to be our next podcast. Okay. Question number two. Outside of artificial intelligence and machine learning to solve real-world challenges, where do you think the greatest source of technological disruption and innovation is over the course of the next five years?

Cris: I guess I’m a bit biased about this, but I would say, from non-domain experts diving into like domain expert fields. The barriers of entry to a lot of technologies have considerably been lower, and so you have people who are able to build on domains that perhaps they’re not their own domains of expertise and bring in insights and thoughts and ways of working and ways of thinking that are completely new. The misfits of those spaces for me is where a lot of transformation will happen. So, I guess for us, it was like, we’re coming from an art background from a creative perspective. We’re changing how video works in businesses, right? We have so many insights and so many ways of thinking about the product and the ecosystem that perhaps people in the industry today are not really thinking of. And that’s just so — unique. And such a differentiator that I’m really excited to see more of those people just jumping in between different kinds of domains and backgrounds.

Ishani: Right, this concept of accessibility begets innovation.

Cris: Yes, exactly.

Ishani: Question number three. What is the most important lesson, perhaps something you wish you did better, that you’ve learned over your startup journey so far.

Cris: Oh, well, a lot, perhaps a good way of summarizing all the learning is, I think something I’ve learned, is that in order to just build a great product a great business is the rate of learning really matters. Like how fast you are learning as a company and as a team and as a product, how fast you are learning about your customers, how fast you were learning about the industry, about the competition, about the market, about technology. That rate of learning and how fast you can just do something you’ve never done before. Experiment with it, learn as much as possible and adapt really, really, really, really is important. And it’s something I’ve seen a lot from other companies is perhaps it’s easy to get stuck, uh, and it has happened to us as well, into something that you’ve realized you kind of like, quote know works. But then something happens and you’re not able to adapt. And so, just having that mentality of always learning — learning never stops in every single domain of the company. Always keep on learning as much as possible. And then everything else will come.

Ishani: I love that in the same way. You’re always launching your product; you’re always learning about how to build a company.

Cris: Exactly. Always.

Coral: Thank you for joining us for this IA40 spotlight episode of Founded and Funded. If you’d like to learn more about Runway, they can be found at RunwayML.com. To learn more about IA40, please visit IA40.com. Thanks again for joining us, and tune in, in a couple of weeks for Founded and Funded’s next spotlight episode on another IA 40 winner.