In this episode of Founded and Funded, Anu Sharma continues her deep dive into commercial open source projects and talks to Jeremiah Lowin, the founder and CEO of Prefect, a dataflow automation platform that is trusted to build, run, and monitor millions of data workflows and pipelines. Jeremiah talks about how he and his team built this open source community from zero to 5,000, and how the community helps the team at Prefect reach their users and intentionally break out of internal echo chambers that often trap early stage products.
To hear the full interview which goes in depth on data workflows, airflow and Prefect – download here.
Transcript (this is machine driven transcription so expect some typos)
Anu: Hi, I’m Anu Sharma with the Madrona Venture Group. I recently spoke with Jeremiah Lowin, who’s the founder and CEO of Prefect. IO. Prefect is an open-source software company that helps customers manage their data workflows. Data workflows help transform data through a series of steps and are often used by data engineers, across companies. Jeremiah came up with Prefect while he used to still work in risk management. As he built Prefect, he discovered others wanted to use it too. Now the team has an outstanding customer list and recently landed a CDC as well as a partnership with Microsoft.
In a previous episode we spoke with Joe Beda from Heptio about how open-source software can help create a thriving ecosystem around a core project. In this episode, Jeremiah speaks about how Prefect has focused on building a rich community around an open-source project. This community helps them build a far deeper understanding of their users and customers. This episode is a part of a longer conversation you can download from Madrona’s website, just search for prefect. That’s P R E F E C T.
In this shorter version, we dive right into how open-source communities help break out of our echo chambers that might trap us as we iterate on the product. Enjoy the show.
And you were just beginning to get into why it was important to not speak to yourself and get into an echo chamber. Can you say more about that?
Jeremiah: Absolutely. This is a complicated thing because I believe that open source has become very similar to how social networks would be pitched or described 10 years ago. Where you would hear these ridiculous business models like, oh, we’re going to start a social network.
And if only a million people sign up, then we’ll have a great business. And it’s like, well, yeah, obviously that’s how business works. The question is how you get the million people, not what happens after their there. Open-source pitches today have very much the same tenor when I hear them. It’s well, after we get the community, then, then this will take over.
After we do this, we’ll hire the, the dev advocates. And it just doesn’t, this is the hard part, which is you have to build that community. You have to give that community a reason to exist, and you have to do it while giving away something for free, in the open, which is a relaxation of the control that a startup with otherwise enjoy, the editorial control.
There’s a list of reasons that building in the open are strictly worse. I think that that’s an important, maybe a controversial thing to say, but from the point of view of a startup aiming to build a successful product, you would typically want to do that with a very strong product vision editorial hand on the rudder.
Not in the open where you’re in progress designs, that you don’t have time to explain context for, can be viewed and adjusted and critiqued. So why do we believe so strongly that this is a good thing, therefore? Well, it’s because you can tap into this diversity of opinions and use cases and problems. If only you can find the spark to get this thing started.
And that is, that I think is the big challenge is how do you start a community? Forget that it’s open source. How do you start a community from nothing? How do you get people together to care at all about this? And I think that there’s a unique opportunity for companies, for commercial open-source entities to do so.
Now there’s an easy answer there, which is sure, just dedicate resources to it. And we see that in the largest open-source communities. We see that in the TensorFlow ecosystem, for example, which is an open-source ecosystem, but of course, with millions of dollars being pumped into it by various stakeholders and it’s very much taken off and it’s self-perpetuating.
But on day one, what does that look like? How do you, how do you get people to buy into this? And there’s a line that Danny Meyer actually says, the restaurateur, and he was talking about the hospitality business. And he’s talking about how do you elevate an interaction away from being a transaction and into being a hallmark of an experience of, you know, the start of, of a longer relationship. And he has this line, which I, which I just love. And I think it’s absolutely true. He says, people will take as much interest in you as you show in them. And that is something that we see born out every day in our open-source community.
It’s a, it’s a value that we try very much to hold on to. If we do not show deep interest and to us, just to be clear, that means in bringing the use cases and us delivering a solution. If we do not show that interest, what right do we have to expect anyone will show interest in us?
The only way that we can do that is through a misguided belief that tech is a pure meritocracy and we’ve built the best product possible just in a vacuum. We don’t want to believe that we don’t want to rely on that sort of hubris. So instead, we want to engage our users. And what I think is very interesting is we view our open-source product.
Well, as a product, we view our product as a product is what I was about to say. We view it as a product in the same way that a company would view any commercial effort as a product. We treat it as a business, we treat it as if it has revenue, we treat it as if it should have resources allocated to it. This turns out to be a bit of a contrarian, maybe even controversial approach, but it means that we are able to make commitments to it in the same way that every startup makes commitments to its commercial product. And so, what I mean by that is if you look at many commercial open-source companies, you’ll see this intense focus on the commercial product.
And then you’ll see this effectively side project, the open-source side project with a hope that it becomes huge, but projects don’t become huge, except by accident. Products become huge. And so, whereas every entrepreneur can probably recite do things that don’t scale and go off and do that in how they make phone calls to their customers.
You see very few of the same people doing the same in an open-source context where you also need to do things that don’t scale. And an early thing that we did that doesn’t scale at Prefect is we invited our first hundred users of our open-source community into the community. We gave them reasons to come in.
We formed user councils. We held, oh, gosh, I think we had a happy hour as we flew around the country and we met with them in their offices to learn about their use cases. We made this unbelievable effort to get to know these people who we had no commercial relationship with. We just thought that they were the bedrock of the type of open-source community we wanted to start. And we wanted to get to know them, and we’d made this commitment and this effort. The next thing we did that doesn’t scale is as the community grew beyond our ability to go meet every single person, we made a commitment to answer their questions very, very quickly. I think originally, we said 15 minutes is the maximum time a question can sit. I think now, just because of the realities of a 5,000-person Slack community, growing a thousand people a month, I think we’ve, we’ve relaxed that down to 30 minutes now, but the idea is the same. We make this commitment to a completely free community. If you don’t decide to purchase an escalated support contract from Prefect technologies, you get what we would call community support. And we caveat that, we say, but it’s a really good experience because it’s still our entire engineering team, our dedicated support team, in the open, doing their best to deliver solutions. It’s, it’s a good product, but it’s a good product because it doesn’t scale.
And, and our effort therefore is figured out how do we scale it? How do we take it from that first hundred users? I remember speaking with, with Chris White, our, our CTO, as we started to cross, I think it was around 700 or 800 users. And just saying, I don’t know how we’re going to be able to keep this up.
And then here we are, you know, getting closer to 5,000 and we’re still doing the same thing. And it’s, I won’t say it’s easy. I have people on my team would murder me if I suggested it was easy, but it is rewarding, it is time well spent. So, this cycle is a difficult one. This is our path through it.
And it’s worked well.
Anu: Open-source projects and communities optimize for sometimes users and sometimes for contributors. Do you have a sense of what you want to optimize for, or maybe both? How do you organize the community to productively contribute and maybe even co-own what you’re developing as a product?
Jeremiah: It’s very challenging. We do not ever assume we will get a single contribution. And contributions are not necessarily the metric we optimize for. I think that there’s a meme that you’ll hear sometimes about open-source commercial open source, which is, oh, you’re doing that because it’s free R and D or it’s free engineering.
And this couldn’t be farther from the truth. You sometimes get very high-quality submissions and extraordinary functionalities, yes, from someone in the community. But you can’t bank on that. And you really can’t bank on that aligning with your product objectives, that it’s necessarily going to align with the objectives of someone in your community, which may or may not be your product objectives. It’s a horrible thing to bank on. Uh, and we don’t. And so, while we have, of course, it’s, it’s an open-source product, so the opportunity is there to collect these use cases. We focus much more on the network effect of discovering best practices for our own products. Funny thing to say, but because we meet people where their tools are and because we’re a framework and because in our commercial product, we, we by design never receive our customers code or data.
So, we actually can’t find out what people are doing with our tool, unless we have a good relationship with them. So, we focus very much on this network effect and the metric that I am most interested in, in our community, therefore is, the number of times someone who doesn’t work for Prefect helps someone in the community.
That to me is the, is the sure sign that people care sufficiently to assist someone with a tool that delivers a common, a common purpose, a common benefit. As we see contributors, and we do, it’s wonderful to see the growth of contributors, don’t get me wrong, but we are a commercial open-source company, we are paid to produce this product, we don’t try to extract that from our community. And I think it’s a bit of a warning sign when folks do. When people use it, when people open issues and yes, including bug reports, including their frustrations, including those problems, those are the best things that can happen to us, because those are the moments that we actually advance our understanding of our product, of its usage, of, of its shortcomings of its frustrations.
I could tell you right now, probably 200 different ways that people say Prefect created a friction for me on the way to delivering some solutions. Great. I have this list now where we can go through and check it off and make sure that our process is as smooth as possible. Whereas if we, you know, this is the flip side of the distribution model of the open-source distribution, while we can get our software to people in a certain way, we can also solicit their feedback in a very specific way, in a very public way.
And one of the things that we do the most important thing in the community is to make sure it doesn’t evolve into toxicity, and it doesn’t evolve into one of these horrible situations of, you know, one person’s unthanked and this person’s ungrateful. And we have this rule, sort of, above our code of conduct, if you will, that we assume positive intent on the behalf of all participants in our community. If you want to come into, into the community and complain and say, everything is terrible, you’re just, it’s not a great way to get help. If you come to us and say, I’m trying to do X and Prefect, doesn’t do it, and I’m trying to understand why. That’s a great way to get help because we are desperately invested in making sure that our product does everything that it should. And that positivity is contagious. And I think it’s when you don’t actively seek it the community devolves, because it is a network effect.
It is bigger than any person in it and it compounds at the, at the worst rate. So, keeping that as high as possible is just a critical part of this.
Anu: Outside of building a community, you’re running a company and growing at, I don’t know a breakneck speed, seems like. How do you make sure you’re staying on top of the decisions, both in terms of speed and quality for the company, but also for the community? Maybe, I don’t know if you tie them together?
Jeremiah: I wouldn’t say we tie them together explicitly, but we apply very similar frameworks to both. We were very fortunate when we started Prefect. We started the company because people tried to buy a prototype, which meant that the normal goal of a very young company, of can we determine product market fit, in many ways was satisfied.
Now the products that we had at the time, remember it’s a prototype, so we still had to develop that product and assure that it continued to meet that objective, but just this enormous de-risking that just took place at the company’s birth, which is just incredible. And so, what a luxury, so instead we focused on building our company side by side with building this product. And this is where, you know, for, just for example, we have this advisory board, which is very large, and this is a model that we learned about from Keith Krach and Marc Carlson, who are the CEO and CCO of DocuSign most recently, and this is a model they successfully deployed there to scale DocuSign to, you know, the height it is at today and at Ariba before that.
And it’s, and it’s all premised in this idea of building frameworks and avoiding what, in my finance days, we would’ve called pin risk, which is the idea, you know, as an option approaches expiration, especially, you know, just for example, if it’s a binary option, but even if it’s an option that’s near its strike price, you have to really worry.
If it goes a little bit up, you’re going to be in the money and it’s going to convert. And if it goes a little bit down, you’re going to have nothing. And so that’s called pin risk cause you’re getting close to the pin. So, we seek to avoid pin risk in decisions and in people. And I find it insane when startups spend a lot of money to hire a very small number of advisers because the pin risk that they’re taking, the belief that they must have that their problems will align exactly with their advisor’s experience, as well as their advisor’s willingness to help them in a necessarily difficult time, it’s not a very favorable position to put oneself and it’s certainly not a position you would pay to put yourself in.
And so instead, what we’ve done is we’ve tried to build this incredible diversity of opinions. This like constellation of people, some of whom are very obviously related to our business. They have experience in open source and technology and finance and healthcare. And some of whom on the surface looks like they have nothing to do with our business. They work in completely tangential fields or in sports or places that don’t look like they’re exactly the same, but they have deep experience in communication, in product design, in marketing, in outreach and communications, and in this host of other things. And everyone’s sort of joined by some interest in decision-making through data. That’s sort of the common denominator. It’s how we bring everyone together. And I think we have, I think we’re approaching 50 advisors now and we intend to keep growing that and we have these phenomenally incredible dinners, and we haven’t because of COVID, but pre COVID, we have these incredible dinners and just this vibrant conversation. When COVID happened, I got on the phone with 40 different people. I got this constellation of opinions and the important thing is we never need anybody to be right. We only need that opinion to fuel this framework. And then we as a company take on the challenge of running that framework.
And again, this is me talking my risk manager book just for a second, but our objective as a company is never to make the best decision we can. Our objective as a company is to never make a bad decision. And so, the way we think about that is given a hundred options, our framework is designed to eliminate 90 of them not to pick one.
So, we leave 10 options and then we pick one of those 10. And the reason we do that is we have this very sort of iterative fail fast philosophy. So, we’re going to pick one of those 10. It’s almost certainly not going to be exactly what we need, but when we do discover where it falls down, we’re going to come back to the starting line and we’re already going to have only nine options left, instead of having to come back and see all hundred. And obviously I’m sort of oversimplifying, but the result of this is that we can move very quickly, with very high confidence under extremely limited information sets because instead of focusing on picking the best outcome, we just focused on getting rid of the ones that are obviously bad. This can be a little bit of a confusing thing for people to get used to, especially in a startup culture that’s usually characterized by outliers and best decisions. In a very funny way, I think we could joke that it is probable that if you look at Prefect’s history at any given moment, in retrospect, it will always look like we made a slightly suboptimal decision.
And I view that as a win. And that is crazy to me because it means that long-term, we maximize the chance of survival, even if at any one moment we didn’t spend the extra, whatever it would have been to be exactly optimal in that moment. And that’s something that we are not only comfortable with, but that’s also a known consequence of the approach that we are taking. Is this guaranteed to work? Absolutely not. And it takes a lot of effort. It takes a lot of effort. We have to put some rules in place around how this, how this becomes possible. One of them, one of the earliest ones is a rule against something called inertial thinking, which is the idea that no idea can be justified simply by the fact that we are doing it already.
Everything must be constantly reevaluated and that helps us again, avoid bad decisions. Because whatever was the right decision at any given moment, if we could live in a world where we magically pick that, it would be insane for a company changing and growing as fast as us in a world that’s growing and changing, it would be insane to claim that that decision was optimal in the next timeframe as well.
This has been such a core part of how we evolve the company and how we move with confidence even when we don’t really know very much about the world.
Anu: Fascinating. In fact, in my previous role, I used to admire a leader and the worst answer I could give him was, when he asked a question, was this is how always been done.
Could you share an example where you converted or used this framework to make a decision? Just kind great example of how.
Jeremiah: We use this, I mean, we use this for almost everything. It almost seems silly like that I can’t even think of like a major example. One of the goals of this framework is actually never to have to make a major decision.
One of the goals is always to make a series of very small decisions so that we can fail as quickly as possible. So, it’s funny on the one hand, almost everything we’ve ever done falls into this framework. And on the other hand, it’s hard to think of like a single big bang example, but going through COVID actually would be, would be such an example.
I don’t have to tell everybody what an extraordinary period of time that was. I think it’s very easy to say that no business could claim that business was the same as it was 52 weeks ago, as we record this, right? As we went through mid-March and the realities of the pandemic set in you were insane if you just tried to continue with whatever strategy you had in place. It didn’t matter if you were a giant company and changing was hard, or you were a tiny company and resources were scarce.
You could not claim credibly that you were going to operate the same strategy to the same efficacy. Maybe you would do better, maybe you do worse, but it was not going to be the same. And so, this was a moment where certainly nobody knew anything. The future was impossible to predict. And so, we decided to adopt a framework that was kind of terrifyingly draconian.
We made a series of assumptions, almost all of which were wrong. Just to show you how effective this framework is, we decided that we had no customers, nobody was going to spend money on anything. We decided capital markets would be closed for at least 12 months. I’m trying to remember what else, what else we, what else we came up with.
There were six rules. I don’t remember them off the top of my head. Those are that those were the two scariest ones and we prepared for that world and we prepared for it in a way where we would be excited to be wrong, obviously. And of course, we were very wrong, but. But this was a case where I think fundamentally the framework that I’m describing is about the preservation of optionality and reducing the cost of failure.
And this was a case where we didn’t know anything. So, we made, we grounded ourselves in a way of thinking where the more wrong we were, the better the outcome would be. So that again, we weren’t pinned to our own beliefs. We had what I would call right-way risk and, and we also set up frameworks in which we would experiment rapidly to move forward so that we would speak with, you know. That was the point where we open-sourced a big chunk of our stack actually, at the end of March of 2020, we open-sourced a big part of our stack or that had previously been proprietary.
That was one of those decisions was like, whoa, we should have done this from the beginning. But it’s a very, very, very weighty, and difficult decision to make. We made it in that moment because it complied with the new reality that we observed. It was the right way to get our software in the hands of people who were not going to pay for it, in our assumption, but who would be benefited by having it, in our assumption.
And that was the beginning of a really incredible run for our open-source product. But again, it was rooted in a decision framework that was optimized around this objective of reduce the cost of failure, maximize the probability of survival and to iterate as quickly as you can.
Anu: That was a one-way door. Open sourcing the product at the end of March, obviously a tough one to make, and we, coming from Amazon take one-way doors pretty seriously. Do you ever fear walking into a one-way door unintentionally through a series of small decisions?
Jeremiah: That’s a fabulous question. No, I think that generally speaking, one-way doors are well enough marked. When, when you really get to a point that something’s irreversible you, you generally know you may find yourself there slightly by surprise, but, but you rarely cross that threshold by surprise.
You need to, you need to commit to that. And I think the interesting thing there is the scary thing is not so much the door, but obviously the, the one-wayness of it, right? It goes without saying. Open sourcing our product was something, you know, we, we had all these very frustrating, at the time, pushback where people would say things to the effect of, well, I know you have to run a business and that’s why you do this, but we really wish it was open source.
And what we want it to say at the time, for, at the time we had no commercial product, we had no revenue, we were just building this thing. We said, no, no, no, it’s actually not that, there’s nothing to do with a business decision right now, it has to do with, we don’t yet understand whether or not this is a good thing to do, whether or not this is a good decision in line with our product vision, and it is irreversible. And those two things coupled together, led us to take the more conservative approach. To give some context for that, I think that the strategy of open sourcing a product and then hoping your community doesn’t notice as you monetize them is a disaster from a sort of just an antagonistic business kind of standpoint.
We strive to never have antagonistic business models, and that means monetizing our community is sort of contrary to our philosophy. It also means that we can’t release a product and then try to convince people to, you know, upgrade I guess, or I don’t really know how people justify that. The other business model that we hate is releasing an open-source product and then just hosting it because we don’t think our, we don’t think our insight is knowing how to host things, right. Amazon knows how to host things. So, if you want to host something, take our product, take it to Amazon, ask them to host it, pay them to host it. Don’t pay us, pay them. Our job as a startup and as a company is to figure out what our unique insight is, what our edge is, and then turn that into a commercial product.
And so, if our unique insight was just building workflow engines, then to be honest, we might not open source it. Fortunately, that’s not our unique insight. We happen to be really good at it. But our unique insight is that we understand how to use the metadata of workflow engines, or rather of workflows, to deliver a meaningful analytic and insurance product over an entire organization’s workflows.
And so, our commercial offering is that. It’s the manifestation of this insight we have into what you can do with workflow metadata. Whereas our open-source product is this most full-featured workflow engine you’ll ever see. Because that’s at the end of the day, not something we charge for, we don’t extract a value for the fact that we’re good at that. We empower people through that.
Anu: Thank you so much. This has been a fabulous conversation. Any closing words?
Jeremiah: No, just thank you so much for having me on. I enjoy we’re recording this late on a Friday afternoon and you get me at my most rambling, but it’s a very fun space to explore. And we have a lot of hard-won opinions here. Some of which are we acknowledge are a little bit, a little bit contrarian, but have proven their efficacy over the last few years.
Anu: Love it the best way to close a Friday afternoon. Thank you so much.
Jeremiah: Absolutely. Thank you very much.
Related Insights
