Today, Managing Director Tim Porter chatted with Craig McLuckie, who’s most known as one of the creators of Kubernetes at Google. Madrona recently backed Craig’s company Stacklok, which is actually the second company of Craig’s Madrona backed (Heptio in 2016).
Stacklok is a software supply chain security company that helps developers and open-source communities build more secure software and keep that software safe. Tim and Craig discuss Stacklok’s developer-centric approach to security, the role of open source in startups, the importance of open source and security, and they also hit on some important lessons in company building — like camels vs. unicorns – and where Craig sees opportunities for founders. It’s a must-listen for entrepreneurs out there. But, before I hand it over to Tim to take it away, don’t forget to subscribe wherever you’re listening.
This transcript was automatically generated and edited for clarity.
Tim: I’m very excited to be here today with Craig McLuckie. It’s the second time I’ve had the privilege of working with Craig on a startup. So, Craig, It’s awesome to be with you here today, my friend. How are you?
Craig McLuckie: Oh, I’m doing great. Thanks for having me on.
Tim: Absolutely, It’s our pleasure. Well, I didn’t do it justice. Tell us a little bit about Stacklok and what you’re building now and a bit of the founding story, and then we’ll double back and talk more about how some of your experiences in Kubernetes and Heptio, et cetera, led you to build this company now.
Stacklok & Securing Open Source
Craig McLuckie: The story behind Stacklok is it’s a little company, a series A company, Madrona backed, that was started by myself and my friend Luke Hinds, who was the founder of a project called SigStore. The story behind Stacklok goes back several years. I’ve been known for my work in the cloud native computing space, and I had some success with open-source efforts like Kubernetes and many other projects that we built on the back end of Kubernetes to operationalize it and make it more accessible to enterprises. Open source has served me incredibly well as a professional, and I’ve spent a lot of time in open source building open-source communities and navigating those landscapes.
One of the things that occurred to me is that it seems obvious, but open-source is important. It is the foundational substrate for what is driving a substantial portion of human innovation right now. We spend a lot of time talking about generative AI, and you look at something like ChatGPT, and we dig into what’s behind the scenes, and there’s Linux, there’s Kubernetes, there’s Python, there’s PyTorch, there’s TensorFlow. All of those were precursor technologies before many ChatGPT-specific IPs even lit up. That’s a significant stack, and it’s all open-source technology.
The question that went through my mind and continues to echo with me is, we’re so contingent on open-source, but how do we reason about whether open-source is safe to consume, as a fundamental building for almost everything we consume? Historically, the currency of security has been the CVE, so there’s a critical vulnerability associated with a piece of technology. However, it’s increasingly been challenging for organizations to deal with this. My interest in this space predates things like the SolarWinds incident, which got people to think about supply chain security. This only predates the Log4J incident, which continues to plague enterprise organizations. It comes down to this: we are contingent on these technologies, but we don’t necessarily have a strong sense of the security or sustainability of the organizations producing them. We’re not necessarily creating the right amount of recognition for organizations going above and beyond to produce more open-source security.
What we’re doing with Stacklok is two things. One is we’re building a set of intelligence that helps organizations reason about open-source technology, not just in terms of whether it looks safe, whether it has a low number of vulnerabilities or static analysis being run against it, et cetera, but also increasingly whether the community that’s producing it is operating in a sustainable way or not. So, producing that signal is a good starting point. Still, we also want to make sure that on the consumption side, when organizations are looking to bring that technology in-house and continue to invest in it and build around it, we have the apparatus to drive sustainability to create control loops that enable folks that are both producing open-source and consuming open-source to institute policies to make sure they stay within the lines of safe consumption. It’s about bringing those two things together in a set of technologies backed by open-source sensibilities to help organizations navigate this incredibly critical landscape.
Tim: Stacklok is built on open-source, or at least around Sigstore for open-source, as companies and developers ingest lots of open-source, put that together to build their product, and then have to keep that updated over time. So those open-source projects that fed it continue to be updated themselves. So you get to track that whole chain. Talk a little bit more about how that works and how Sigstore plays into the strategy here.
Knowing where open-source tech comes from
Craig McLuckie: One of the most important things to know when consuming an open-source piece of technology is where it came from. It’s less obvious than people might think. If you go to a package repository and download a package, and then you install the package and run the package, you’ll look at the package repository’s metadata for where that package was published. So hey, it was built in this repo by this organization, et cetera. But how do you know that that’s the case? Because most of that information is self-reported, it’s not being generated. The answer is that folks aren’t even publishing the signatures associated with the public keys associated with signatures.
You need something that can deterministically tie a piece of software back to its origins. This is what Sigstore has done. It’s effectively a bridge between an artifact and the community that produced it, and it’s a set of tools that make it easy for a community that’s producing something to sign that thing and say, hey, we produced it, here, and this is what it contains.
Now, Sigstore is one little piece of the story. It ties the software to the origins but doesn’t tell the origin story. It does not necessarily give you insight into the community behaving responsibly or show what the transitive dependency set will look like. We’ve created this technology called Minder, which helps communities or consumers of software or producing software operate in more intrinsically sustainable ways. Let’s say, hey, I want all of these policies applied to all of the repos associated with producing this piece of technology and make sure that those repos stay in conformance.
When a new merge request comes along with something sketchy, let’s block that merge request and recommend an alternative instead. If someone misconfigures one of your repos so that branch protections are off, let’s catch that in the moment and make sure that that can be remedied. In so doing, you’re producing valuable information about how that community produced that piece of software, which then feeds the machine so that all that information then becomes a context that can get written into a transaction log like Sigstore so that the next person who is coming along to consume that piece of software now has that intelligence. It can make informed decisions about the consumption of that software. It’s turtles all the way down because when you look at a piece of software, it’s composed of other pieces of software that are composed of other pieces of software. Enabling these organizations that are in the process of composing things together and enhancing them to document their practices and write them into a record that someone can then consume subsequently is an incredibly powerful metaphor.
Tim: You mentioned supply chain security and talked about some of the major exploits that have occurred, like SolarWinds. For the audience, put Stacklok in the context of the broader supply chain security market. If it’s a market, it’s an umbrella term that gets used by a lot of different companies. It might be a little confusing for a lot of people and confusing for customers, too. Can you help frame this?
Craig McLuckie: First, Let’s look at the security landscape and consider how the world has worked. Historically, it used to be a world where hackers were effectively like burglars sneaking around your neighborhood, and they would look for an unlocked window and then open the window and sneak into your house and steal your silverware. That was the world that existed.
With the SolarWinds incident, we’ve seen that it’s not enough to lock your windows and turn on the alarm system with some of these active security practices. The burglars are now getting jobs at window manufacturing companies or are breaking into window manufacturing companies and tampering with the apparatus that produced the latches on windows so that the windows that are being produced and installed are intrinsically insecure, so when everyone’s out at a town banquet, they can go and clean out the entire neighborhood. That’s the sea change that we’re seeing in the security landscape.
It’s not enough to look at a piece of software today and say, hey, this thing has no known vulnerabilities, it’s good. No known vulnerabilities don’t necessarily mean anything. It could be that it’s good, or it could also mean that it’s perfectly malicious, meaning it was produced by someone sufficiently sophisticated to make it look exactly like what you want to consume. Still, they added a little naughty something that will ruin your day when it gets deployed into your production system. This idea is about understanding and knowing where your software is from and its origins, like understanding the farm-to-table journey around your software.
In terms of our positioning, where does this start? It starts with your developers. It’s not enough to say we will insert this into the production environment because, by the time you put something as a control in a production environment, your organization is incredibly invested. Failing a quality scan in a production environment is the first time you discover a Log4J instance in something you’re trying to deploy. That’s very painful because you must return to the development team. You have to figure out who produced that thing and go back to them and say, hey, this is not okay. They then have to go and reproduce it, revalidate it, redeploy it, and get it to you, and it takes an inordinately long time to deal with.
You want to intercept developers not just at the time when they’re submitting a merge request but when they type that import statement and give them signals saying, yeah, what you’re doing is good, or, what you’re doing is probably going to get you hurt later down the pipeline. Next, instituting further controls, starting with the Git repository, moving into the pipelines, the container registry, and the production environment. You have these controls along the entire journey, and you can look back on the farm-to-table story of a piece of software you’re looking to deploy.
The Role of Developers in Security
Tim: You mentioned being focused on the developer. Stacklok’s a very developer-first, developer-centric company and product you’re building. A lot of times, you think of security software as something that’s more top-down; it’s a necessary evil, and it’s being imposed on you from above within management. Talk more about why it’s important to start developer-first and how it can become something equal that a developer wants to embrace and ultimately create a better experience for their users.
Craig McLuckie: One of the things that is true about developers is they generally want to do the right thing. It’s not like developers are sitting there saying, you know what? I want to produce insecure code, or you know what? I want to mess with my ops team’s day and produce something insecure. They don’t want to go back and forth with the operations team, but it’s also worth recognizing that, at the end of the day, they’re primarily being metric-ed on producing software that creates business outcomes. The thing that’s going through their head is that I need to create customer delight through my code, and in creating customer delight, I’m going to create money for the company I’m working for. If you’re starting to produce capabilities that inhibit that, they may still want to do the right thing, but they will find ways to work around you.
The later you leave the discovery of something in the left-to-right flow of a developer’s workflow, the more intrinsically painful it’s going to be, and frankly, the less likely people are going to be to accept what you’re doing. That’s from the pragmatic side of getting this technology adopted. This is a constant challenge for the CISO, which is, hey, I want to introduce this, but the minute I do, my business counterparts yell at me because I’m slowing down their production world. So starting with technology that’s developer-friendly and developer-engaging is a good story.
Now, it’s important not to confuse developers as buying centers. Developers don’t have budgets by and large, but increasingly, when you look at the IT landscape, developers are just a disproportionately significant cohort, not because they’re necessarily going to buy your technology directly, but because all of the people that do buy your technology care about them, and want to enable them, and are going to see you as being a more intrinsically valuable capability if you appeal to the developer.
To Open Source or Not
Tim: A lot of the listeners are thinking about, hey, I’m building a new company. Should I open-source? Should I not? How do I capture developers’ attention, hearts, minds, and community? I’ll back up just a little bit. So Craig, you co-created Kubernetes at Google and left a while later and started Heptio, which was all around helping enterprises adopt Kubernetes. Kubernetes became arguably the most popular open-source project in the world. There was a point where only Linux had more daily interactions than Kubernetes. Of course, then at Heptio, you built it for two years, sold it for 600 million to VMware, and it continues to live on there. I know there’s a lot here and a lot of founders come and talk to you about it, but what are some of the principles that you thought about in building out Kubernetes and now are thinking about Stacklok or other companies that are focused on developers and using open-source to build this adoption flywheel and community?
Craig McLuckie: There’s a hard truth here, and this is important for the listeners to embrace, which is that open-source is effectively mortgaging your business future for lower immediate engagement costs. So, you get lower activation energy. It’s much easier to get the flywheel turning with open-source particularly if you’re a small company. it’s going to create a virtuous cycle with a lot of the individuals that you want to engage. They may even contribute directly to the project. They’ll certainly make it their own. They’ll give you feedback in near real time; they’ll build the project with you. It’s a wonderful way to build technology. It reduces your barriers to entry in enterprise engagements, particularly if you’re a rinky-dink little startup. If you have an Apache-licensed open-source project, particularly if the IP is Foundation Homes, you’re far more likely to get through procurement because, at the end of the day, they know they can just run it themselves if things get weird. So your path is easier.
When I was thinking about Kubernetes, what distinguished us at Google was that I didn’t need to commercialize Kubernetes. I just needed it to exist because I had something to commercialize it. It was called Google Compute Engines, which had decent margins. I needed to disrupt my competition with the technology to run better myself on Google’s infrastructure, which motivated us to drive the Kubernetes outcome. With the success of that project, Heptio was enabling enterprise, this idea of enabling enterprise organizations to achieve self-sufficiency, to bridge the gap between them and the community to fill some of those operating gaps associated with the consumption of the open-source technology, and that worked out well.
When I look at what I’m doing with Stacklok, I recognize that over time, if I’m successful with the Minder technology, I will have to accept a lower value extraction narrative than if it were proprietary technology. But realistically, the probability of me succeeding and getting something that’s consumable out there is far higher if I can embrace the community. You have to have a plan. What is your plan for commercialization?
In the case of Stacklok, I’ll be very open with our customers and the community. Our plan is to create incredibly high-value contextual data, which we’re manifesting as a trustee right now, to support the policy enforcement that Minder has. That represents a differentiated thing. It’s hopefully something that our customers will value over time as we bring more and more richness to that data set. It represents something that’s differentiated from the community open-source technology. That’s my broad plan. I’m very open; I wear my heart on my sleeve. I don’t plan to change the licensing model. I have to stand by my commitments to my customers and to the community that I stand by. But the point is, I do have a plan for commercialization. It’s not just, I’m going to be the RedHat of this thing because it turns out RedHat is a pretty singular creature.
There is another story here, and we see a lot of this, which is that people are happy to pay you to operationalize something. If you have built a system that’s reasonably complex and you’re able to operationalize it better than other people. You’re able to set the boundaries of where commercial starts and ends and open-source starts and ends; you can navigate building a SaaS business around a single vendor, open-source technology. We’ve seen great companies emerge in that sort of context.
Tim: You built Heptio over a couple of years and it was bought, great journey, faster than probably anyone anticipated when you started. What were some of the things you learned? It could have been about some of these open-source threads we were just talking about, or there’s a whole host of other just general building startups successfully. What things did you take from that experience that you’re making sure to bake into Stacklok? Were there any things that you didn’t want to repeat having done it once before?
The Importance of Culture in a Startup (Camel v. Unicorn)
Craig McLuckie: I enjoyed the Heptio journey, and it’s hard to complain about the outcome. It’s also worth recognizing that we were riding on the momentum of Kubernetes. It was a very luckily timed undertaking. I don’t claim to be able to create that kind of luck for myself again. We need to approach this from a bottoms-up perspective.
What is different with Stacklok is that I definitely had a bit more of the unicorn versus camel mindset. A common narrative around my leadership circles is no, that’s a unicorn thing; that’s not a camel thing. We’re camels, not unicorns. We’re building something that is incredibly lean and purposeful, that’s going across the desert to an oasis, and if we don’t make us die, that’s how we think. We know we have to get this far on this much money, and there are no alternatives.
Opportunistic hiring that’s a unicorn thing; that’s not a camel thing. Getting crazy with swag, that’s a unicorn thing, not a camel thing. I think one essential delta is just a reality of the environment that we’re in. It’s like you have to have a plan for commercials. Like the days of being able to raise on promises and a winning smile are over, you better have a business plan. We’re more thoughtful in terms of our approach. We’re much smaller than I was with Heptio at this point. We’re twenty-one engineers, but they are crazy efficient and really hardworking. I’m very proud of our two products in nine months. So that’s the one difference.
The second difference is that I’ve approached this with what I think of as a hardwood growth model. Heptio was like a linear growth model, resulting in a lot of fog of war. We struggled with folks feeling disconnected because we were growing so quickly. What we’ve instituted with Stacklok is a grow hard and grow hard and grow hard, like a growth execute and run the team really hard, produce an outcome, then grow, run the team really hard, produce a result, then grow. I haven’t seen that referenced or talked about much. Still, I’m finding it to be incredibly beneficial just to the culture and to the organization because you establish the team, by establishing the norms. Then you use those norms to direct your next wave of hiring.
The other important thing is that in Heptio, we were a very culture-forward company. I think that I’m retaining that culture-forward perspective with Stacklok, but I’m now very purposeful in this case that the culture isn’t the mission, the mission’s the mission, and the culture supports the mission, and making sure that as we embrace our hiring practices, we’re very, very diligent about not just hiring the kind of people we want to work with, but hiring people that have that sense of purpose, that sense of mission, that willingness to engage in the way that we need to. That camel mindset is very powerful.
Tim: I love the camel versus unicorn mindset. It’s absolutely essential in today’s market. The days of the unicorn are distant memories in many respects. You’re talking about culture and how you’re really intentionally and thoughtfully building it at Stacklok. You wrote this great blog early on in the company called The Operating System of the Company. I like your point about how it supports the mission but is different from the mission. You have these great company tenants that you sort of alluded to. Maybe just say more about how you’re building culture and how you thought about those company tenants. This is the most common thing between all startups and all founders, is that you’re building this culture, building it right for authentically you and your team, how that carries forward for the long run in the company’s just absolutely essential foundation.
Three Jobs of a Founder
Craig McLuckie: Yeah, I heard a founder, say this, and I’ve adopted it as my own story.,which is I have three jobs, right? I have to effectively set the strategy for the company, I have to execute the strategy for the company, and I have to define the culture of the company. And of those, the third is probably the least understood, and most important. So culture for us is it’s our operating system. It’s not just the warm and fluffy things you write on the whiteboard to make you feel good about yourself. It defines how we hire and how we make decisions, and It’s indistinguishable from things like our brand. Our culture is our brand. Our brand is our culture, it defines us in ways.
In my previous company, I built with Joe Beda, and Joe and I are old friends. We’d worked together for ten years and done impossible things together. You would not find any air between us. You could present us with the same data, and we’d always formulate the same conclusion, which was easy. So when Joe and I started Heptio, we first sat down to write the culture and then went off.
Now, with Luke, an amazing human being whom I’ve come to respect in the most fundamental ways possible, we were relatively new. We had yet to work together. We had spent many hours together and gone hiking but had yet to work together. So, we both did this exercise and wrote down what we believed, just drawing on our lived experience. These are the things that define us and how we make decisions. Those little statements Luke says, like you always have short toes. You can’t step on my toes because I have toes. You can say anything, and I’ll just take it at face value. I’m not going ever to take defense. So that’s an example of something he came up with.
We both wrote those down and then I did a day exercise where I built an affinity map and went through all of the tenants, everything that Luke and I felt. I tried to distill it into a set of five things that I could then articulate that would represent what I believed to represent the way we operate because you can’t fake culture. The minute you try to fake culture, at some point, a hard decision’s going to come up, and you’re going to make a decision that’s not aligned with your culture, and then hypocrisy is going to creep and your culture’s dead and you have to start from the beginning.
I wrote down the five tenets. One: We’re a team. No one gets lifted by pushing someone down. The organization is invested in community-centricity. Community is a symmetric advantage. Two: We’re mission-oriented, we’re a startup, and we’re a camel-based startup. There will be some long, dry patches, and you better have the world to make it to that oasis. The only thing that will get you there is if you believe in the mission. I want people that feel that burn, that really want to engage in this mission with us. Three: We’re a startup, so we have to be data-centric and find truth in data. We cannot gloss over a hard truth staring us in the face because we refuse to engage with and believe the data. Four: We have to be humble but determined. The camel is a humble but very determined creature. Some of the best leaders I’ve ever worked with have embodied this. Five: We’re a security company, so we have to stand vigilant. Bringing that all together and then operationalizing it. When discussing hiring candidates, guess what exists in our candidate management system? It’s these five cultural virtues. Our interviewers should provide feedback on those five things. We talk about how to assess this in people that we’re talking to. When we’re talking about promoting individuals, we reference back to the cultural elements. Whenever you’re having a complicated conversation, and you have to make a decision, you point back to these tenets. So these are the things that are informing our decisions. That’s how I’ve approached building a culture.
Tim: Fantastic articulation. It’s energizing to hear you talk about it, Craig. Of course, you referenced your fabulous co-founder, Luke Hinds, and he was a long-time senior engineer at RedHat, co-creator of the Sigstore project that Stacklok is drawing on significantly, and is also based in the UK. So you also have a bit of a geographic/ time zone that you’re working through, which in this world where companies are at least to some degree hybrid, this cultural blueprint and operationalizing it, I think is that much more critical, compared to when everyone can just be together in the same office.
Craig McLuckie: In this kind of remote first world that we find ourselves living in, canonizing that and expressing it and being very deliberate in your communications is so critical as a startup founder.
The Future of Stacklok and the Cloud-Native Ecosystem
Tim: Awesome. Let’s come back one more time to Stacklok. So you just formed a company and founded it earlier last year. An incredible pace of building this great team of predominantly engineers that you referenced, launched the first two products which you referenced Trusty and Minder. What’s up for 2024? Maybe without giving away anything confidential, what are you looking forward to from a company’s standpoint this year? If there’s a call for customers, who should be interested, what type of developer or what type of enterprise, the developers there should be looking to Stacklok for help in the areas that we were talking about earlier?
Craig McLuckie: There are a couple of things that we’re looking to bring to market. One is, there’s Minder, which is a system that helps you mind your software development practice. This year, we want to help organizations embrace and engage with Minder. We want to make it free to use from an open-source community perspective. We want to support and engage open-source communities that are looking to improve their own security posture by providing this technology. They should feel like it’s, hey, Stacklok’s bringing us something that’s really useful. So if you are an open-source project and you have, say, 500 repos, and you’re worried about licensing taint showing up for some purpose. Having something that you can run reconciliation loops across 500 repos and reason about the trans and dependency set and make sure that nothing’s showing up in your world that shouldn’t show up, is useful. We want to engage with communities to help support their use of this technology.
We think of this as a way to give to these communities to enable them to start operating securely but also to be able to show their work so that they can produce Sigstore-based signatures and generate the attestation signal that the people who are looking to use their projects are starting to expect. We’ve focused almost exclusively on GitHub integrations, but through the year, we’ll add other critical integration points, such as Bitbucket, GitLab, Kubernetes, and pipeline integration.
On the other hand, we wanted to start the flywheel going with some high-value information. So, while Sigstore’s gathering ahead of steam and these communities are taking time to start producing consumable attestation information, we wanted to precede the segment with some very high-value intelligence. We started by doing some statistical analysis using Trusty, which is data science against open-source packages, looking for signals that tend to indicate both vulnerabilities and health. You can expect us to continue to enrich that. I’m not ready to announce, but keep watching the space. We’ll start to introduce some very, very sophisticated and cool ways of thinking about open-source technology that complements a lot of what’s already out there in the ecosystem. We’ll make that all well integrated into the Minder capabilities so that you can start to define and enforce policies based on those signals.
Tim: Awesome, looking forward to this year and beyond. A lot of people wonder your point of view on Kubernetes and the Cloud Native ecosystem. You’re firmly focused on building the security company now. Of course, lots of interrelated work with development that takes place around Kubernetes, but what’s your view on the state of that community, Craig? Is it at a maturity phase? All the big companies have their hosted and managed services. Do you still see room for more innovation for startups broadly across the Cloud Native ecosystem? Are there any pain points that you’re continuing to hear about? Any advice for founders who are looking to continue to build in your old world?
Opportunities for Founders
Craig McLuckie: There’s tremendous opportunity in that space. Call me crazy. I don’t want to be that ’80s band with that one hit song, like singing at corporate events for money in my 70s. I want to branch out and try some new things, and the supply chain stuff has been something I’m very passionate about.
Honestly, I think one of the things I’ve observed is bringing platform sensibilities to the security space, it introduces a very novel way of thinking. Look at the security ecosystem, why isn’t everything just running reconciliation because they work so damn well for platform sensibilities. It’s like, why is this not just a Kubernetes-esque integration platform? Why doesn’t this exist? We should just build it.
I think there’s a lot of work to be done. I mean, obviously, generative AI is hot, and from firsthand experience of building and running large language models that we use behind the scenes to produce some of the value that Trusty offers, there’s a lot of fragility and brittleness there. I think there’s an almost unfettered demand for capabilities to simply operationalize and deliver as a service-like experience for some of these community-centric large language models. I think in the Gold Rush of generative AI, the operationalization pickaxes are going to sell very, very well. I think that’s something that I would certainly be interested to see where it goes. I would certainly be looking to consume that myself because right now, we just find that operationalizing those models is brittle and can be a bit of a challenge.
I think there’s still unfinished business in the gap between the platform as a service and the Kubernetes layer. So, the gap between Heroku’s and Google App engines and the pivotal cloud foundries and the world of Kubernetes still exists. We haven’t seen a lot of really great turnkey experiences. I think the work that we started doing with the Tanzu portfolio was a nod in the right direction, but I definitely think there’s a wonderful opportunity to continue to explore and play with the idea of the service abstraction and basically looking at service dependency injections for modern workloads.
I’m also acutely interested in the way that WebAssembly is going to start to shape up and represent ways to bring new atoms of computational resources into new form factors using and borrowing a lot of the distributed system sensibilities that Kubernetes created. I think there’s tons of opportunity. If I hadn’t met Luke Hinds and fallen in love with supply chain security, I can think of three or four great startups that I’d be happy to do tomorrow, but I’m very happy with the one that we’re doing.
Tim: We’re as well. Thanks so much, Craig, great insights. We could talk about these things for days, but I really want to thank you for spending time and sharing these insights. It’s just a great pleasure and fun to be able to work together here and do a little bit to help you and Luke and the team as you build Stacklok. So, thank you.
Craig McLuckie: Thanks, Tim, really appreciate it.