Today we’re bringing you a special live episode of Founded & Funded, featuring Madrona Managing Director Soma Somasegar and Jay Parikh, Executive Vice President of Core AI at Microsoft.
Jay leads the team responsible for Microsoft’s core AI stack, the systems that power Copilot, the tools developers rely on, like GitHub, and the infrastructure that makes large-scale AI possible. In short, his group builds the underlying tech that Microsoft and thousands of companies use to create AI-powered applications and agents.
In this conversation, Soma and Jay dive into what Jay calls the Agent Factory — a new paradigm reshaping how software gets built in the reasoning era. They explore how AI changes the development lifecycle, why observability and evals are becoming mission-critical for enterprises, what it means to collapse traditional engineering functions, and how organizations should prepare for a world where models, agents, and human builders all collaborate in real time.
Listen on Spotify, Apple, and Amazon | Watch on YouTube.
You can also read Soma’s takeaways for founders here.
This transcript was automatically generated and edited for clarity.
Soma: Thank you for being here today, Jay.
Jay: It’s a pleasure. It’s a pleasure.
Soma: It’s really a pleasure to have you here. We’ve been sort of spending the whole day here talking about how we are in the beginning or the early days of a reasoning revolution and we sort of think about the industrial revolution and fast-forward to today, to me the big difference between when we say Industrial Revolution and when we say reasoning revolution is the machines that were part of the industrial revolution where right from day one, I should say, they were a depreciating asset, whereas reasoning machines, I think from day one, it’s an appreciating asset. And it’s sort of the flywheel that happens with data that makes it more powerful and appreciating in value. And so I look at the opportunities ahead, it is phenomenal.
As I was at Microsoft for many years before coming over to the side of the house, one of the things that I remember vividly is what Bill used to talk about in the early days, in the early days of Microsoft, it was like, “Hey, we want to build a software factory.” And then I heard you talk about this a couple different times about, “Hey, this is now the time for us to go build an agent factory. If you think about it in today’s day and world with agentic AI, can you tell us a little bit about what you mean by Agent Factory and how that’s going to change the world that we live in?
Jay: Yeah, so fun story, I didn’t actually know that Bill used to talk about Microsoft as the software factory. So my one-on-one with Bill, my first one was I was telling him about this concept of the Agent Factory, which I’ll answer your question, he’s like, and he stopped me. He’s like, “Jay, you know when we started a company back in 1975, I always had this,” and then he went on to explain the software factory thing. And he was very exuberant and excited, but I was like, “Wow, 50 years later, here we are, and we’re talking about the Agent Factory. So I had no idea about the software factory, so I kind of sketched this out in a completely different kind of time. And it was just fun to hear that original inspiration and vision for the company.
So the idea behind the Agent Factory, I think originally I thought about this internally in terms of how we need to evolve our infrastructure, our platform, our tools, and effectively how we make products in the next era of Microsoft. And then I realized, as we were cooking this and thinking through, it’s a cultural thing, it’s a technology thing, it’s an incentive thing, it’s a systems thing, that I actually realized that every company that isn’t maybe founded today is going to have to actually create a similar system.
And the idea is, I mean, honestly, I drew it out, and it’s literally like, I did it in ASCII art for the team in terms of how this looks. And there’s these things that I envisioned showing up on the loading dock, and they’re like models, they’re different things, there’s technology from MSR that shows up here. And then we kind of need to put those together into a production line. Then at the end it produces some AI capability, which today likely is some type of agent, whether it be a coding agent, a code review agent, a biology agent, a fraud, something, something agent. And we’re building those agents internally for our core products, but as more and more customers adopt different parts of our tools, our products or platforms, they are enabling their business or transforming or changing their businesses to be building their own agents to transform their organizations, their business.
Some of it is about time savings, but I always push customers to really raise their level of ambition in this reasoning era, to use your talk track here, to raise their level of ambition of the types of things they’re building because we’re on an exponential and we’re all terrible about understanding what’s actually possible with the collective set of intelligence that we have in all of these models today.
Soma: As Microsoft, as a company that is sort of in the forefront of AI in a variety of ways, how do you think this concept of Agent Factory is going to change how Microsoft operates and how Microsoft is going to be building software?
Jay: I mean, the way I think about it is what we’re doing in our team with several others is building the future of Microsoft. I think it is going to change everything we do.
And yes, we are a 50-year-old company and there’s lots of technology that’s there classic and is going to be running for a long time and we’ll continue to optimize and support those products, but everything that you see going forward from the company, whether it be knowledge worker in M365 products to security, to platform, to tools, to the health and life sciences, to the focus of research, to just keep going, I think is all going to be around this concept of really unlocking, what can you do to drive productivity, drive creativity, drive collaboration, solve these technical or science problems that have been eluding us for decades faster now because we have these frameworks, we have these tools, we have these models, we have the ability to get data, synthesize data, do this flywheel of reinforcement learning, and I think we’re still early in terms of how that is going to manifest itself in both Microsoft’s products, but then how do we enable those capabilities in our platform, our tools, our data, our systems, so that every organization out there can use these to really unlock that in their own enterprises.
Now, the fact is, there is inertia. In organizations, it takes them time to adopt this stuff. And so it’s not happening overnight. The technology is way ahead in terms of capability than most cultural transformations will take in an enterprise. So there is that, we don’t have time, and everything is moving fast, but these organizations, the larger enterprise organizations, global 2000, they are slower to adopt all of this and change.
Soma: Great. All kinds of numbers get bandied around that, Jay. The most recent thing I heard that was inspirational, and it feels like, as much as I want to see that happen today, it’s probably a little bit in the future, was where Dario at Anthropics said, “Hey, by the end of this year, 90% of all code is going to be generated using AI.” So if you have to pick a number for where Microsoft is in the journey, just internally from a development perspective, how would you characterize that?
Jay: Yeah, so I’m going to give you a non-answer, but in answer, and the way I think about this actually is these numbers around how much code is written by AI is largely uninteresting to me having run and scaled some of the largest engineering teams because I really think it’s more about the capabilities it’s creating or giving to builders in the company. So to me it’s less about lines of code that AI is generating, it’s really a few things.
One is of our run the business stuff, of our technical debt, of our operational stuff, things that we’re optimizing or we need to, technical debt, upgrading frameworks of stuff, fixing security, improving performance, there’s a lot of toil that engineers spend in organizations dealing with that. And I can say, “Hey, we have X percent of our code being generated by AI,” but who cares because I still have engineers toiling away at all of this tech debt and this other stuff and bug submissions from customers.
So what I want is to shift that to, hey, this run-the-business stuff, this technical debt stuff, this bug fixing, this… In some ways, there’s true toil there, but there’s stuff there that is just holding these very talented people back from achieving higher levels of creativity and collaboration. I want to shift that dramatically and I want to shift the amount of time that we spend, classic engineering time, whether it be in meetings or upgrading stuff, fixing vulnerabilities, pushing stuff to production, optimizing some performance bug, I want to squeeze that with AI, I want to get that down so then we open up and we give back time for the creativity part.
So we actually have an initiative inside our team that we run, and we’re scaling across the company called Engineering Thrive. And we actually do measure the time when engineers are stuck. So largely there’s unfocused time, there’s focused time, and some other categorizations, and we’re watching longitudinally as we make cultural shifts, as we improve the tools, the technology, and even just the know-how, the skilling of our technical organization, how that percentage is, how they shift. Because, ultimately, if I can get more builder time back, more creative time, or focus time, then we’ll just accelerate our ability to build great products and to bring more value to customers and all of that good stuff.
Soma: That’s great.
Jay: There’s one other thing I would just say from a macro perspective, I would say we’re at a point today where if you think about it, if you add up all the software that’s ever been written in humankind, we are probably sub-one percent of what’s going to be written in the next 10 years. And I think that’s the more interesting thing, of how much more we’re going to be able to do, solve, create, collaborate, because now we have this superpower that’s getting better and better every day in terms of being able to build, prototype, solve these types of problems.
Soma: I should tell you this little anecdote thing because it involves GitHub Copilot. This was probably a year ago, or so, I was talking to Marco Argenti, who was here earlier today, the CIO of Goldman Sachs.
Jay: I know Marco.
Soma: And he was telling me that, “Hey,” because he had deployed GitHub Copilot across his organization, I asked him, “So what are you seeing in terms of productivity benefits?” And this is within six months of him having deployed GitHub Copilot, and he said, “Hey, I can, without thinking too much, I can tell you that my workforce is 20-25% more productive.” And I asked him, “So that means you’re going to fire 25% of your organization tomorrow?” He said, “No, no, no, we don’t think that way. The interesting thing is that it has made my job easier in that I go into a meeting now knowing that I’m not waiting for a shoe to drop at the end, where people come and tell me exciting things and then say, ‘Here is a headcount bill.’
Now, instead, it’s like, hey, that person knows there is headcount available, meaning resources available in the team, because of the productivity benefits, I know that, so there is no headcount corner. It’s all about what do we want to prioritize? What do we want to do? And so to me, I think that’s an exciting part of what AI is able to do for all of us.”
Jay: I agree with Marco on that.
Soma: Microsoft is one of the companies, Jay, that is spending on AI infrastructure as much as anybody else, I would say, if not, maybe more.
Jay: Maybe more.
Soma: This year, the stated dollar amount from Microsoft was $80 billion or whatever. But even apart from the dollar amount, the Azure platform, the Azure AI platform, the Azure AI Foundry, all the tools, all the developer stuff that you guys do, you have a fantastic platform for AI builders or AI developers. But having said that, and having made all the progress that you’ve made, what are the two or three things that you think Microsoft ought to be doing more, whether it is from an operational perspective or a technology/product perspective or even a cultural perspective, to position Microsoft in the best possible place for an agentic AI world?
Jay: The thing that I would say is with our team and our focus here, and some of you were, I think, supported many of these teams when you were at Microsoft, so you know them well, is really putting together this full-stack approach to how the future of software development is going to look. And it’s starting at the core, at the infrastructure handling and evolving how the workloads are going to change and scale. Then it’s the platform in terms of these AI applications, the agents, remember for decades we’ve been building software where we go interview our customers, we come back, we design some schema, we write some business logic crap in front of it, and then we put some UI around it. And that’s what we’ve done in terms of software. And we’ve built a lot of incredible stuff over the decades.
Now though the entire, it’s not even that the paradigm goes on its head, it’s like a completely different alternate universe because now you have these models that can think, they can plan, they can reason, they can call different tools, they can collaborate with each other, they can do things faster, better, sometimes slower and worse as well. And then you have to think about the scaffolding that sits on top, below left and right of these models, from a, what is software, and, what is this new era of applications and software and agents, whatever comes after agents is going to look like. So it really is actually a completely different stack, and that’s why we put this team together, it’s really, first principles, think about the [inaudible 00:16:35]], the platform, the tools, security and trust and everything.
Now to answer your question, it’s really important, especially in the tools and the platform part of this where… and my philosophy is that we have to do this… Sure, we’re going to build a glorious platform and it’s going to be delightful, but I actually believe for builders, for developers, for scientists, whoever is building on this platform, you also want choice. So having a vibrant ecosystem of different partners is incredibly important to our strategy.
Now, the things where I think the world and we need help with is, some of this was covered, I’m sure today is, I’m getting into a very specific example here, I’ve been saying this for a while, but I think it’s finally starting to be more of the conversation, I think evals are going to be more and more invoked in terms of what everybody needs to go solve. And I think there’s some clever startups out there tackling this. We’re all trying to tackle it. Everybody’s struggling with this. Marco is struggling with it. I struggle with it. Because there are benchmarks and there’s, what I say is one-dimensional evals, and we’re pretty good at that.
But humans like the lived experience, they’re so personal, they’re so different. And then you can have a set of evals, build your application, evaluate some models, be like, “Hey, we got these scores,” et cetera, et cetera. Then you put it in the hands of your customers and they vomit all over the experience and you’re like, “Wait, my evals are great, look at my scores.” And they’re like, “Yeah, this sucks. I don’t want to use this thing.” So that there’s a human touch or a three-dimension, four dimensionality to the lived experience of these products, and I don’t think we’re still really, really, I think basic and not great there.
The other area that I think is, and it comes up in every conversation when I talk to the enterprises, nothing in AI is going to work in the enterprise without observability, and I know there’s some great startups out there doing cool stuff out there. We invest in this a ton, there’s a bunch of stuff that we’ll be announcing through the end of the year on observability orchestration, but I think this is also one where we want, and I expect a vibrant ecosystem of startups out there to be building these really sophisticated and awesome observability features, products, whatever it might be, because that’s going to help really drive the diffusion of this technology in the enterprises.
Soma: When you talked about-
Jay: I have a long list today. Is this my shopping list?
Soma: When you talked about a vibrant ecosystem, one of the things that I think the world knows this now, that Microsoft gets a lot of credit for the partnership that you guys struck with OpenAI a handful of years ago. I think it has served Microsoft really well. It has served OpenAI really well. It has served the world really well.
But having said that, I would say in the last few months, at least in the last few months, maybe even a little longer than that, we’ve started seeing Microsoft come out and say, “Hey, for this part of Microsoft,” meaning say for office Copilot or for some other thing, “Hey, we might take advantage of different models, whether it’s Anthropic or what have you,” which is fantastic. And Microsoft was one of the first companies that came and said, “Hey, we really want to build a platform that delivers model as a service,” which is great because it’s all about a vibrant ecosystem and embracing the ecosystem. But having said all this, what do you think Microsoft should be doing? Maybe it’s already doing some of it in terms of having investments and building first-party large language or frontier models. Is that important or not important?
Jay: So there are three things. One is our partnership, and I think history with OpenAI is still incredible for both companies and we invest both sides a lot in lots of different things, because the world is changing, the products are changing, the risks out there are changing, the business models are changing. So there’s a lot of collaboration across every level of both companies, and that continues to be a huge focus, and I think we’re both very happy and there’s lots of stuff to do, and we can’t get to everything, so I think it’s, one, it’s a very valuable relationship. It’s a very, I think, positive relationship.
The second thing that I would say, and you kind of gave the answer here, which is building these platforms, building these tools, it’s really important that we also meet developers and builders and knowledge workers where they are and what they want. So being dogmatic about, “Hey, we’re just going to have only one choice, or one model,” that’s not what people want. And so our strategy I think has always been, but I think it is way more noticeable now where there is that choice. I mean, GitHub Copilot has always offered the best models, irrespective of who the provider is. We offer Gemini, Grok, OpenAI, Anthropic, our own models, etc., in the model picker. And then in Foundry, we have thousands of models that somebody can pick up and use to build what they want, open source, closed source, it’s all there. And we’ll continue to invest in that breadth of choice.
And then I’d say the third part of the strategy is our own models. And we released, I think probably three or four weeks ago, our MAI models. There’s an initial set of models that have been built, trained all inside of Microsoft that we’ve released. There is more coming along that frontier, investment there. We have lots of other models that have come out of Microsoft research. There’s models actually that are Microsoft-trained models that sit inside of GitHub Copilot that most people don’t know today. And then there’s stuff that’s in M365 that’s also not one of the companies we’ve already talked about.
So there is all of that going on. And it is hard, honestly, it’s sometimes confusing that there is so much choice, but we’re all figuring this out and we do have to listen to our customers in terms of what they want, and we have to provide the best because everybody wants to tune and I think optimize for different things, whether it be quality or cost or performance or this thing that works in a specific region or in a certain language or a certain discipline better than another model.
Soma: Why don’t we turn it around and see if there are any questions from the audience for you, Jay?
Question: So I really just, as a point of clarity, when you say observability, is that also auditability, or are they merely observable in transaction, but don’t necessarily lead to an audit trail?
Jay: I’d say observability in the royal observability, I think it’s a, depending on the enterprise, observability is the bigger category. Inside of it, you can say there is monitoring stuff, there’s traceability stuff, there’s compliance, there’s audit, there’s even some security tie-in right there. So it really is I think, an expansive term and I think all of that falls within the comment I made earlier of where enterprises need help and they are evolving, but this is an area where whether it be the monitoring platform for monitoring usage and cost and performance and that stuff to having something that will do the audit so that you can hand this stuff over to compliance folks, et cetera.
So they’re somewhat obviously different functions in a company, but from an observability perspective, you need that data, you need that correctness, you need that evidence. And as the regulatory environment keeps changing on the compliance and audit side too, there is going to be a whole set of shifts that are happening there, but the data that we get has to be made into these different outcomes. But I think we’re really early on all of that, super, super early.
Soma: Great. I think there’s a question here.
Question: I’m curious to hear your thoughts on, you talked about the model picker. So with GitHub, with Cursor, with a lot of different tools today, there’s lots of different model options for the end user. Do you think the fascination with choosing your own model dies down as AI becomes more commonplace? I think of this as the electricity example. Edison built the electric bulb, everybody wanted to know how it worked. Today, when you flip on your light bulb, you don’t even think about it. I’m curious to hear your thoughts, do you think as models or will models become commoditized to the point where the end user doesn’t have to have the burden of thinking about what is the best model to use for what task and applying that also to the developers?
Jay: I think there will always be a need for developers to kind of go into manual mode and pick something to solve a particular task or something that may appeal to their sense of craft, so I think that’s always going to be there at least for a long time. I don’t think it’s just going to be automatic or hidden for everybody. So I think we have to make that option there, or there’s some personalization aspect to it.
I’d say the cognitive overhead that we’ve put on developers today in terms of picking models, testing them, trying them, I don’t think it’s as necessary in the long term, especially if you think, “Hey, we have 10 models in the model picker, but imagine a world where there’s 50.” And nobody’s going to know one from the other at that point, so I tend to agree with you on that.
But where the product then needs to evolve is how do we deduce and learn what that individual’s preferences, style that lived experience is and go into auto mode and that we have that long-running context, that memory, and then we can more, I think, abstract away what these different models do, but it’s going to be hyper-personalized to that repo, to that developer or to that team. And this is where part of the vision that we talk more about, but just think about, okay, hey, there’s end users, but there’s the repo in GitHub, and that actually has a ton of context today. Issues, discussions, the security stuff, documentation, et cetera, team dynamics, et cetera.
Then you have what’s happening to me or you as an end developer and how you are evolving, how you’re working, that memory, that context that you carry with you, and how do you put those two together and then really make a lot of this much more streamlined, but I think it’ll move even beyond just picking the right model for you. I think it’s going to do a lot more than just auto-picking models.
Soma: Why don’t we wrap it up with one last question?
Question: If you had to fast-forward five years into the future, what does an enterprise engineering team look like?
Jay: Five years in the future? That seems like a really long time. For me, I think there are a few things. So one is I think in two years the companies that have figured this out will look entirely different. I think functions will collapse. I think applications will collapse, and I mean merge, not collapse, and go to zero, but I do think functions will collapse, and I think that’s going to be a big reckoning, so to speak. But I think companies that embrace and understand how to flatten structures, how to combine functions, and really use this as a way to pick up the pace, and also prioritizing and understanding how to shift that time from this run the business time to this creative time, those are going to be the ones that start to really run lead. I think there’s going to be a lot of enterprise teams that look like they look today, but where those companies are going, probably nowhere.
I’d say the other thing that’s going to happen here is I think the enterprises are going to have to work through the system because there’s just a lot of, today, in some ways, we talk about waterfall, and we’re like, “No, we’re agile,” but honestly, humans still work in a waterfall way. We work on a project, then we move to the next project, then you have to go through some set of reviews, somebody has to do 45 approvals before it gets into production at a big bank. Those systems, those things have to be automated, those functions inside of the company have to be, I think… We have to rethink them from the ground up. So I’m talking about the companies that are going to figure it out, the the companies that are still going to try to operate the way they are today, I think are going to just see the, “Hey, I’ve got 15% productivity gain,” but the rest of your competition is away in a speedboat and you’re like, “Hey, I’m paddling faster.”
Soma: Great. Okay, thank you so much, Jay, for this wonderful conversation.
Jay: Thank you.
Soma: Thank you.