OSS = Open Source ... Strategy!? Google Is Doubling Down on K8s in the AI Era, and Yo... ago Macleod¶

Playlists: KubeCon + CloudNativeCon Europe 2025 - London

Google explains why and how it invests in Kubernetes and open source in the AI era.

Categories: AI & ML

Topics: open source strategy, kubernetes, google, ai, community investment

Transcript¶

▶ 0:00 Thanks everybody for taking the time to come today at the end of the day. I know these things get exhausting, but it's the end of day one, so you still got some energy, I hope. Uh we are going to walk through a lot of content. Uh I tend to pile a lot in. Uh and so we'll see how the pacing goes as we go through it. Uh and sometimes the end catches up with me uh faster than I meant to. So we'll just have questions and answers if we get there. Uh so what were you here for? Uh we talk a lot about open source software. Uh we share software. We understand uh what other companies uh customers need and want. Uh they bring requirements from their own end users and then we bring our own solutions and

▶ 0:44 we collaborate. Uh but we don't talk a lot about why and how uh we invest in open source software. Uh how do we rationalize this in our companies? Uh and so I'm going to share some background and some context uh and a few of the waves that we've been through in the Kubernetes community uh bringing us up to present uh and share kind of how our plans have changed in recent years and just share where we're going and why. Uh so our open- source uh software office opened uh in 2004. So we're already contributing to you know the Linux kernel and others in at that time. uh but this is 21 years ago. Uh and

▶ 1:27 through that there's lots of open source projects from Google. It's really a core part of our DNA. Uh and it's really important uh to us and for us. We consume open source software. We contribute to it. Uh we often find that the best place to solve a problem is in the community. So then we don't have to do something special. We don't have to reinvent the wheel. No one gets to Google because they want to solve problems that someone else already solved. Um, so it's really important to our culture there. I don't know. This is a list of all kinds of other projects. I just got it from the uh open source at google.com website. Uh, and I realized that even in the United States, our

▶ 2:09 open-source program can drink. So, happy birthday. Uh, okay. So Kubernetes though Kubernetes though 2014 to 2017 the first phase was really about disruption. This was us entering public cloud market um we had app engine but really truly entering public cloud. We were looking for some unique aspect something that we could contribute that others didn't have. Uh this sort of coincided with the rise of Docker. Uh, and I still remember running docker run and feeling the hair on the back of my neck stand up thinking, "Oh my gosh, this just solved the whole it works on my box problem." Uh, and then of course you end up with lots of containers. Uh, and so I was

▶ 2:54 working at Nest in 2015 2016 as a user of Kubernetes. Uh, it was not wise to run production workloads on Kubernetes at that point. We did anyway. It was kind of entertaining. uh and then I in at Google Nest was part of Google there's an internal mobility and so I started working on Kubernetes in about 20 and early 2017 um so through the end of 2017 was sort of the disruption phase we felt we had an opportunity to expand the public concept of cloud from just VMs to also containers and in Google cloud our VMs run on containers so this is really core to our way of thinking about everything uh and we had leadership here. We had we were the world's leading

▶ 3:37 experts in container orchestration and we could show what we had learned internally and share that with the world. Uh and so that was the open source Kubernetes collaborate with Linux Foundation and create the CNCF contribute Kubernetes to it and create this thriving ecosystem. This was our vision. We never believed it was going to be this popular but uh maybe some of us did. I it was a a long shot. uh and there is a degree of luck and timing to all these things. So what made Kubernetes successful? I think six container orchestration tools were announced on the same day that Kubernetes was. So why Kubernetes? Uh I'm going to argue that there are really three or fourish things, three plus one

▶ 4:21 things that made it successful. Uh number one was that it was declarative. Instead of saying do this then that you declare the desired state and then decoupled controllers act independently to bring the system into alignment with the desired state. The second is that it is extensible. You the community end users vendors can extend the Kubernetes API. That's important not just so that you can do what you want but also so the Kubernetes community doesn't have to do everything. And that's critically important. And number three is that it's modular. You don't like the scheduler, swap it out. You don't like some aspect, do something different. But you can still use the rest of Kubernetes. And that's really powerful, too. And these

▶ 5:05 three things combined kind of got the community engaged. And this is critically important. There's this kind of uh network effect that comes, but this is the the core Kubernetes loop, right? It observe you act. You say here's my desired state. you observe and see what is the the current state and then compare the desired state to the actual observed state and do and then act again. Uh and this just continuously loops over and over and this image is I probably 10 years old. We've been using this image in slide decks the entire time. You've probably seen it before. So this declarative API active reconciliation and then we'll go through each of the other areas. So extensible

▶ 5:50 CRDs are kind of evolution of a third party resource initially then we rejiggered it. Uh CRDs I think we made uh GA in 2019 but we've been continuously improving CRDs over time. Uh and they're actually better than the built-in types in many ways at this point. So uh maybe we just replace all the built-in types with CRDs. Uh but this is really important so that end users the community you can extend the Kubernetes API use the same tools on your new resources. Uh this is super powerful. Uh and we've even implemented core functionality for Kubernetes out of tree using a uh CRDs like gateway API is

▶ 6:33 a great example. And the last part is this modularity. So you can plug it in, you can create a newuler, you can have multiple schedulers for different workloads uh and enables this higher level automation. So that network effect that I was talking about is sort of these three groups interacting together and they all have their own loops. Developers are creating libraries and use cases go back to the developers and then users are posting what they what worked for them and what didn't work for them and their own complaints and it and they kind of raise more awareness about this solution. Uh and then vendors see

▶ 7:17 gaps and they provide solutions and it sort of goes you know starts to grow and there's this network effect that grows the entire community. The other really important thing to the Kubernetes success was the incremental evolution. We like been very clear about what it was good for and what it was not good for from the beginning. Uh and that helped us sort of decide what was out of scope not forever but for right now. Uh and starting with stateless applications was really critically important and then sort of incrementally evolving over time to support stateful applications. uh you know damon sets and stateful applications and then batch we're still working on batch but batch and uh

▶ 8:01 scheduled workloads. So uh deciding what we're doing and being really clear about what we're not doing has been really important too. So then we get to 2018 to 2022. These dates are kind of arbitrary but they're directionally accurate. And this was the ecosystem expansion. This was like the sort of the result of that network effect. Uh and there's just an explosion of ecosystem projects and solutions to problems. Um it's you knowto comes from that time uh OPA uh gatekeeper stuff like Argo and K native and open telemetry. Uh and so the you know the more users there are the more

▶ 8:45 valuable it is to create for this ecosystem. You get portability across cloud providers to be on prem. Um, this is super useful for a developer. You get to just have this distribution channel for your solution. We've also incrementally expanded the workloads that are supported. Uh, and this one's important to the overall strategy because uh, initially Spark and Kubernetes didn't work so well together. This was an example where we we contributed to the open source the Apache Spark project itself and Kubernetes and sort of made Kubernetes aware of Spark as a first class scheduler. Uh and this opened

▶ 9:28 Kubernetes clusters as a distribution channel for Spark. Uh and it was super interesting to go through that project as well. And that was in 2018. this slide. I have mixed emotions about this slide. Like I'm kind of proud of it. I'm also kind of feel a little bit guilty because I don't know how a new user walks up to this landscape and understands where to start or how to go through it. We tried at one point to have a road map thing. Do you remember that? At some point there was like a and the road map got really confusing because it was hard to tell who was allowed to be on the road map and where they belonged on the road map. It was really difficult. Um but I think we can

▶ 10:12 declare that this part has been a success. There is a thriving ecosystem. Uh and it is important for the next part. So until then at this so far we believed that Kubernetes was following this sort of typical adoption curve right you crossing the chasm and all that cross the chasm in about 2018 uh and so then you get more stability. We stopped you know just deprecating APIs willy-nilly. We stopped yanking the rug out from under you. We made backwards compatibility a thing. Um, so things got more reliable, uh, and more predictable. I got a little bit bored, I

▶ 10:55 want to say, like, you know, predictable is great, but also, you know, not as interesting as some of the really fun innovation. So, uh, you know, so far so good. And I thought I wrote this slide at the end of 2020, no, uh, I don't know, midway through 2023, no, 2022. Uh and I thought what are we going to do in 2023 and beyond? It's going to be about consolidation. That previous slide with all those ecosystem projects, we have to provide opinions, not just options. Uh we got con feedback from end users that it was just too confusing, too complex. Uh and I thought it was going to be

▶ 11:38 about stability, simplicity, and being comprehensive. You need a whole platform. You don't just need one part and a bag of Legos. You want the toy. Uh, and so that's what I thought the future was going to look like. And then there was this plot twist at the end of 2023, right? The chat GPT moment. Uh, and the whole world overnight was just obsessed with this idea. Uh, and I've seen our capacity projections from the end of 2023 and then about 3 months later, I was like pretty hilarious. Like it's pretty easy to ship uh electrons. Uh, it's not that easy to ship atoms. So plugging in new capacity is something you probably all have had to deal with the capacity

▶ 12:21 constraints over the last year. Uh we cloud providers also have to deal with this as well. So uh demand has out uh stripped supply quite significantly. Uh and so we had this crazy overlapping stacking adoption curves and it's really interesting. There's like these two there there's enterprises still coming to cloud native to you know learn about containerization and microservices and then there's an entirely different plane that's trying to upend the whole thing at the same time and so we have to kind of balance this innovation and stability because we have both groups of users. So when we get to this point,

▶ 13:03 uh the consumption is not the same as the adoption curve. Uh and that these overlapping stacking uh adoption curves lead to some pretty bananas consumption growth. Uh and anytime you grow like 10x, you generally have to rethink the way that you did something. Uh and often at this point this would be an opening for a new platform to emerge that solves the new problems and you sort of just move on, right? You check out the old thing, you move on. And we had serious conversations about is this the right platform for the future workloads. Uh and so my old slide got thrown overboard. Okay, it's not that anymore

▶ 13:47 or at least it's not only that. some of the challenges that we discussed about Kubernetes when we started we started with the assumption of this sort of a few things uh Kubernetes as we move through the adoption curve we're getting into more missionritical business critical applications things like healthcare and telco uh the second observation was this evolution from what what we sold as what you perceived to be infinitely elastic cloud, right? And we didn't even really we weren't even very well regionalized

▶ 14:30 early on. So it sort of felt like cloud was just one thing, one end point in space that was, I don't know, magically scalable forever. You ever seen the bumper sticker, the cloud is just someone else's computer? It was like our computer actually. So we saw a little bit more of the details there. Uh, and so what's emerging with these new AI and ML workloads is that the hardware is super specialized. Uh, it's it's sparse. It's expensive. You can't just swap them out. You can't scale infinitely. The capacity you want isn't necessarily in the region you're in. Uh, and it's really much more complicated than it was. Uh, and edge makes things a lot more complicated, too, especially in the

▶ 15:14 networking space. Uh and then third this expansion from the predictable longunning workloads to these very dynamic workloads. Uh and they're sometimes multicluster or they have these specific hardware requirements. Topology really matters uh proximity the networking throughput and all these things and special purpose frameworks that understand that underlying hardware. So why did we choose to continue on with Kubernetes is part in part based on this concept of the path dependence feedback loop which is essentially the idea that you know what got us here informs what

▶ 15:56 will get us there. Uh also accepting that what got us here won't get us there but uh it's a pretty important idea. Uh and so the early adopters were super risk tolerant. I was one of those. I mentioned that I when I was working at Nest, we were running production workloads on Kubernetes. It was probably not wise. Uh but we did and it was fun and interesting and we contributed back. Um and so that brought in the early developers and and ecosystem vendors. Many of you are still in the community today. Uh and it led to that growing community. And so now we have a bunch of uh vendors. People understand Kubernetes. They know how to work with it. They've

▶ 16:41 built their tools on it. Uh and so if you want to build underneath, then anyone who built those tools can uh get access to what you build underneath. And if you build on top, you get access to all those clouds and environments that it runs on. So we came up with these three stories to rally around in inside of Google. Uh and this was how to evolve Kubernetes to meet the needs of the next trillion core hours is like the tagline. I like the sound of it. Uh and the first one is reliability especially at scale and even across upgrade boundaries. We've heard a lot of feedback that upgrades still create problems. Uh and we're working really hard on that. And we're working hard on it in the open source community

▶ 17:25 because it's important to us that all Kubernetes users get the benefits of the innovation that we're driving on the and the new versions of Kubernetes. It doesn't help anyone if everyone's stuck on version 127. The second was redefining the Kubernetes relationship with the hardware. And what does that mean? It means that early on we had CPU and we had memory and you could have more or less of it, but then you just add more nodes, right? A node is a node is a node. it's kind of all the same. Uh, and you know, storage, they're not that different from each other. Kind of this fungeible hardware concept. Um, but these GPUs, TPUs, these accelerators are super expensive and sparse. Uh, hardware's gotten really fun

▶ 18:09 again and interesting. Even CPUs like the there's a lot of innovation happening there that this elastic fungeible cloud thing didn't really consider in the past. Uh and then finally these purpose-built frameworks for these new workloads. Uh and it's really important that there's some operational consistency so that platform teams don't have to learn a new way to operate and do chargebacks and security on all these new frameworks. Uh you know we we started this way with a little bit of hubris too like how hard could it be to lock it down and it turns out it'd be really hard. uh and so I think it's really useful to some new frameworks uh to leverage what we've built in the

▶ 18:54 Kubernetes community. We'll skip the first one, reliability and you know reliability across upgrades is pretty self-explanatory but this you know what does it mean to redefine the Kubernetes relationship with hardware. Uh it meant that we invested really heavily in what uh Patrick from Intel and Kevin from Nvidia started with DRRA and try to make it more Kubernetes native and friendly and kind of really invest in it. Um so that there's some portability across accelerator like even driver versions require sort of finicky uh use pre the the new work that we've done in DRA. Uh so this is really

▶ 19:37 important. Uh and then after we intend to kind of make it more Kubernetes native, make it more portable uh and provide what we have across cloud providers for the hardware underneath as well. And then this one's really important too. So framework orchestration again people run for AI workloads. Um people run slurm for HPC. Uh and something really interesting has happened in the last year or so uh which is that we're we're positioning and are these frameworks are seeing Kubernetes as a distribution channel rather than as a competitor. Uh and so this is a really symbiotic relationship. I talked about how Kubernetes early on made a decision

▶ 20:21 not to be all things to all people but to really draw a line about what was in and what was out uh and provide the value where we intend to provide value uh and then leave you know HPC scheduling if you've ever looked at the slurmuler like it is pretty impressive and complex and nuanced and it's got a couple decades of experience in it. uh they're going to know about that better than we are. Uh so having concept of guest schedulers is a is an idea and to make Kubernetes support those frameworks as first class citizens. So the goal in all of this is this consistent operational model across

▶ 21:05 the frameworks that exist today are emerging and growing and the ones that don't even exist yet but are coming soon. And we think that that's a really powerful and valuable model. So Dan Khan uh the CEO of the Linux or of CNCF and I years ago talked about the idea of Kubernetes as the uh like based on the hourglass model of the internet. Uh and this was a diagram that circulated I don't know 40 years ago. The whole concept is that like IP is the center, the narrow waste of this, you know, thriving ecosystem on top and different technologies underneath. Uh, and so this is my terrible reproduction

▶ 21:49 of this. Uh, and you can tell that I'm not a graphic designer in my free time, but uh, you get the concept, right? IPv4, IPv6 as the narrow waist, uh, browsers, mail clients on top. There's all kinds of ways that that can travel around down below at the physical layer. Uh and so making Kubernetes the narrow waste the you know the hourglass model of infrastructure consumption is our vision. And so we want this to be the case for additional layers on top, widen the top of the funnel at the top and widen the bottom of the funnel at the bottom. um but ensure that there's some consistency in the middle so that those can evolve independently. So what is our strategy

▶ 22:36 then from all of this context? Our our strategy is pretty simple in three parts. Number one to maintain to ensure that Kubernetes continues to thrive extend Kubernetes as the de facto standard for infrastructure orchestration and expand and especially for a IML workloads that are so important to every business. Uh make sure that it works for those frameworks and the new workloads that are coming as well. Now how do you differentiate? Right? If we're doing all this in open source it would be totally open and honest. Our intention is to differentiate on performance really and then that gives us business opportunities to do price performance like you can offer the same

▶ 23:20 performance for a lower price or better performance for this like there are options there but really it's about differentiation on performance. How do we do that? Uh well we own a public cloud. We do deep integration through all the layers of the stack with our friends that work on uh GCE where Kubernetes runs on the VM layers on the networking layers uh TPUs. So there are opportunities to accelerate the hardware uh and the full integration through the stack and sometimes it's about scale too. So when we I talk about asymmetric opport advantages as an example is

▶ 24:03 spanner. Spanner is an engineering marvel. It's another team. We love it. It's amazing. Uh it gives us the opportunity to use that in place of CD to support 65,000 nodes on GKE. um that we feel like the standard remains the same but the implementation is slightly different and it gives us the opportunity to expand uh and that provides uh some differentiation that's sustainable and differentiation that's not sustainable is just friction for adoption. So we really don't want to differentiate in too granularly. Uh and finally stickiness. This one has a, you know, senior executives at Google even have worried

▶ 24:48 at times, how do you build a sticky product on an open source project whose entire point is to be slippery? Well, it turns out that all of you and all of those ecosystem projects as folks use Argo and open telemetry and all these other, they actually just tend to stay because it's working and they find solutions for the problems they have. So, it actually is stickier than other products that we have at Google. So that continuous loop that we talked about before, we apply that not just in Kubernetes technically but also in our uh operation of GKE. So this is not an academic exercise for us. It's really

▶ 25:32 important that our team work on open source Kubernetes and they are very closely tied to GKE like we are solving actual people's real world problems. That is super important. Uh it's the these are you talk about scheduling across multiple frameworks. This is like you could nerd snipe anyone on my team. Uh and they just go down a rabbit hole for years. Uh that's super dangerous. Like it's it's really distracting. Uh and it's really important that we stay pragmatic. So we launch something in GKE, we observe how it works in the fleet, we see real world problems, we change it in upstream open source Kubernetes, we fix it for everyone, we fix it for ourselves at the same time. And that loop sort of continues over and

▶ 26:15 over and over. That is critical to our approach. And so when we talk about evolving Kubernetes, we're talking about every single layer. And we go through phases sometimes with workloads uh or with frameworks where the first iter first phase is succeeding in spite of Kubernetes. It doesn't really work but you stretch it and you kind of jam it in there and you do some nasty things on the side. Uh and it kind of works sort of but our goal is so that it works because of Kubernetes. We want native support for those things. Uh and it requires from the bottom in the hardware layer. We're talking about DRA and uh

▶ 26:57 accelerators. Uh the top multicluster use cases, things like Q and multiQ. Um it goes the full stack and there are no immovable mountains. Uh so if you have a challenge and we're not meeting it, please reach out. Let's work together. Uh and then there are all these things that kind of stretch across all of those layers on the other side. This is not again just a slide. This was just day before yesterday. There was a maintainer summit here. Uh and this was one of the unconference events that was proposed that John proposed. Uh and we're talking about evolving Kubernetes for a IML across these layers. Uh you can only see the top, but you can go to

▶ 27:42 the GitHub issue and see the rest of it. Uh and we talked about this with the community specifically for this case. So this is act this is real work going on right now. We've gotten a good start and there's a ways to go. I think maybe I put this slide in the wrong place, but this is the differentiation slide. Uh I feel like this is uh a perfectly fine place to differentiate and to [Applause] compete. So again, why Kubernetes? Um it's declarative, extensible and modular. that makes it super well positioned to evolve to meet the needs of the next round of workloads. Um, we've done that. We've done that with Spark. We've done that with stateless

▶ 28:24 workload or stateful workloads. Um, creating that path dependence feedback loop where there's now network effects and there's a lot invested in it. There's a lot of value and so we have a window of opportunity. We really do believe this is a window of opportunity to evolve so that we don't get pushed out of the way in favor of something else. Uh we're well on our way. We're accelerating. We're speeding up. We're actually investing a lot more in Kubernetes than we were a year or two ago. And we think you should, too. And we're stoked to see what we can build together. Thank you so much for taking the time today. I really appreciate it. Enjoy the rest of the show. Um I don't know if there's time

▶ 29:08 for questions, but I'll stay around here if anyone has any questions. Uh this was some goofy AI generated uh content. I'll call your attention to the interglass xnules in the upper left there. You have to pay attention to the interglass x nules apparently. So thanks so much [Applause]