The Cloud-Native Workloads Journey Through the CNCF Tag-Runtime - Ricardo Aravena, TruEra¶

Playlists: KubeCon + CloudNativeCon NA 2023

Overview of CNCF TAG Runtime activities and working groups and how to get involved.

Categories: AI & ML, Serverless & runtimes, Edge & IoT

Topics: cncf tag runtime, wasm, container device interface, batch system, ai working group, iot

Transcript¶

▶ 0:00 thank you for being here and I'll be talking about the cncf TAC runtime some of the activities that we've been involved with and some of the communities that we've been reaching out to and this session is mostly about the audience to understand what we're doing and how they can help uh with the tag and the different groups in within the tag and also how they can maybe attract some folks that are interested in contributing all right so a little bit of what I'll be talking about today is some of the overview of the tag and what it

▶ 0:45 means uh the activities that we've been working on or we've been having presentations and the different working groups within the tag we have the iot the batch system initiative working group we also have the container device interface wasm special purpose operating system recently created and we have an AI Cloud native working group that we just created the charter and we expect that to be approved in the next uh two weeks then I'll touch on some future work and activities that we're planning to do and how you can get involved so uh I want I have a slide so

▶ 1:29 on the charter with chat GPT at that I showed at cucon Amsterdam and I went to chat GPT this time and they didn't actually have a lot there so it seems like they they shut down their some of their algorithms so it doesn't say much about tag run time but I did actually went to Google B and he came out with a better answer so just for the fun of it this is the answer so it gives you a pretty good detail answer what the tag I guess it looks at the the GitHub repository at the website and it provides pretty good information so it's it's good to uh understand some of these things that are happening out there and I think um tagr time is no different and

▶ 2:13 just for the fun of it also I went to a image generation site or generative AI image uh logo creator and it did actually create uh three different versions of our logo um it's not actually very close to what our actual logo is but I mean if you want to get started is it's it's something um but a lot of talk on generative AI so but what it what it is uh with tagr time it's there to help different users and communities uh use cloud native Technologies when it comes to workloads and these workloads can be badge type of workloads like high

▶ 2:58 performance type work workloads that run in multiple machines or they could be latency sensitive workloads for example microservices so all of these in the cloud native uh context we work with the TOC we to is the CNC of to the technical oversight committee we have three to leas we also have uh chairs and one of the co-chairs and we also have tech leads and the tech leads uh help out with the different activities uh that some of the chairs also do but they tend to focus a little bit more on technology we meet every first and third Thursday of every month and communication happens over email and

▶ 3:41 slack so these are some of the sample projects within the tagr time scope uh as you can see there's a variety of different projects in different areas you have things like uh container D there's a container shim you have Harbor which is a container image registry you have other projects like kada that helps you autoscale workloads in an automated way with different metrics unicraft is another project that allows you to run unic kernels so variety of different projects in different areas and this different areas uh we Loosely Define them into scope areas within the tag have the general workload

▶ 4:25 orchestration where kada and kubernetes fit in and you have the run times the VMS such as the container D cryo shims uh lately there's been a lot of conversations around wasm so the wasam run times also fall within the scope Ser serverless workloads is another area and K native is one example of those projects in currently in the cncf then another special area is the special purpose operating system area so there are several projects there like bottle rocket flat car uh recently we just created a new working group that will tackle some of these areas in special purpose operating

▶ 5:08 systems another area that is super exciting is AI machine learning uh so we do have a lot of projects that fall within that scope um such as uh Q flow K serve that allows you to serve machine learning models and then obviously we have the the areas of the different working groups under the tag and describe here that and that I mentioned also previously so now to some of the activities and presentations uh that we've had in the tag so we do have a website so there's updates on the website so some of our community members are are

▶ 5:50 posting information there uh and also we request anybody in the community who has any feedback or information that would like to share or would like to update on the website they're free to do so and and they can uh create a PLL request on the GitHub repo and make the change to the to the so the tag runtime website so one of the exciting things that we have going on now is the proposal to create a cloud native AI working group there's been a lot of uh excitement and hype if you will about AI and generative Ai and llms and we feel

▶ 6:34 that we need to address address that area so there's a lot of community members have gotten together and started creating a charter and as a matter of fact they already created a charter and that's being reviewed by the cncf TOC and we expect that to uh get started maybe in about two weeks from now and some of the sample deliverables for this working group are things like white papers on cloud native Ai and we could have things like uh the landscape with all the cloud projects and how they can help enable machine learning Ops and and AI type of

▶ 7:19 workloads uh some surveys that they can send out to different Community uh members or or organizations to understand more of the ecosystem reviews and recommendations are another area that that is interesting and that the working group is interested in tackling uh also reports on new trends and AI industry that can help Cloud native so all of these in in the context of cloud native and how AI can help Cloud native and also how um Cloud native can run AI type of workloads another thing that we have going on is is that we recently created

▶ 8:00 the cncf tax GitHub organization so Nikita from the TOC created the the poll request or the or the issue to to start this uh initiative and right now this is available to all the taxs but one of our container or working groups uh in the tax actually created a art act that is actually hosted in in this uh organization and this is just a home for this working groups to to have a place and and to have some something that can be shared across the community

▶ 8:44 and and and and it can be actually used by several organizations or by by by different community members this is a sample of presentations that we've had in this year and last year in different areas we have the container tools projects like uncraft container D presented and some others in terms of workloads uh we had cuet graduation presentation so the Q project it's pretty mature so they wanted to provide feedback to the community and and they presented uh in our meeting and other areas of presentations include like kubernetes

▶ 9:27 related projects that enhance the kubernetes ecosystem like kada eraser CU stellar and so forth and finally we had quite a few presentations also on operating systems and this will actually um be more uh developed with within the special purpose operating system working group that I'm going to be talking about in a little bit and just to give you an example we had the Eraser project presented in our meetings and so eraser is a project that uh allows you to remove uh non-running

▶ 10:10 container images in in a kubernetes clusters or in the nodes I'm going backwards oops sorry and this is a demo of the project so you can see uh that you know here we're creating a sample cluster with with [Music] kind and the idea here is that it'll it'll create a a service and and uh that brings a container image or or create a deployment that brings in a container image and that container image or deployment demon set will will be

▶ 10:53 delet deleted in this case is the demon set that it's getting created and and then it will be deleted so you see it's coming up this uh de demon set and it's a job that is completed and then the the the the image is not being used anymore and you can see that the pots have been deleted and right now it actually installs the control plane of eraser and the the idea here is that after this is created then the there won't be any container

▶ 11:37 images related to to this specific uh demon set that was created so you can see the and the cluster got the image got deleted and it's not there anymore so another project that actually presented is the uh flat carp project which is a special purpose operating system just to run containers it's a minimal distribution uh of Linux it helps the H ecosystem create this server that is more secure more automated um and it provides this declarative uh way of provisioning and you can see that it's

▶ 12:23 actually being used by a wide variety of different organizations commercial aure AWS BMW Etc unicraft is another project I presented um it's a tool a go-based tool framework based on uh sorry it's craft kit it's and it's presented presented and it's based on uh unicraft a craft kit it allows you to create uni kernels so it's a tooling tooling around uni kernels uh created for unicraft so it's go based and and as you can see it's a CLI based uh system and

▶ 13:06 here we have a just a sample demo of hello world and you can see that we update the pack the craft package update we run and we modify uh main.c file to create the unic kernel which is with this command line and it Al has this craft file that defines how you define that unic kernel and right here you we're building the unic kernel with craft build and we just do a a craft run and you can see your hello world of your unic kernel so another project that we had was kada but they were interested in

▶ 13:51 graduating so they presented and they they showed us their progress uh so K allows you to automatically scale deployments Up and Down based on different scalers and the scalers are Integrations with different endpoints like Apache Kafka Messin or AWS sqs or different cic CD systems that allow you to um spin up many workers to run your cicd jobs uh it it actually provides uh over 55 scalers or 55 different integration points it can run on Linux or arm uh so the idea with this project is

▶ 14:34 focusing on scaling your app but not the the scaling internals and then and as you can see the progress um of Kat towards graduation and in V2 they had about 20 scalers but when they went for incubation there were up to 40 scalers and now that they graduated I think they're over 60 scalers uh so it continues to grow over time and as you can see there's a lot of end users uh using uh the project about 11% of kubernetes users are using kada asure container Ops is using this project and in production and AKs and

▶ 15:19 open shift are offering a managed version of Gada so another working group shifting gears a little bit um or now going into working groups rather uh so weate recently created the wassom working group so there's been a lot of uh conversations around how web assembly can help cloud dative and web assembly runtimes can be used to run Cloud native workloads and initially started with this spr uh trying to re reach out to the community members uh who were interested in the space and we did actually get a lot of interest and and we came up with a a

▶ 16:02 charter a document and and folks started collaborating and and up they created the working group and right now they're actually having meetings almost every week uh these meetings are actually getting recorded getting posted on the cncf tag runtime Channel on YouTube and there's a large lineup of different presentations uh so we we got a lot of particip ation and it's a very exciting fi so the other working group that we recently created was a special purpose operating system working group and there's a lot of interest in this space as well so you can see that we had

▶ 16:46 some presentations on October 5th and we have a full lineup of different operating systems and there's a lot of conversations about standardizing the apis that talk to this different operating systems so excited to see this uh evolve and we'll probably see a lot more the rest of this year and next year and some of the things that we we thinking about this doing or addressing the special operating system working group for example standards to run containers uh common apis to manage these operating systems speed up the OS provisioning for kubernetes and as different standards to run the containers or run like specific things like web assembly modules on on

▶ 17:30 top of these operating systems so another another working group is the iot working group and and in the scope within this working group uh includes many different projects that are about workloads at the edge uh for example Cube Edge uh k3s but now they're also uh collaborating quite a bit with the wasm cloud and wasen Edge communities or the wasen ecosystem and um we'll see continue to see uh progress as well in this working group but this working group is not recent it's been there for maybe about two two and a half years so there's a

▶ 18:13 lot of uh different deliverables that actually have come out from this working group and one example is this Ed native applications principles white paper it basically talks about the differences between what it is to be CL native and Edge native so with Edge native uh there are some differences uh with the applications where they need to be aware of the different constraints at the edge for example they don't have large amounts of CPU and memory they have uh access to less power they they need to be more resilient to network outages or not being able to connect to a centralized location they

▶ 18:55 need to be more secure in terms of like having a lockdown mechanism if somebody breaks into the closet where they're actually stored so a variety of different principles that need to be taken into account and this uh working group has actually created this white paper to to educate the community and now they're working on the next step of this working of this uh white paper and that's the application principles application design behaviors white paper and this is how you put this principles in practice so this is currently in the works we expect this to be publish in the next maybe month or

▶ 19:37 so and that's quite a bit of work on what that we can say that happening there in in that working group another one is batch system initiative and this one has been there for maybe two years uh but right now with the Advent of AI U there's a lot of overlap with uh some of this high performance type of workloads so we might see collaboration with with this working group and and the cloud native AI working group but right now they're working on a batch uh landscape uh as you can see there not that many projects um there's a few but we expect this to continue growing for example there's one called Volcano that allows you to

▶ 20:21 schedule batch jobs and communities and and some other projects that allow you to manage uh how these uh jobs get run across multiple uh set of notes in in a kubernetes fleet uh but yeah it's an exciting feel and I think uh I think we'll see more um more tiles here popping up in the next few months they're also working or have uh created a white paper on batch scheduling tools and and the features around batch uh or details that are looking at our scheduling policies uh preemption you know access to SDK and

▶ 21:03 apis or of the different project characteristics within the space with within the batch space like uh whether they're open source whether there's active de development or whether these projects can become a cmcf project or are already a CM CF project and finally we have the container orchestrated devices working group and this working group has been working on container uh device uh interface standard for um all kinds of different container run times so the idea just to have support uh in all of them across the

▶ 21:45 board and if you can see uh they've added support for container D they added support for cryo they V support for potman and Singularity and they also have uh uh vanilla Docker support in the works they've also these artifacts that they've created which I mentioned earlier uh have been moved to the cncs tax organization uh this is again this is just not a specific to tag run time but uh this was the first working group and group of people actually both artifacts to this uh or uh organization in the

▶ 22:29 cncf and they're also converting uh some of the taxs cncf uh that that I O URLs uh with package names so what's next uh for the tag runtime and what are some of the things that that we're looking at so obviously we want to have more people participate and tag I think this is a common problem in a lot of the open source communities but and this is no different in the tag run time uh some of the things that we can do is actually revisit some of the old

▶ 23:11 projects have them present again and provide an update or re-engage or provide advice on how they can improve or how they can pivot maybe pivot in in different ways if if they don't have a lot of traction with contributors uh keep on reaching to the different projects in the different areas for example Cloud native AI is Happ happening now so there there's llms everywhere so continue to ride that hype uh another project that is interesting is open tofu so there if you have actually followed some of the

▶ 23:54 developments from hashicorp and terraform light licensing their licensing was changed to business license licensing so there was a group of folks in the open source Community they decided to Fork terraform and they' started this project called open tofu which is an exact Fork of terraform but it would actually evolve into something else so we plan to engage them as well uh continue to look at uh the variety of different run times that may may come up in the ecosystem wasm are the one that are that come to mind or some of some of the ones that are used in iot Edge also continue looking at different tools in the

▶ 24:37 kubernetes uh managing ecosystem how how you manage multiple kubernetes clusters as they grow as they become more of a challenge and obviously continue to have updates on our T runtime website and have have this on a regular basis yeah with that you know I just um um you know feel free to reach out and you know talk to me or talk to anybody in this room feel free to come come up to me and happy to answer any questions thank you