SIG Cloud Provider Intro - Fabio Rapposelli, VMware & Nick Turner, Amazon¶

Playlists: KubeCon + CloudNativeCon North America 2019 (San Diego)

Intro to SIG Cloud Provider and its project to extract and migrate cloud provider code out of Kubernetes core.

Topics: sig cloud provider, cloud controller manager, cloud provider extraction, migration

Transcript¶

▶ 0:00 welcome everybody this is say club provider intro session and today we're gonna give you like an overview of what's the club provider does and we're gonna be our lovely host for today so my name is fabio riposte le or for vmware they work in kubernetes for the past I don't know four or five years working mostly upstream so you probably saw some so my PRS on vSphere if you use vSphere there's a pretty good chance that I wrote some of the code and we me there's Nick Turner from Amazon you want to introduce yourself Nick my name is Nick Turner I work at AWS on eks and before that I worked at a couple of startups in

▶ 0:43 Seattle and where are you it was a user of kubernetes and now I'm a cloud provider I guess awesome so the agenda for today is we're gonna we just we want to make this as interactive as possible so please feel free to like ask questions as if you get any questions you want to ask us just do it right away don't wait until the end we have some slides we're going to go over so we're gonna talk about you know some logistics first we're gonna introduce the cig we got going over one of the main projects that we have currently in the cig which is the lot extraction and migration project and at the very end we're gonna offer some topics for a discussion so if people

▶ 1:27 have you know question they haven't thought about just look at the slide we're gonna give some you know food for thought there but again feel free to ask any questions any time we'll be walking around with a microphone to give you a chance to ask your question to the microphone so please raise your hand let me know if you have anything you want to ask us some rules of engagement please you know just be respectful be kind we're here just to collaborate no you know no hard feelings with anybody so just you know just be kind that's really the only thing to keep in mind right now and this is the Charter for siccola providers so if you never be exposed to sick cloud provider before this is what sick liberal sick club provider is all

▶ 2:11 about so sick law provider and it's going to the Charter right now cyclop providers mission is to simplify develop and maintain club or Club provider integrations as extensions or add-ons to kubernetes clusters so what does that mean so it means that we have you know some of the things in the kubernetes ecosystem that we want to make sure our you know working in concert with everything else happening in kubernetes course so ideally what we want to do is we want to you know be able to leverage class specific components and services with kubernetes so if your cloud platform has some services that can be beneficial to kubernetes we want to make sure you have all the integration points all the api's you need to use them

▶ 2:55 effectively so we want to make sure that you know you have all the right abstractions all the right api's to do the lifecycle management of those resources so think of it you know has you know a way to provisional balancers or you know use your I don't know encryption keys from your cloud provider we want to make sure you have the right abstraction the right api's to use them and last but not least there's actually a very important part of the Sickler providers to ensure Ventor neutrality in the project there have been a lot of tensions before when all the effort with club providers was focused on kubernetes core you know lots of tensions since you know vendors have different agendas

▶ 3:37 sometimes and this you know the creation of Saqqara ensure that there's vendor neutrality in the kubernetes ecosystem itself so we want to have make sure that we have a testing framework a testing environment that is able to maintain that vendor neutrality across the board so this is a very big focus of siccola provided these days so again what is the Cygnus possible for common interfaces all the things that we do in cyclop provider are meant to create this you know this bays underlying interface for cloud providers to integrate with we also work with utter SIG's to you know get to disco think of it as you know a

▶ 4:21 way to integrate for example with state cluster life cycle so cluster life cycle has projects around lifecycle of clusters so we work with them to make sure that all the interfaces are you know working in concert together so think of cluster API for example that's one a big you know point of contact that we have with them we want to make sure that you know clock controlling manager which is a you know curious core component is you know work in and all the core controllers are managed this is under the responsibility of the sig and we also kind of we're also the umbrella sig for all the projects that don't have that our club provider specifics and they don't have a

▶ 5:03 specific six sponsoring them so one big example that I could give you here is CSI so CSI is the storage the common storage interface the CSI spec itself is sponsored by six storage so six storage is looking after the the spec of CSI itself but not of the specific club provider implementations of it so give an example you know the AWS CSI implementation it's not under six storage because six storage doesn't want to have you know doesn't want to sponsor those efforts so they are under state law provided right now and right now the only counter example I can give you is

▶ 5:46 cluster API so cluster API has providers for specific platforms ABS vSphere and others and those are under six life cycle so depending on the governance of the sig itself it can be either under that or if not it will be collected under state cloud provider and these are some of the sub projects we currently have under state law providers so there's as you can see there's a bunch of you know providers listed there with the name of the product itself so this is a kind of an interesting distinction so you'll see that there's some names here but you probably know that there are auto club providers that offer kubernetes integration but they're not listed here why is that so these are the

▶ 6:31 providers that decided to upstream their work so all these providers are listed under the six sub project list because they are part of the kubernetes six goetaborg so basically they're under the CN CF umbrella under the kubernetes umbrella they're not tied to their own vendor vendor or goetaborg they're basically just part of the upstream community and if you're a if you're a vendor if you're a club provider and you want to move your code under that please reach out to to us during one of the sig cloud provider meetings we'll be happy to accommodate we actually had a few of those top of my mind will be Alibaba

▶ 7:13 that you know recently got moved under the urban age six org we also have utter pride of that participate in the sig but they're not you know comfortable to move those could their code for the club provider under the sick because of other reasons so either way just be engaged with us even if you don't want to move your code under other kubernetes on the club provider sick but it's important to have you on the same page with what we're doing as part of the broader sig and now we're going to talk about the club provider section migration project and I'm gonna hand it off to Nick yeah so a

▶ 7:58 little bit of background when obviously when kubernetes began all of the cloud provider specific code was embedded in core and people quickly realized that this was a little bit untenable we had all these cloud provider SDKs directly imported into the project and that caused started to cause significant code bloat has more and more providers added themselves in so we're currently in the product process of removing that code from the kubernetes core codebase but it's a very long process so we were working on it last year we're continuing

▶ 8:43 to work on it next year and so some of the benefits of this it's going to be easier to support new providers because they don't have to actually go through the process to be merged in entry providers can release whenever they want they don't have to follow the kubernetes code releases and then I think the third the third benefit is just reduced bloat in core the the SDK is actually significantly bloat some of the binaries like cubelet so and you can take a look at the removing entry providers kept for a little bit more information on that so

▶ 9:26 one big piece of this extraction migration is the cloud controller manager so as you might know a lot of the loops in the cube controller manager right now have cloud provider specific code in them so part of this extraction is breaking those out into a separate binary which would be the cloud controller manager and so there is actually a kept which is in the process I think it's I'm not sure I don't think it's implementable yet but we're getting there so we're trying to get there by the end of this year and basically the

▶ 10:10 process that we will follow or the proposed process is - so right now cube controller manager has a leader election lock so if you have multiple cube controller managers running only one of them is the leader and actually acting on anything but when you break a number of controllers out of that binary and then you try to perform an upgrade you can imagine that as a new node comes up with this broken out binary those controllers that have been broken out the cloud specific controllers aren't gonna listen to the lock right they're just going to try to do they're just going to start acting on whatever resources they act on so the the way

▶ 10:54 around this is to introduce a new migration lock which the broken out controllers will sort of obey so you can see here the cute controller manager has two sets of controllers in it it has the core controllers which are continuing to watch the cube controller manager lock and it has the broken out cloud controller managers still within the same binary and those guys are looking at the migration lock and then the next step of that migration would be to actually move the cloud the cloud

▶ 11:38 controllers over to the cloud controller manager binary and so when a new node comes up with those controllers they will be looking at the migration lock as well so that allows the process to happen without any conflict there questions on that if you have like any questions just raise your hand I'll gonna walk over you get a microphone yes so you said yes about right now is the documentation

▶ 13:16 project so the idea is right now to get started with any club provider you have to go and find your documentation right you have to go search the internet for you know how do I you start up you know my cluster or well how about operating my cluster on you know vSphere or anybody else or whatever else whatever other providers so right now we're trying to get in sync with sig Docs to make sure we have you know the right Assad the right side lock the right set of expectations for provider that are listed under state cloud provider in terms of documentation have that standardized so you know the benefit of being part of that your org be part of that be part of secure provider as a project is to leverage those you know synergies between all the various

▶ 14:01 projects right and be part of the upstream community but you know if you feel like you're you know you're you're okay with having your own documentation your own repo your own issues your own you know way of governance for your cloud provider that's that's absolutely fine that the kind of benefit you get from participating in the community is standardization basically extraction work so 2019 encompassed

▶ 16:45 beginning with began the migration out of tree and cloud providers were moved into the legacy cloud providers repo and staging towards the end of 2019 we're still working on the strategy the migration strategy for a cloud controller manager and then 2020 should see the migration strategy become a little bit more battle tested and TBD when cloud providers actually make the cut over I will see hopefully 2020 but

▶ 17:26 maybe 2021 anything else I just want to add one thing if you experience the KK repo we built it in tree it was part of KCM

▶ 20:00 so now with the AWS cloud provider we just import the legacy cloud provider into our external binary and because legacy is sort of under kind of us I would say soft code freeze and ideally we're just trying to accept bug fixes there so that's our incentive to get it out to migrate that code out completely as soon as possible so maybe a little bit an opportunity for some discussion here what is a reasonable time line to fully remove entry providers is the

▶ 20:43 first question and so also what is the state of existing cloud providers and they're out of tree providers if anybody wants to share or has any thoughts on that I want to ask another question I mean how many of you are using out of tree providers either be the current organization so we talked

▶ 23:05 about this before so how do you how do you discover how do you find out you know what the providers are the documentation how do you actually get this thing you know off the ground so we previously had six specific sick claw provider specific SIG's like cig VMs tikka WS cig OpenStack and so on so some of some of the people kind of like you know I'm running you know my cluster on VMware so just go to cig VMware get all the information that I get identity for them we're gonna like briefly touch on that after but you know all these seats

▶ 23:48 are now folded into siccola providers so there's no longer a sick VM or a ciggy ws meeting we only have you know provider specific means for that so wonder if interview either as end users or you know integrators or even vendors find this discoverability of information around how to run kubernetes on a specific law provider find it difficult or it's hard to figure out how to understand hard to Google even yes is successful so we're gonna try to help

▶ 27:27 with that as much as possible but that's what every cluster operator is going to have to go through the other thing is even for managed kubernetes there are a couple of subtle places you're going to see changes so as an example moved PV type for a gcep D on CSI and so the olden old one will work but you will now see this new option and there'll be a bunch of guidance about you know where possible please use the new type not the

▶ 28:11 old type today you this presentation and tomorrow you want to get started you know contributing to

▶ 28:54 this first thing first step join the meetings we have a biweekly meeting on Wednesday right Wednesday and you know that is the best place to start we do have obvious obviously we have open issues so people can just look at them but more importantly is like to get on we were what the what the worst stream is right claw destruction of migration it's it's a big topic right now it's probably where we need the most help at this very moment and it's gonna be it's gonna be huge impact on kubernetes as a whole so it's actually a very interesting problem to solve also a very complex problem to solve so there's a lot of work there that needs to be done and there's also separate meeting

▶ 29:37 for that you know besides the main six o'clock provider meeting there's also a cloud extraction and migration meeting specific for that so that would be a very good you know starting point if you want to contribute to to the work here there's also a lot of work in the cloud providers themselves so every single cloud provider has a lot of work to do it can be on the CCM itself it can be on CSI it can be on any other provider so if you care a lot about one specific club provider or one specific part of a club provider say storage on vSphere or storage at NWS or encryption provider there's a lot of work to be done there too so plenty work needs to be done