Using Custom Resources to Provide Cloud Native API Management - Frank B Greco Jr¶
case study · Northwestern Mutual · 33:27 · Watch on YouTube
Playlists: KubeCon + CloudNativeCon North America 2017 (Austin)
Northwestern Mutual presents Kanali, its open source Kubernetes-native API gateway built on custom resources, with a demo of how it manages APIs.
Topics: kanali, api gateway, custom resources, api management, kubernetes
Transcript¶
▶ 0:00 thank you guys for coming my name is Frank Greco I am a cloud native engineer at Northwestern Mutual in Milwaukee Wisconsin if you haven't done so you can check out our booth we're not a vendor but we are showing off how we do kubernetes at scale talk about Canali and maybe attract some talent along the way so Northwestern Mutual built a nice tower opened earlier this year all right I do I do 27th floor it's 32 so ITV gets the top floors so I'm here to talk about Canali which is an open source API gateway kubernetes native cloud native the plan is to blow through a bunch of
▶ 0:43 slides do a demo and then leave plenty of time for questions because I'm sure you guys will have plenty and so the first thing I want to talk about is API is why API is why an api gateway etc brush up on some of the terms if you haven't dealt with them a while or introduced them if you haven't so we'll start with a simple example so let's say if a Google home and you want to talk to a nest what you'll do is you'll actually go through Google home will call the nest services and then the nest services will talk to the nest for you but what's going to happen is it's going to go through a little piece of software called an API gateway and that's going to make sure that you're not maybe calling the API make sure you know it
▶ 1:27 rate limits it maybe does some transformation you get all these things that come with what's called API management so things like API API key authorization and a JD Beatty authorization you know rate limiting transformation monitoring routing tracing etc and it's this piece of software it's one piece of software it's a gateway and it's the entry point to all of your API s and then traffic goes from the Gateway to your back-end services so some common examples of those in the industry might be Apogee the azure API gateway mule soft Kong the AWS API gateway so if pictures help you associate vendors to solutions think of these when
▶ 2:12 think of an API gateway and this is what Canalis helps to solve and we'll talk about it more in a little bit so that's what an API gateway is I'm so now I'm gonna take you through a little bit of history on the API management at Northwestern Mutual and how we've evolved from where we were two years ago on kubernetes 9 to where we are today with Canali in production so here we have two VP C's this is a setup that we have for our Northwestern Mutual website so here we'll have a VP C on the left and a VP C on the right and we'll peer them and we'll have our we'll have our micro apps or the apps that are running our UI in one VPC and then we'll have our kubernetes cluster in another and so here whatever our cluster and the way we
▶ 2:55 started out is we gave every kubernetes service and ELB me expose it by a node port and then this blob of squares represents the vendor that we were using and all traffic from the left V PC would go through our API management solution and it is outside of the context of kubernetes is a separate vendor and then traffic would get routed to the appropriate ELB and then he'll be would route traffic to the service on prom woes it got pricey over time as we scaled a number of services each service needed its own EOB and so we needed a better way and so we did what's lots of people in the community did is we had an ingress controller and then we threw one
▶ 3:38 alb on top of it and then service would go from the API management vendor solution through the EOB onto the ingress controller and then based on context route and some other some other items traffic would get would land on whatever kubernetes service you wanted and so then you got rid of the price now this may look calling all well and good but what it's not showing you is the mess that may or may not be like the vendor solution so what happens is when you have a cloud native tech stack you want the tools that service your application to be as cloud native as possible and so if you're using a vendor as an API gateway that's outside of the context of kubernetes you lose a lot of insight that you can get from having something running inside of the context
▶ 4:22 of kubernetes and then because it's maybe not a cloud native lucien it's different and hard to automate you may need like two or three different types of databases to store the information and it's hard to integrate into your into your pipeline in your CI CD model because now you may be forced to configure stuff through the UI when you want to do them as code and so there's a big mess of stuff and it didn't really fit well with the cloud native approach that we wanted to take with our environments and so we thought that we could create a an open source solution for this and if we were going to create an open source solution to solve this problem we have some design goals and have very high-level design
▶ 5:07 goals so when I had to be innovative so we needed to be solving a we need to be solving a problem not only in the context of Northwestern Mutual but also solve a problem in the context of larger open-source community so that we can contribute it back and that others could benefit from it I need it to be lightweight so it needed to be just minimal resources minimal you know amount of pods if it's in kubernetes running and just stupid simple to deploy very easy barrier to entry performance centric so this probably goes without saying it's middleware components so it needs to be very performant robust API key management at the heart of any you
▶ 5:49 know API management tool this is a big piece of it so we needed to have a robust kubernetes native a way to handle it you did have an extensible plug-in framework so again you'll also see this with API management vendor solutions it's maybe not a good idea to hard-code every single possible thing that you might want to do into an API gateway but create a way where other people can add their own functionality so if they want to do their own complex transformation for their business use case for example they can do so it need to be kubernetes native so kubernetes is all about its metadata and when you have your Gateway in the context of kubernetes you can do some pretty cool stuff with canary
▶ 6:32 routing and also things so we'll look at in a second but you know custom resource definitions you're able to create kubernetes native specifications and you get things like storage for free divers tooling so the gateway is just one piece of it but you can't have a full API management solution without tools for tracing analytics and monitoring automatic API key generation etc so it needed to have this whole ecosystem of tooling around it and needed to be easy to automate and this is one thing that we did not get with our vendor solution that we were using before and by easy to automate it not just that it need to be easy to automate it needed to fall in line with the same patterns that we were using for
▶ 7:13 our existing cloud native automation so that spawned Canali logos very new and there's stickers at the sticker table in our booth so feel free to pick them up we have a lot of them so what is canale Canali is a lightweight kubernetes native API gateway that together with robust to network policies or together with network policies provide a robust open source solution to kubernetes ingress API management and API security so I'll talk a little bit break this down a little bit as we go through the next couple of slides but first I want to go back to the picture that we looked at before and look at how with Canali this picture changed
▶ 7:57 so basically traffic would all go through the EOB and now we'd be running our API management solution which would also do our ingress force and traffic would land on one of the pods and it would get routed to the service so the complexity reduced tremendously resulting in a lot less infrastructure needed and then yeah so so now now let's look at some of the differences between a gateway and a service mesh so a lot of you have probably want to talks about a service mash talks so SCO envoi link your D etc so we have Canali at the top and then we
▶ 8:41 have our three application pods on the bottom and this is what we want to happen we want everything to go through Canali our API management layer and traffic to land on a pod problem is that this is also passable by default and this is not desirable because service service on the left could talk to service on the right and not get managed not get verified you know with the API key etc but kubernetes provides a native construct for us to fix this Nats network policies so we control these lines in here and say hey only traffic is allowed to come from Canali and so it'll make this um it'll it'll make this this traffic not possible and so with this we're able to take we're able to completely get around the
▶ 9:26 service mesh and make our clusters a lot more lightweight and just have the gateway model alright so I mentioned Canali is lightweight and when you think of an API management solution rate limiting and quota come into their some people think as quotas a subset of rate limiting but just take that and/or take that as a whoever you think about it but we needed a lightweight way to basically enforce this across multiple pods so if you're using a vendor as an API management solution or basically just like you know this monolith application it can you know check its database get an information back then perhaps see the
▶ 10:10 requests but one thing Canali boasts is zero network overhead on any requests so when you Dan incoming requests the only network traffic that happens is the actual proxy to the backend application and back and so basically Canali implements a ventually consistent method to do base to do rate limiting and so what will happen is we'll have an incoming request here on the right it'll hit one of the Canali pods that pod will talk to at CD / g RPC and then via G RPC it will let the other Canali pods know that has had a traffic point that it's able to update it sufficient in memory store and we have eventually consistent rate limiting so this set CD is different than the actual kubernetes at CD or at least
▶ 10:53 that's the recommended approach so Canali is open source open source it's it's a very very new project it's like two months two months old we've gotten some good traction on it so far and it will continue to grow but now let's dive into how Canali works from like a user side of it and this gets into more of the custom resource definition side canalis a couple weeks away from a two dado release and so the specs will be two title specs that we'll be looking at so just to give a brief about custom resource definitions not going to spend a lot of time on this but as you probably know keynote mentioned
▶ 11:38 it this morning but kubernetes is a very API driven application and there's three parts to every API there's a group name a version name and a resource name and so like if you look at the changelog you know things might be moving from one version of the next a resource may be graduating to a different version or they may be moved around to different groups I think one date did some of that and Canali does the same thing so Canali has a group name canale Tadeo versions and then resource names and it has four major resources which we'll look at but this is basically how it extends the extends the API so the major
▶ 12:20 spec is the API proxy spec and these three examples provide examples a lot of functionality not all the functionality but the main thing to note is the different type of backends that you can do so Canali allows you to proxy two services inside the cluster allows you to practice services outside the cluster and allows you to proxy to mock targets which define mock responses that can be returned and this is key for or the most interesting here's the middle one is the external endpoint so it's a large organization will have a lot of our services running inside of kubernetes but what about all the services that are running maybe on the legacy applications you're still going to need some api
▶ 13:04 gateway solution to do those right but with if you think about it you can still proxy traffic to legacy applications but still have part of that as a cloud native so you can take your API management piece and make that cloud native and it has a lot of benefits because you get things like storage for free you get cloud native configuration or kubernetes native configuration you get a nice pipeline around permissions as code which I look at it a second but that's something that that is cool and will be new in - dado but the other thing you'll see here is and we'll talk about plugins some more in a little bit but you'll notice a plug-in section and this says what plugins will get executed in every response or on every request and one of
▶ 13:49 the big things for us is we needed a way to have version controlled plugins and that was something that was a foreign concept to different products that we had looked at so here we're saying that we're going to use an API key plug-in on every request we're gonna use this version of this plug-in and then we're gonna pass in some arbitrary configuration to this plug-in that the plug-in will need to do it's logic and it's a list of plugins you can create plugins you can add them to your proxy and it will run in a synchronous manner so this is the API proxy spec we'll see a demo this in a little bit but next is the API key bindings back so Canali files in our back like approach so we
▶ 14:33 have API keys which are abstract from which are abstract it from proxies and then you have API proxies and then you need something that basically binds a set of keys to approximate the API proxy spec does and it basically says here's a list of keys that have access to this binding and this example were giving Bob's API key access we're saying that whoever uses this binding Bob's API key is able to call this proxy 100 times a second and by default they have access to everything let them let them call everything except if you come in on the slash balance sub path then we're only gonna give you permission to use the to the get HTTP method this is just a simple example of how you can configure
▶ 15:15 how you can configure permissions but notice that it's completely abstract from the from the API proxy another thing that we're doing in v2 here on the right with the API key spec is we needed a way to enforce or not enforce but to allow for non-breaking API key rotation and so we follow we bump back and forth a few ideas I github and ultimately we landed on this one but say you have an active API keen you need to rotate that you'll basically create another version of a revision of that API key and make it active and then you'll keep the old one active until teams are able to update their projects at which point you'll make it inactive and then eventually removed from the
▶ 16:00 spec but this allows non-breaking API key rotation any API keys back then there's a mock targets back which I'm not gonna spend a lot of time on but if you do use the mock target back-end option just way to configure a mock response Canalis a go program so this just shows how you can configure go if any of you are go developers it uses Cobra and viper so standard configuration you can do off of that I'm so plugins so Canal uses the go pack or the the go plugins package for plugins and the only requirement is that it implements the plug-in interface and you can do whatever you want within that
▶ 16:43 with within these functions that will get called in the lifecycle request and it will execute you conversion these so for example the API key plugin is stored in the separate repository that is dynamically brought in when Canali starts and and the logic executes and then will bring in the version that we want and so if someone else wanted to design their own plugin they would create a project implement the plug-in interface they would change the configuration and then dynamically load that plug-in at runtime and then or yeah every one time and then it would execute their logic so decoupled extensible plug-in framework where they can do whatever they want there's a yeoman template which which does some of it so
▶ 17:27 tooling and we'll see more of this in the demo but the first tooling is you need a way to automatically create API keys and so we do that with Canali cuddle which is a CLI tool for Canali and we use this extensively at Northwestern Mutual and developers seem to like it seems to make their lives easier but it's just a binary that we distribute and allows them to create API keys and then they're able to add it to the cluster and because API keys me nothing until they're actually bound we're able to let developers create the API keys and then go to the service team and say hey here's my API key at its here add it to your service and that models worked for us but it's also in permissions is code too and we'll see a
▶ 18:10 demo of this next we use graph Anna and in flexi b4 for analytics and monitoring and we've found a completely configurable the default dashboard comes with like response time total requests error rate stuff like that and then we use open tracing Jaeger comes out of the box with it but the main part is open tracing so it works right out of the box with open tracing so it'll play nicely in that in that life cycle and then it's Yeager's the default client for that all right so let's do a demo that's how am i doing on time doing REO all right so I have I've went to the Canali github page
▶ 18:53 and I've executed the QuickStart steps here so I've started mini coop I have so I've flown the project I've started mini coop I've spun up Canali as well as Jaeger and influx and Griffin etc and I basically have this just as just to make sure it's working here I have all right so I have a deployment that just is that big enough for everyone I have a deployment just a simple service just returns just returns a simple message when you hit it and then I have an extremely simple version of an API proxy now this is a V once back it's very similar some subtle differences but
▶ 19:37 basically I'm coming in on slash coop con and that basically with this configuration the only thing it's doing is ingress for me so if I go here to my cluster and hopefully everything's still running awesome for do coop cuddle apply I'm just going to apply this and then try to curl the service I'll get a response I'm just using postal use for a future example but like I have my ingress working so now let's go ahead and basically add some API key management to this so I'm going to go and create an API key so I'm gonna use
▶ 20:17 Canali cuddle and I'm going to create an API key I already have this command saved somewhere alright so when I create an API key i'm going to call it Frank's API key when I give it the name I'm going to give it the location as a public key and I'm gonna save it to this file alright so here's my API key that I'm gonna use the actual API key spec has been saved here so I can open this up in my editor and there's the API key that it created for me I can go ahead and add this to the cluster all right so now I've went ahead and
▶ 21:02 added this API key to my cluster now I can go back to my specification and I can basically say so now that I have my key I'm gonna go ahead and enable the the API key plug-in on this proxy and I'm gonna create a binding which basically is gonna bind Frank's API key to this proxy I'm by default I'm gonna let it call everything over the get method but if you come in on the Foo sub path you can only you can only call it via the post method so basically it works by finding the highest priority rule so if there's no that's if the incoming request isn't overwritten by a sub path then it's gonna fall back to whatever the default rule is I'm gonna go ahead and apply this all right so that's applied now I'm
▶ 21:52 going to go ahead and try to try to curl this service and it's gonna say that API key is not found and requesting this is enforced by the actual by the API key plug-in I'm going to go back to where I can figure the key which is right here and I'm going to add a header and we call it API key then I set it equal to the value and then it's gonna give me my my response now if I change this and let's say so I was using the post method if I use this as the get method will see that API key is an opera and why so if
▶ 22:35 you see my subpath its flash coupons /qu confer bar now slash coupe con is the incoming path but we're overriding the target to slash now here we're saying anything that starts with /foo we're only gonna give post access to and the increment request matched this sub path so it knows that it only has access to post and not get and so that's why that's why the traffic or the API key was not authorized so now we're going to we're going to we're going to curl this service a whole bunch of times and it doesn't matter if it's doesn't matter for airs out I want to show well actually i want to show you a ger so
▶ 23:19 maybe it does API key equals this and then we'll set the method equal to post okay so I'll have this go a couple times now as this is going we'll go over to Jaeger and just by default out of the box you'll you'll get your tracing with it so if you went to any tracing talks specifically your own Jaeger there great talks but here we see the overall span we see the actual plug-in on request lifecycle hook happening and we see that Frank's API key is the API key being used because the backend service also instruments open tracing we can see the extra requests hitting the back end
▶ 24:01 which is this right here etc so the talk this talk is not about open tracing but it does come with open tracing out of the box and if I look at my metrics in graph Anna I can see some metrics start to come across total requests average proxy response time etc response time percentiles so see it's a very brief demo so a future work so v2 at O is like one or two weeks away but everything's tracked by the normal the normal open source flow everything is tracked by issues etc so give the demo so there is a tutorial a tutorial that
▶ 24:45 Canali oh this is where the documentation lives now it'll probably be ported ported to something else in the future but for right now this is where it lives all right so five minutes left so I'm gonna ask the first question so what about Sto so earlier I hope I kind of justified this a little bit but Canali is not a service smash it is an api gateway and so because it's not a service mesh it does separate itself apart but then also it's not an API management tool Canalis what are the only ones that I know of outside of Kong that is actually an actual API
▶ 25:27 management tool viseo does have plans in the future to add API management to their product or to the project and when that's done we will see what comes about it but I know it's more for vendors to be able to hook in their specific API management to it to fall in line with that life cycle so Canali is an api gateway now to service mesh it's a very lightweight one and everything is fully included so you get the API key management you get the API proxy you get the binding everything is out of the box you get storage for free because it's kubernetes because the customer resource definition and it when when you start to think about SEO is versus what Canali is
▶ 26:09 the differences are very different so so yeah so adapt let me answer whatever questions you have and we'll start yes I'm correct so it's so it's not a super smashed and lightweight meaning it's it's just one pod it is the Canali pod and that's it yes I'm so HTTP is the only protocol that it supports right now there's future work to add other protocols but we just want to get it out the door and currently the
▶ 26:49 only support station or HTTP um might support HTTP two as well because I believe by default the native go HTTP library supports HTTP - I don't I don't believe so now we have it in the back there support what oh so there is no official JD Beauty plug-in yet there will be that one that comes but that's the thing with the extensible framework is the frameworks there and you can build whatever you want so they're like the JD Beatty would be a plug-in that would that would live in its own repoed
▶ 27:31 that would be included in so the beauty has the beauties with open tracing is you don't it doesn't matter how many pieces are in the puzzle as long as you play nice and handle handle the context as it comes in and then pass the context along when you're done with it whatever downstream may if someone else wants to do something with it they can but Canali plays nice and that flow will it will accept and and use and then it will also you know start if not or
▶ 28:14 impasse along so yes so I say one pod but it's one like deployment so it's it's one so in our deployment we run it as a daemon set but you could run it as a deployment as well but it's it's like one like there's only one container yeah correct correct but it's it's it's a it's a replicated pot in the back yep correct so API so Canali at its best is
▶ 29:02 an API management solution but API management or the API gateway is also a proxy the thing that I didn't show is Canali does do it does canary routing as well and dynamic back-end service routing so you do get some of that as well but yeah yeah we so so we've been running Canali in production for a few months now we I think there's been times during our batch job we'll be running like 300 TPS and it's I don't I don't know if we've hit it with a lot of low pass 300 TPS but it's it's been running
▶ 29:46 in production at times that 300 TPS so it's kubernetes native so when you're inside of the context of kubernetes you could do very powerful things but then also by using custom resource definitions you get storage for free the way you can figure it is kubernetes native everything about it is like it's like coop that that kubernetes native cloud native of format and so like like with the AWS api gateway like that's abstracted from kubernetes and so you lose the context but then you also have to worry about how that's managed behind the scenes and then if it's and then
▶ 30:30 like in a cloud native context to be portable you'd have to revisit your your solution so so we're running v1 in production right now v2 will introduce the sed G RPC and and so we don't have v2 in production yet but yes it will be another one the reason why it's not recommended to use the same sed as kubernetes is because if you have access to the sed that kubernetes uses you have root access to the cluster and so it's it's always recommen even with other solutions that use extra separate entities like calico they recommend separate FTP
▶ 31:11 so there's so Canalis Greek for canal and so like all of these projects o Canali open for kubernetes open-source projects I'll fall like the nautical theme and they find a word and they turn into Greek so Canalis Greek for canal and it's kind of like that canal owns the cluster but correct so there's probably opportunity to also have them write and see and then pour them to go and whatnot but yeah and the way that'll work is they'll be in v2 they'll be like a requirements like diamo file or whatnot where you say like here's the plugin here's the version canal you use like in an it container to load them in and start the Canali pod and you'll be
▶ 31:55 good yep to make the wet yeah so right now it uses in flux out of the box but there's no reason why I couldn't use like Prometheus or something else right now it's not pluggable but it yeah it it should be in the future in terms of yes with CPUs so it out load we'll see each instance of a Canali pod to use
▶ 32:38 like half a CPU and probably like a gig of memory it's 'men it gets tweaked as we find different ways to improve on it but that's that's that's that's the v1 production numbers yes so um so Canali does do SSL termination if I look I believe it was in the spec you were able to specify an SSL object and it basically you give it the secret name and it's a type kubernetes that I owed to slash TLS it enforces the certain the key and if see a dot served as present it'll be bi-directional so so believe we're out of time thank you guys very
▶ 33:21 much I can answer questions after [Applause]