The Art of Documentation and Readme.md for Open Source Projects - Ben Hall, Katacoda¶

Playlists: KubeCon + CloudNativeCon North America 2017 (Austin)

Ben Hall explains how good documentation and READMEs drive open source project adoption.

Topics: documentation, readme, open source, developer onboarding, katacoda

Transcript¶

▶ 0:00 good afternoon everyone so as we realized in the keynote this morning the community at large it producing some amazing open-source projects we've got new innovations constantly happening new projects constantly updated hitting their 1.0 relief is and it is awesome everything is awesome but also on the flip side of this from a user point of view it can feel overwhelming it can feel a lot of information being thrown at us constantly and this is why it's super important that all of these projects and open source and general has awesome documentation and that is what I'm hoping to share with you today about how documentation can turn formal projects and make it easy for people to get started and have an impact and how

▶ 0:43 the adoption and help and so my name is Ben Hall I am the founder of cats coda cat coda is an interactive learning platform for software developers and so directly in the browser you get given an interactive environment which has been pre-configured with all the various tooling so darkly Bonetti is openshift and alongside that to guide you you get a step-by-step tutorial and so you can learn and follow along and solve real-world problems and you may have seen us on the Red Hat stand downstairs which had you then catch Khoda and the kubernetes website and have cat code and embedded in order to how bootstrap and kickstart people's experience another third the aim of the talk is to discuss

▶ 1:26 and share some of the experiences which I've had and some of my kind of viewpoints on how documentation can shorten can act and form a project and how it can go through and make a user adoption easier and a certain what your projects more successful and while my focus will be on open source where are cloud native con their fame patterns and fame approaches apply whether it's an internal API for your internal dev teams whether it's a commercial product or whether it's a side project which working on em and hopefully will turn into a larger thing at the end of the day all of these techniques can be applied in exactly the same way and so documentation is a huge area and so it can be very difficult to understand

▶ 2:10 where to begin and what to folks are and where spend your efforts and so family man he's awesome leader within a micro service community did a very timely tweet and very recently and and his viewpoint and his definition is that while creating resources and creating projects is amazing is that our responsibilities of creators to also give guidance and give explanations about how those projects should be implemented it's important that we're not just producing the toolings but we're also making sure that people are getting the best experience they're getting the appropriate experience we're not just sending people down a blind path and misleading configurations which will end up hurting them later on and this is that documentation can help

▶ 2:54 ensure that people are getting this great experience based on how we do creator think the tooling should be used and the problem is documentation at least within a development and technical sector is another time we assumed that people would just be able to read the code and understand exactly what the code is doing it and then as it how it should be implemented and 50 with another great timely tweet where historically a lot of the initial documentation was just read the tests or just read the code and understand what it was doing and how it works and this is a fundamental barrier for people to start implementing projects and getting started if they have to understand what the code is doing if they're that burdens if your code is not correctly and created to then have to vote it could be a different format and it can

▶ 3:39 really make a barrier of people going and with so many different projects and so many different implementations it's also very easy for people just to go to the next project or just close your tab and just move on to something else and the general vibe and a late a lot of the documentation which we saw can be thumb looked by this awesome diagram which I'm sure many of the theme but the premise being that we expect people to simply install X technology and then be able to successfully run it in production at scale as if we were Google and the reality is we need we need to have we need to have much clearer guidance we need to help people get from being able to install it to up to

▶ 4:22 winning at full scale and so instead of expect some people just to jump straight in our documentation and our approach needs to be building on expertise we need to give the right information at the right time to make people successful and not overwhelmed on with it complexity with confusing information with too much technical information too soon because this would just be confusing and demotivating when it didn't make sense or the user can't understand and so instead by by providing information in a peaceful way and building up that knowledge we can get them to the end goal we can get them to it's a great experience of how the product works and how the product operates without completely overwhelming them and in our experience the

▶ 5:05 documentation begins long before where most people focus on most people focus on creating developer portals and making sure that the reference API documentation is perfect but the reality is documentation starts way before that and it starts and it attached much more closely to the user experience and so we spend time taking a step back and looking at how users actually adopt products and adopt new toolings and new ideas that five key stages they'll go for an exploration like this is a new awesome technology what did it do like what problem is it solving and now they'll try and user they'll experiment fear how it works how to get started if it do that and throw problems what problem does it solve and then eventually get hooked to the point where

▶ 5:49 they need reference and they need and that expertise and these five stages map very cleanly unto what an emotional journey looks like from a user's point of view so how people will actually engage with projects so there's a technology trigger ie a new project was just announced on Twitter or a keynote like this morning announcing something like Cordy in essence like everyone's super excited it's like this could be the answer to all of my dreams until instantly they'll get with expectations and it will be probably a little bit over eggs compared to reality and so when they try new technologies and trying to implement it they'll start getting disillusioned it will be too much trouble the code samples will be broken the examples aren't and detailed enough and so stark and disillusioned

▶ 6:34 and it's not the answer to the dreams and they'll potentially move to something else and try different nology but if users can get beyond none and I'll start see the gold I'll start to see where the project sits and I'll start being productive and this is the aim and this is where we want to make sure people going to within our documentation journey and so these five stages map very cleanly onto this kind of emotional curve of user that having to and so when we're thinking about the types of documentation which we want to produce and how we want that documentation to work we want to be making sure that we provide in the right information which Maps this journey about how people are experiencing and learning more about the technology itself and so I just want to start by going into some more these

▶ 7:17 stages in more details so let's start with the extra their exploration and fundamentally at this point you thought like why should I care like why is this project important to me and at this point they're confused they may have seen a tweet hundred reports characters didn't go into much detail or it may have been mentioned in a talk and if I okay I'll look at that later and so they need some more information and what authen technologies are great it can be confusing when people land on new websites so this with him that example of docker while docker has completely transformed our technology industry and synthol amazing work when he gone to their home page the third thing is discussing if kubernetes and from a user committing blind to the container

▶ 8:00 ecosystem this is somewhat confusing not only do they not know what docu is but also now being introduced to the concepts of kubernetes which did more things they have to understand and have you know the ghaib for three more keeping it for more at the daka website they introduced daily lives which is great but again it's not really explaining what containers and what docker are and then they'll drop people into the far ends like how do you contribute back to docker but contribute add something called movi like what's Moby wave docker and so this journey can become extremely confusing from a user's point of view and again this is why I think documentation starts really early on in the journey because this is all forming into users perception about how the project works

▶ 8:43 and how they can start opting it and using it and so if we look at some other projects which I think a really good strong examples keeping ettus so very clear instantly what the purpose of Cupid enters is if production-grade container orchestration and so instantly you know whether this is something that you're interested in or something which if not for you and you can move on and the fame as you're going through it it's a very clear tagline it's got some good explanations and then it's telling you those words go next I in this case try the interactive tutorials so you didn't know where they were expected to go and how that journey is expected to flow very similar with project calla girl and tied to Keira it's very clear what that project is there what the purpose of why

▶ 9:26 it exists and again be new than either how to get started or where they can go for a demo and more information same with terraform again great example so clear tagline three-step lines and giving people the pointers in a very clean direct way and so when people are motivated and excited and no one to explore more they know instantly whether it's relevant for them and then where to go next and this is something which we trying should be aiming for and even complex things like google still managed to have key touch points like using google core infrastructure machine learning and ease in all of the great buzz words to get people excited but still telling people how to go and where to go next and so once we get people over the initial buried like diffa the project which i should be paying

▶ 10:11 attention to we want them to get started as quickly as possible and what we want to demonstrate if answering a question like do you actually fold my problem if we're looking at something like calico is like cold secure networking okay if that the problem oil is that the answer to my secure networking problems or they're another vendor which i should be looking at and have many barriers which people putting away in order to make this what should be a very awesome exciting experience for the other somewhat problematic and having lots of barriers in a way and the one which i personally enjoy if here download this 10 gigabyte virtual machine image and in order to have a demo of how my product works and like this afford with danger if you're think that you're in somewhere like Australia

▶ 10:56 ten gigabytes is a significant amount of data to download even in central London ten gigabytes will still take you an hour to download and anything VM so how do you win though by are you going to start introducing and teaching people about Fagin teaching people about hyper-v KVM like how did that work and so there's all of these other questions that the user is going to be engaged with when they just want to simply figure out whether your product is what they need or not and to help them get started what we see more of more frequently is deploy using cloud formation or deploy using agile or giggle again now you have to start explaining how to use cloud formation or how do you particular cloud vendor and at the same time hoping that users forget remember to turn off the machines

▶ 11:40 at the end of it and they don't get this an expected invoice for $300 when they all they wanted to do would figure out whether they wanted to use the product or not in the first place and so these types asking for people for a huge amount of commitment often before they've actually even proven or been shown whether the problem is right for them or not and so there's a few techniques which we find beneficial when we're training people and getting people started and to help with their adoption and how people understand what a product offers with kubernetes and we're doing with the qnet his team we've got the cat coda interactive test drive and so people can start keeping it is I start Cleve the NetID cluster in the browser they don't have to download anything

▶ 12:24 they don't have to install anything they can start deploying containers and just experience what it looks like and how it works and how it operates we see it with other kubernetes projects as well so the people from hipty oh and we have costner and so you've got the Jathan netcode defining a cabinet it's cluster and then live in your browser you can tweak and edit it and fee instantly to transform Djamel and so you can start putting together and exploring how this new approach works and whether it's right for you and whether it's something which you're interested in and for me this is all part game people are excited them motivated in standing where the product 6 which is the whole purpose of documentation another great example is

▶ 13:07 stripe so stripe is the payment gateway very API developer driven and so the first thing you get an experience when you go into that documentation site is a code snippet and then it walks you through and gives you information so tell you what you expected and what's going to be required of you when you're implementing the API but then also give you kill commands and so you can execute these kill commands locally and actually experience and interact with the live API and so this was in in this case okay a customer and you'll see that customer created on the journey and then you can Kofu and simulate creating car payments and the end wins everything's successful within five steps you've gone for the entire stripe API integration journey

▶ 13:50 you've made live API calls and then it tells you where to go next like how can you continue being successful how can continue pushing forward and keeping that motivation and that energy about learning this technology alive and so it's a really nice example of well how you can demonstrate API is in a nice clear way and the flipside it did also still into the barriers like it's got dependency and curl like what if I don't have curl available boy if I'm on my iPad or if I'm on my mobile I still want to be able to go through that great journey and take the documentation but I now have get dependence in this blocker and so a view into treating these different elements it's important to think about the context about where the user is at that time and what the user will be

▶ 14:33 doing yes most people will have curl but it's something out which they need to consider and so there could be alternative ways to solve that problem in a much simpler experience from the users point of view but the premise and what we're trying to get to at the end of this kind of initial gala 30 stage at initial five minutes where people are trying to learn an experiment is you want to showcase what your product is offering showcase what it can do and kind of allow users to experience that and allow them to prove for themselves that this is something which is were I'm continuing and worth exploring for more and if gets returned to the point of kind of their onboarding and now we start getting to into the more of the traditional longer form documentation and if the point of the user like

▶ 15:17 they've been shown what it does and so now it's totally don't want to explore more they want to start using the product interacting with it and they need some support documentation in order to be able to do that successfully and ironically in my experience best documentation into documentation which we don't have to write if we don't need to write long explanations on detailed blog posts or long detailed guides to help people then or the better and fundamentally that because people just don't really read on the internet like historically we're always skim reading we're always grants in were distracted with a multiple different tabs open and so it's very hard for people to concentrate and read and

▶ 15:59 digest these long-winded very detailed long for documentation guides and so the different ways that we can break that up and we're very fortunate by writing technical documentation about the tips and tricks that we can use obviously code snippets are a great example developers who will stop and read and digest a code snippet Natur easily than a paragraph of explanation defining how to implement something one thing which we or which I personally get really frustrated by it half completed code snippets so with this it's a great example it's very clear that we work fine it's got even some nice comments but if I took that and try to win it in a goal and rebel it wouldn't work because it doesn't include the additional functions the import

▶ 16:43 statements only the additional additional information nor what's one needs in order to be able to experience to its full extent and so it's important to kind of think about how you can included all of this noise and there's additional information without overload and use out with how it being come to complex until what we're seeing a trend of is people having buttons or show/hide toggle and so that when you have your code snippet but before it's all hidden and it doesn't have the additional import statements doesn't have these additional function definitions etc but if you don't want to go ahead and execute that they can very quickly just discover it with a button and have that information there and so they don't have to go off and discover and look around

▶ 17:28 for additional bits of information so this we think of the really nice technique we're also seeing more and more integrations with a/b is to Spotify we develop in their developer portal and for now part of that documentation they're getting people to authentic an authentic a and gain an access token wouldn't they've gone through and has authorized and they've got the access token they then update automatically all of the code snippets which uses that live real person's personalized access token and so when user going through that copy and paste in there trying to execute the code they don't have errors they don't have weird and things which don't work they get great experience out the box and without having to change anything and so what I've appeared in the documentation

▶ 18:13 needs code snippets sometimes we hit problems on we hit areas where it's actually just very difficult to explain and if they difficult to describe accurately how something works and we've had um so many of its experiences with doing things like installing kubernetes Kuban it is is somewhat of a complex thing to get set up and getting thought hence why we have awesome tooling like tectonic and open shift and all of the google managed services but also it's an highlight that there were the problem there is a missing gap within the workflow without developers are experienced in about how they're going around and how they're getting kubernetes set up and so a great example is were work with the team of doing around cue battle where instead of

▶ 18:56 writing these long formed explanations and documentation about how you get it set up there you have built at all which will do it for Don and so instead of trying to document the missing UX experience they've wrote and they've all tomates that for people to get started with and so now as people are experiencing it and people going through it's more of a self documenting clear definition about how you that can get started they can go in they can talk ooh bad on it and now do order the leg work and all it's a hard work for the developer which also means we don't have to document all of the hard work when he said happening underneath so once people have got started they've started to experience the product they understand a motivated they want to learn more we

▶ 19:41 gain to a stage of guidance and discovery and so this is now where people have ideally solved a problem or they they identified that this is potential solution and they're now exploring and looking at what other problems can be solved what is a document what problems do that solution offer and this is where you start getting the more traditional longer form documentation which is very focused on particular problem spaces and it can get very niche and this is also a great opportunity to start promoting and highlighting things which are happening within your community right and documentation solely from a team it's hard work it's time-consuming it takes a long time to produce very good detailed documentation

▶ 20:24 and also it can be hard to discover all the documentation which is required and so what people like digitalocean are doing is leaning on that community and promoting all of the autumn work all of the awesome ideas that the community has and then showcasing that part of their documentation and this then video is how people can get started and so when you're 13 things like how do I win Prometheus and Abby - the chances are that you'll land on digital ocean because that's got the best documentation in Google and I've advert it's ranked more highly and I respect the community and they respect the contributions and so now there's over 13,000 different tutorials created and digital ocean will compensate people when new tutorials are accepted and

▶ 21:07 added in to the catalog because they respect that community effort so it's a really nice way to start reaching along more the longtail documentation and long tail of guidance and discovery that people may be looking for because the community probably been and dinner already and a really great way to foster this and help if I having things like white as guides and defining what's expected from a contributor how do people get started how can people start writing and sharing content with the product and with the company and for this is a really nice way and we've seen this with some of the work that we're team are doing with the portal and so as part of the 11 release works all of this it's on github it's

▶ 21:50 all very open and it creates that learning pathway about how users can go ahead and experience open shift and what different problems that can solve in an interactive way and so as part of this because people don't read on the internet and because people can't focus on long-form tags everything poking down into the step-by-step processes and more of a bullet point very clear heading so clear signals about what's the users expected and how they can actually go ahead and throw of different problems and if you've started building these awesome documentation websites it can lead to a problem our discovery and how do people find where other valuable content is that's something which I was super impressed with is from Twilio and so when you go into that dogs website

▶ 22:34 there are sketchy questions what's your coding language and what product are you interested in learning more about so you can go foo you can select nodejs and then select them to like SMS or phone calls or voice over IP and then it will take you to the appropriate getting started guide and so instead of going through huge amounts of links and indexes about where documentation is instead users can just jump straight into the game started for the product and for the problem in their language and in their contacts where they're familiar with and so it's a really nice workflow about how you can have lots of different content and make it very easy for people to find their very easy way another approach towards Google who have done is having a single page which then

▶ 23:19 has examples in all of the different languages so in this case this is their big query API and instead of having different pages so Pete I paid for PHP fdk I paid for go pay for tisha they've got a central point where people can navigate to and discover the required documentation and then they can flick between the different examples based on what their language is and so it makes it really easy for people to get started and see the differences between different languages that they may be implementing their technology and once we start getting to this point we start getting to the traditional reference type material ie how can I start becoming an expert with this product and this is generally the documentation which we generally start writing first

▶ 24:03 it's the one which is focus more in the long tail it defines all of the different API events different API calls and we can use templates such as one from stripe and read the docs in order to highlight and showcase that but we also want to see about how we can make up phirni make that learning experience more interesting from a user's point of view so instead of it being logs of blocks and traditional ways of documenting methods how companies again using too much fine an example make it more interactive and so they are helping people explore and understand what's being offered by the API in an interactive way by having an API console and though people can use real detail or they can fill in some sample data

▶ 24:47 automatically and start interacting and exploring the API that are trying to get more information about and then obviously end of the covers they've got a longer form traditional guide if people do want to know what all of the API calls are all of the API calls are and the methods which are possible and so that for me is how our users journey maps from them getting familiar with the product and kind of becoming familiar and then getting to the stage of where different types of documentation fit into the users different mindsets but what about Lemus like github it now a huge source of information and a huge central place where people go to explore

▶ 25:31 and discover what's possible and so the readme is now an equally as important as IU third a product homepage for example and for me the readme thought about signposting it's about giving people just enough information that they can be productive and they know where to go and where to find out more information and it's really important because the readme fetched a tone for the entire project it says to tone about what is considered important how much effort had got into the project in some cases and whether it be maintained or not and it kind of gives you that initial instant experience about whether this is something which you're gonna build confluence and Trust and build on top of or it's something which you may not have so much faith in and so again if you use

▶ 26:15 crude I won't keep Adam as an example it's got some great time posts it's got information about where to go for support and how to get started it's got something for information about road maps etc but it doesn't really tell you how to install it it doesn't how you where to get started how to push forward and so because it's open source so with a pull request pending which I will submit but generally my honest opinion this is what I kind of look for when I go into a open-source project the another kind of cheat point which I'm generally interested in understanding more about perfectly what are the goals of a project why did that project exist in a search base why did someone invest time and effort in creating it because the next important thing for people to or for me to

▶ 27:00 understand and then it like how do I install it how do I get started do I need a package manager what kind of API see that did they require etc and then starting to get productive and actually user and because it's very open source and we want to encourage that community giving people an indication of the status by if they say an actively developed on or has it been put into maintenance node and we no longer push them forward with it and then of course things like contributor code lines code of conduct so we're building a good strong solid open source project and then the license to make sure that everyone is happy and so a really good example of this is minicab it's a great explanation at the top about why many keep exists and how what problems it so

▶ 27:44 then it tells every people they're clearly given an operating system that is how you install it and then it moves on to a quick start guide so it tells people right once you've got many keeping stored this is how we start deploying pods creighton clusters and fetcher and then finally summarize with hints about how you can get more information putting into advance segments contribute to guidelines and a community as well and so it's really nicely formed that at any point no matter how experienced you are or inexperienced you are when you go into mini Klebe it had all the information which you need in order to be productive them and make sure that you're making the right decision it's a really good technique which I think are sometimes overlooked when you're thinking about readme is that you have the ability to

▶ 28:28 include images which are always great but you can also include gifts for this is an example of a gif which will deploy openshift and then start deploying pods and start deploying services so again it's a nice way for people to get an understanding of where that project fits in a nice interactive way but we also don't want to start overloading and paints humans information into the weed meat because that can make no noise or more confusions and so we can use a details element to start hiding links and hiding examples behind the drop-down and so instead of having all of these examples and next steps within the mainline text they can be hidden and so if you did want to find it and they want to know more than they can but by default

▶ 29:13 we're not overloading them with image information and because we're on github we important that we start thinking about how we can foster and how we can sighing a community and so by having a Doc's in a separate website or a separate repository makes a great play from people to know and understand how to start contributing they know exactly where they need to go and if not being it doesn't have the noise and the other conversations happening around the actual core project and so docker is a really good example as is the kubernetes website and the documentation and so people know and it can start complete contributing they can start with an issues and improvement if they want to without getting in the rest of the noise about what has happening in a proposal

▶ 29:57 and it also starts you having more focused communities things like the special interest group around the documentation and the docs print which happened yesterday it's a great example of how with this focus energy in the focused community you can start pushing dog forward and much more interesting passionate way and finally what we want to do is start making it easy for people to give back and give feedback and share if they notice a typo of their noses and stake making sure that they can actually highlight that to the team making sure that that can be improved for everyone else in the community and so within communities documentation every page has an edit button which takes you to the relevant part in the github repo and so that people can start easily making changes and making suggestions about how

▶ 30:42 it can work the one issue I do have with this is we need to think about like how does that work from user who didn't understand get or didn't understand github and like interesting concepts of pull requests and Forks and either the aspects can be too overwhelming especially when you just want to say like it should be a small typo needs to be fixed and so instead of people will put some people to work down a part of a fork and a repo making change directly what we've started doing with our dogs are people pushing and encouraging more people to submit issues and it's highlighting the fact and then one of the community team members can go ahead and make the fix directly and we don't have to worry

▶ 31:25 about contributor guidelines or a good life into agreement etc and so these are some of the tips were tweet experience in which we find personally valuable and what we have found valuable when we've been promoting products and encouraging other users to get started and all of these new shiny technologies but a common question is who creates the best occupation we've seen lots of different examples are things which I like and things which I find beneficial and what you'd find beneficial but who does the best and if it's a hard question because every documentation different and every project have different objectives but an interesting one is when you start looking around the community and start seeing what other people with recommendations are and so there's an

▶ 32:08 awesome blog post by cristiano better in london about the developer experience lessons from lego and this is not something which you generally look at either source of inspiration for technical documentation but Lego is the perfect example of what awesome documentation looks like it's very simple and straightforward about what you're trying to achieve from the outset I you're building a car or you're building a plane it doesn't tell people to start farm just hit one Lego brick and there at the end we thought instead it's building up people's confidence is showing all the different stages at the Void point other white detail about how people can be successful and so instead of overloading on which information it's just providing just enough that they can

▶ 32:52 make the next step and the next leap forward and it's also very good at promoting a community it's very good at fostering new ideas and so instead the particular Lego set you would like to see you can go on and you can submit an idea and if it gets enough sold then Lego will produce and Lego will actually create a Lego set in this case around Red Dwarf and because it's been designed in this way it allows people to be very creative it gives people enough support and enough ideas that they can build whatever they want which is the whole purpose of people using log in third place and if step-by-step nature has allowed people to get to the point where they can do awesome things like build u-boat elephants or build entire parts

▶ 33:36 of America and so I think that's a great example of what we can look for when we're thinking about the documentation and think about what we need to do and how much detail we need to go into is how would Lego do it and so with that I'd just like to summarize so when we're thinking about the art of documentation it's important to make sure that we're giving the users the right information at the right time so how can we not lead them down a path of here just install the thing and now go ahead and draw the rest of the owl instead we want to make sure that we've given them the white bits of information and then we this needs to be doing as people start engaging and start using products and as some started opting it their emotions change and a third we want the

▶ 34:21 documentation to be able to change with that and provide them longer form documentation as they want to start being more productive and start being more of an expert with it it's important think about what are the forms we can provide like can we stop doing more interactive elements I didn't things like what we're doing sketch coder or API sandbox is like with Spotify in order to give people a better experience about how they can explore and how they can start playing and if we're looking at what the readme needs to be doing it needs to be that fine post that gateway where people can go where all the information is so people have all the information which they need in order to make sure that they're solving the problems which the autumn project is doing and with that thank you very much for your time

▶ 35:05 if you ever would like some documentation advice or me to look at support and how I show ideas please do reach out please do email or tweet me and with that thank you very much enjoy the rest of your conference [Applause]