This post is a transcript from James Messinger’s lightning talk at
The Austin Homegrown API Meetup in May 2019.
Hi everybody! Can everyone hear me okay? All right. We’re going to talk about documenting APIs. I see that there were a lot of engineers in the room, and also a lot of documenters and technical writers in the room, so it’s going to be a good topic tonight.
I am the Product Director at ShipEngine. If you don’t know what ShipEngine is, we are a shipping and e-commerce API; so if you need to ship packages, track packages, validate an address, really anything related to shipping, our API can do it and we can do it for any carrier in the world. And then, similarly, if you are a e-commerce seller and you sell—maybe you have a Shopify store, or maybe you also sell on Amazon or eBay or any other channels—ShipEngine, our API, allows you to bring in all of your orders from all of those channels into a single place, fulfill them using our wonderful shipping APIs and then update the marketplace’s VR API to let them know that the order has shipped, and let the customer know what the tracking number is. There you go. That was the one minute spiel on what ShipEngine is.
All right, so here at ShipEngine we are just about to start a big revamp of our documentation; and our current docs, they’re pretty good. But, we’re at the point where we’re ready to level them up and kind of, you know, bring our A-game to our docs.
So, we’ve been doing a lot of thinking about what we want in our documentation, the ways that we can use our documentation to improve our overall developer experience, how our docs can actually tie into our broader content strategy and improve awareness and growth for our products? And then finally, the age-old decision of: Do we build it ourself? Or, do we use an off the shelf product? So, I’m going to talk about all that tonight.
Let’s start off with types of documentation.
Broadly speaking, every API—in fact, probably every software product—should have these four types of documentation. The first one that we… and this, by the way, I can’t claim; this is a tiny little link down here to Divio, who I’m actually very grateful to for this concept… but your tutorials are your absolute beginner content.
“Broadly speaking, every API—in fact, probably every software product—should have these four types of documentation.”
This is the absolute beginner guides that are a step-by-step guaranteed recipe for success. So, this will include really basic things like how to create an account, how to get an API key. The important thing to know about this is that it’s learning oriented. It’s not necessarily solving a business purpose. So we have a tutorial on ShipEngine’s website that’s how to build a Twitter bot. Not really a real business use case, but it’s really easy for a beginner to wrap their head around, right?
How-to guides are the next level up. So, a how to guide is very much like a tutorial; it’s still a step-by-step guide, but it’s learning oriented… or, sorry, it’s goal-oriented. That’s what I meant to say. The goal of a how to guide is to literally help someone solve a business problem. So in Ship Engine’s case, we have how-to guides on how to print a shipping label, how to track a package, how to validate an address, that kind of thing.
Explanations are your broad concepts. So, these are not necessarily about any one endpoint of your API or anyone feature of your product; these are broader concepts about the domain that you are operating in, right? So, in the shipping and logistics domains, we have all kinds of broad concepts about how warehousing works and how shipping workflows work. And we’ve got explanation guides on all that.
And then finally you’ve got reference docs. Now, reference docs… every developer in the room knows what reference docs are; these are the thing that you need in order to do your job. It tells you what the endpoints are, what the fields are that you need to send to them, which fields are required, which ones are optional, data types, all that fun stuff, right? That’s your reference guide.
This is the same four types, but now we’re looking at them on a spectrum. So, what you see here is that tutorials and how-to guides tend to be more a list of practical steps, whereas explanations and references are just general knowledge tutorials; and explanations are most useful when someone is first learning your product, learning what it can do for them, whereas how-to guides and reference docs are what someone needs when they’re actually coding, actually trying to solve real problems with your API.
So, this is just an order of priority. So, it takes time, it takes investment to write all those types of docs I was just talking about. So, where do you start? Well, for my money, you start with reference docs for a couple of reasons. One, it’s table stakes. It’s literally what every developer is going to expect you to provide. And secondly, it’s really easy to auto-generate them. Like, if you’ve got a swagger definition or an open API definition, you’ve got a postman file; there’s any number of tools out there that can just auto-generate really nice reference documentation for you. And if your API is internal only; so, it’s not a customer facing API, not anything you’re charging money for, it’s just something that you want to be able to document so that another team and your company can use it? Reference docs might be all you need. You might not need the rest of these.
So, once you’ve got your reference docs settled, then we go onto how-to guides. The reason that you want to move onto these next is because these actually help people solve real business problems. So in ShipEngine’s case, you could figure out how to ship a package using our reference docs; you just have to figure out which API calls you need to make. But our how-to guide is a step-by-step guide. Here are the three calls you need to make to solve that one problem.
And then finally you’ve got your tutorials and explanations. The reason those were at the end, it’s not because they’re any less important, it’s just that they tend to be more long-form. They take a lot more time and effort to write. So, the other ones are easier wins.
So now that we’ve talked about the different types of documentation, let’s talk about how you present them to your user. So, it would be really easy… really tempting, in fact… to think, well, I’ve got four different types of documentations, I’m going to have four different sections to my docs. And you know, if you want tutorials, you go to the tutorial section and you find the tutorial for feature number one; and if you want a how-to guide, you go to the how-to section and you find the how-to guide for feature number one.
The problem with this is that… two problems, actually. One is you’re forcing your users to understand how your documentation is structured and figure out that, oh, what I’m looking for is a how-to guide, so I need to go here. Really what your users want is a solution to their problem. They don’t care if that solution comes in the form of a how-to guide or a reference doc or whatever. So, a better way to organize your docs is by feature or by use case, where I can simply go to the documentation for feature one and all of the tutorials, all of the reference docs, everything about feature one is all grouped together.
“What your users want is a solution to their problem. They don’t care if that solution comes in the form of a how-to guide or a reference doc…”
Let’s talk about developer experience and specifically how developer experience and your documentation fit together. So, the first thing I would advise is that… think bigger than your documentation. Developers who are using your API have a whole tool chain that they’re using along with it, right? They’re using Postman, or they’re using Stoplight, or they’re using any number of code generators. My recommendation is that in your docs, you should provide files in those formats that they can, you know, one click download.
A Postman collection is easy; you go, you fire up Postman, you create your Postman collection, you add the link to your website, people can download it and immediately start using your API. Open API, otherwise known as Swagger and JSON Schema, are both supported by literally every tool out there, especially in the API space. Everyone supports Open API these days.
And then the final one that I have there is the ‘Run in Postman’ button. And you’re wondering like, “That’s the same thing as a Postman collection.” Does anybody here know what a Run in Postman button? Oh, I’m about to give you guys some awesome knowledge. Okay, so Postman has this feature called the Run in Postman button. You could just take a Postman collection, export it as a JSON file, and then provide a link to download that JSON file in your docs; or, in Postman, you can build your collection, you can generate a little Run in Postman button… it’s like two lines of HTML that you drop into your docs… and what that button does, is with one click, the user… it automatically launches Postman, downloads your Postman collection, and opens it for them.
So, one click and they’ve got your API up in front of them, ready to use. Even better than that, the Run in Postman button has an API of its own, so you can do things like customize it. So now, imagine personalization; so you click the Run in Postman button; it not only downloads the collection, but it’s a collection that has your API key already in it, so you can just immediately start making calls using your actual data. Really cool feature.
Next in developer experience, we talk about SDKs. So ultimately, every REST API needs an SDK. The only question is whether you’re gonna write it or your customers are going to write it. If you’re a Python developer and you’re trying to consume a REST API, you’re going to need to write Python wrappers around that REST API; same thing goes for PHP or Go or any other language. The problem is that if you’re not providing the SDKs, then each of your Python customers is having to reinvent the wheel each time, and with different levels of quality, right? Some customers are going to completely forget about error handling, and other ones are gonna forget about retry logic, and et cetera. So, it’s better if you can provide an SDK that has all that stuff built in it, so that you can kind of encourage some best practices. You know your API better than anyone else does, so you can probably build a better SDK than someone else could.
“Ultimately, every REST API needs an SDK.”
Oh, the one last point I wanted to make on this is that once you build an SDK, it becomes the primary method of using your API, right? So, a Python developer who downloads your Python SDK could care less that behind that SDK is a REST API. To them, your product is a Python library. The REST API behind it is just an implementation detail. And so, that’s where this comes back to your documentation. When you have SDKs, they are the primary method of using your API; therefore your documentation, you’re no longer documenting a REST API. You’re now documenting a software library. So it’s not about endpoints and verbs, it’s about classes and methods.
Your code samples should actually use your SDKs. There’s a lot of tools out there that will generate multi-language code samples for you, and they’ll generate Python and Go and PHP, but the code samples they generate are just using raw HTTP requests and responses. So, it’s 15 lines of Python code that creates a HTTP request, sets the headers, populates the body, sends the request, waits for the response. Once you’ve got an SDK, really what the code sample should be is one line of code that says Ship Engine dot track package, right? That’s really what the code sample should be.
Because we are now documenting a software product, you need to include things that you normally wouldn’t for a REST API, things like installation instructions, or IDE and editor screenshots. So, if you’re documenting a Java SDK, then people are going to expect you to have instructions on how to set up an Intellij product or a, I don’t know, whatever other ID Java people use these days. Anyway, yeah. Oh, one other one is project templates.
So, you’ve got all these languages, you want to provide first-class support, you want to get people up and running quickly; provide a simple GitHub like Boilerplate, Hello World project for them, for each of those languages, so that someone can literally just clone that project and get started with your API right away.
Let’s talk about discoverability. So, the point I want to drive home here is that your docs are more than just documenting your product. They’re actually part of your content strategy. Talk to your marketing people, they’ll tell you all about their content strategy. Your docs are an important part of that.
Your documentation should have search engine optimization or they should be optimized for, you know, SEO, so that you can actually drive new customers to your product. People who have never heard of ShipEngine should be able to find us just because they went to Google and typed, “How do I print a UPS shipping label?” Right? That’s SEO. All right.
“Your documentation should have search engine optimization.”
Shareability: you want to encourage people to share parts of your documentation. That’s going to get you organic growth. So—and notice I said parts of your documentation—you don’t just want to, you know, have a big share button on your homepage. You want it to be easy for people to link to a specific section of your documentation. “Hey look, I found the docs for printing a UPS shipping label,” not, “I found ShipEngine’s docs,” right?
And then there’s analytics. So, having good analytics on your docs… this could be Google Analytics, it could be any number of other analytics tools… is going to give you the ability to find out which parts of your docs are most popular, which parts of your docs…or the flow that people typically take through your docs. “Oh, we’re noticing that people typically go to this page and then this page and this page,” and then you can use that information to then optimize your documentation, suggest relevant pages, that kind of thing.
Okay, so the first trick, I guess, to getting all of this wonderful SEO and shareability stuff that is talked about is metadata. Now, if you’re using a CMS like WordPress or many of the documentation, the API documentation tools that exist, already generate all this metadata for you, so you have to do nothing. But if you’re rolling something from scratch, you’re going to have to generate all this metadata yourself, and there are several different formats. There’s a JSON LD, which is mostly used by search engines. oEmbed is used by Slack and a few others; it’s a much more modern metadata technology. And then you’ve got Twitter cards and OpenGraph, which Twitter cards are obviously made by Twitter, and OpenGraph was made by Facebook, but they’re used by way more than that. So, if you’ve ever shared a link in Slack, and all of a sudden you get that nice little preview image and then a description and a summary of the page? That is either oEmbed, Twitter card, or OpenGraph. So, Slack will actually fall back to whichever one you have.
So what you want in your docs is the ability for someone to, you know if they share the page on how to print a UPS shipping label, ideally, it comes with a nice little preview that says “how to print a UPS shipping label,” and has like a picture of a, I don’t know, UPS shipping label or something like that; as opposed to just… Here’s ShipEngine’s docs, right, with just the ShipEngine logo and something generic, right?
Page per topic. So, what I’m trying to get at here is there’s… a lot of the API documentation tools that exist today generate a single page of documentation. So, your entire API is documented in one really long HTML page, and they’ve got a left nav that lets you easily jump to different sections of it, but it’s still just one page, which means it’s just one entry in Google’s search index, and the title of that entry is whatever the title of your API is; so, in our case, it’d be ShipEngine. And so, someone can find us if they search for ShipEngine API, but if someone searches for “how to print a UPS shipping label,” they’re not going to find us because while that’s in the docs, it’s buried way down on that page somewhere, right, And Google doesn’t think it’s very high priority.
So what you want is separate pages for each section of your docs. Now for you, that could be a separate page for each endpoint, or maybe it’s a separate page for each feature, or whatever; it’s up to you, but each page becomes something that people can search for, something that people can share… I talked about links earlier, so obviously having a separate URL for each section of your docs makes it nice and easy to share… and because they’re separate pages, you can have separate metadata. All that metadata I was talking about earlier, you can have separate metadata for each section of your doc. So, rather than just having… if you just got one page, you just got one piece of metadata that says this is Ship Engine’s docs, and now when someone shares a section of your docs, all that the little preview says is Ship Engine documentation. What you want it to actually say is “how to print a UPS shipping label.”
The other one that I want to talk about with was analytics. So, one of the benefits of having separate pages for every topic of your docs is it makes it really easy to do analytics. Right now, you’ve got separate pages, it’s really easy to monitor that someone went to this page and this page and this page; you can kind of do that with the single page version, but it’s a bit harder because now you’re trying to figure out that they scrolled to this part of the page and then scrolled to this part of the page, and scrolled to this part of the page. It’s a little tricky.
Progressive enhancement. So, this is an old piece of advice for SEO that’s been around since, I don’t know, the ’90s or whatever, which is like, “Don’t rely on JavaScript. Your page should be able to render without JavaScript because most search engine crawlers don’t use JavaScript,” and that’s good. That’s important. You do want Google or whatever other search engine to be able to see your page, and you shouldn’t rely on JavaScript to render the content of that page. But it actually also ends up being important for sharing. So, when you paste that link in Slack and you get that nice preview, Slack is going to the page and it’s getting the information off that page to give you that preview, and the only information it gets is the static content. It doesn’t execute your JavaScript.
“You do want Google or whatever other search engine to be able to see your page, and you shouldn’t rely on JavaScript to render the content of that page.”
So, if your page is actually blank and you’re waiting for React to load and then load your entire React app to build the content, you don’t get a preview in Slack. All right? And same thing goes for Twitter and Facebook, and everything else you might share on. And then, of course, other benefits to progressive enhancement that have existed forever, which is if you don’t rely on JavaScript, then even if your JavaScript fails to load or has a syntax error, your page still loads. You still get your content.
The good topic, build or buy? This is what everyone was waiting for, right? You were all just excited about this one topic; I’ve been building up to it. All right.
So, yeah, so reasons to buy: we are spoiled for choice in the API community. I don’t know of any other area of software engineering where we have as much choice for documentation as we do in the API world. And you know, if you’re in Java-land, there’s Java Docs and maybe a couple of other choices, but that’s it. And if you’re in JavaScript, you’ve got JS Doc and that’s it. But in APIs, you’ve got a plethora of really good tools that can generate really good documentation painlessly.
“In APIs, you’ve got a plethora of really good tools that can generate really good documentation painlessly.”
Some of these tools do more than just documentation. They’re actually an end-to-end life cycle tool where you can design your API, build your API, test your API, document your API, even release your API. Again, I’m doing free advertising for people, by the way.
So, reasons to buy. So, why would you choose to buy an off-the-shelf product rather than building your own? The obvious one is faster time to market. Obviously, building it yourself is going to take a lot of time. If you want to get something out the door quickly, you just buy something off the shelf. If you’re still building your product, and you actually haven’t gone to market yet or you’re a brand new product, it makes no sense to spend time focusing on your documentation. You need to focus on your core product offering.
Obviously, your costs are going to be lower going off-the-shelf. If you have a limited budget or your team doesn’t have the experience to build all that stuff I was talking about earlier, then all those are good reasons to to buy something, or… I guess you don’t necessarily have to buy it. There are some free options, but you generally get what you pay for, right?
But reasons to build—I got to give the pro and the con, right?
It’s never been easier to build your own. We’ve got tons of great tooling out there that makes it easy to generate static sites, generate documentation. These top two rows are just different static site generators that are out there. They each have different focuses. Gatsby and React Static are both heavily built around React, and they do pre-rendering so that you still get SEO, but still get the benefit of React, as well. Jekyll, Hugo, Metalsmith; they’re all your more traditional static site generators, and then Docusaurus… great name… is tailored specifically to documentation, so it’s got some some cool features like being able to have multiple versions of your documentation, that kind of stuff.
This bottom row here is just tools that you can use with these to build docs yourself, right? So, you have an open API definition or a Swagger definition; that’s a great starting point for auto-generating, using something like Swagger code gen, auto-generating your own documentation page.
There’s also a new technology… still brand new; it’s a little flaky to use right now, but it’s got promise… which is called Markdown Extended, or MDX. It combines Markdown and React components; so, you get all of the ease of writing Markdown; nice, simple Markdown, like we all write in Slack and GitHub. But then whenever you want to have some really rich interactive piece of something, a live code editor or something like that, it’s just a single React tag, and you have it in your doc. It’s really cool.
Obviously the big reason that you would choose to build something yourself and take that time and that investment rather than buying something off the shelf, is that you get complete control and flexibility. Along with that comes the ability to do personalization; so now, you could do things like having your code samples automatically include the user’s API key, right? So rather than just having a code sample with like, “Insert your API key here,” it actually has their API key, right? You can do other levels of personalization. Maybe you have a pricing model where you have different tiers of your API, and so you could, based on whatever tier the user has, you change the contents of the docs.
Other reasons to build would be if it’s a product differentiator or competitive advantage for you. So, if all of your competitors are using auto-generated docs, that’s an opportunity for you to differentiate yourself by taking it up a level. And if you have any kind of specialized requirements, then that’s obviously a reason. Integrations; workflow integration. So, if you have some tools that you need to integrate into your documentation workflow. Several of the off-the-shelf tools that I was talking about earlier actually do have a ton of integrations with, you know, all those CICD platforms and GitHub and all that stuff. So, it’s not like you don’t get integrations if you go with off-the-shelf, it’s just those integrations may not work the way you want them to, or they may not have the integration that you need; you know, this one weird tool that you’re using. So, one of the benefits of building it yourself is is that. And then, as I talked about earlier, search engine optimization; when you build it yourself, you have complete control over your search engine optimization.
So my final piece of advice, and this is literally my last slide, so get ready… is build versus buy. If your API is your product, i.e. that is the thing that you sell and it’s the only thing you sell, then it’s probably worth the extra time and effort to build custom-tailored docs that are specific to your API and help it stand apart and give you a really one of a kind developer experience. That does require a dedicated DX team, though, so be aware, it’s expensive.
“If your API is your product, then it’s probably worth the extra time and effort to build custom-tailored docs that are specific to your API and help it stand apart and give you a really one of a kind developer experience.”
Reasons to buy is basically everyone else. So, if your API is just a feature of your product, you know, you have some UI or other things, and the API is just some really awesome feature? You know, you probably don’t need to have some bespoke documentation solution. Similarly, like I said earlier, if it’s an internal-only API, you’re better off just using an auto-generator. And then if it’s an early stage product, you should probably be focusing on your product, not on the documentation yet.
View the video and slides from James’ presentation.
About Author
Leave a Reply