This is a transcript from Mike Kistler’s keynote presentation at
The Austin Homegrown API Meetup in August, 2019.
Hi, my name is Mike Kistler. I work for IBM in the Cloud Developer Services organization. This is a presentation, originally built by a colleague of mine, Dan Hadlo. I adopted it to be my own a little bit, but he deserves a lot of the credit for it. We’re going to be talking about API design guidelines.
I got into this because, in my day job, what I do is SDK generation. So it’s SDK generation from OpenAPI specs. And one of the first things that we realized was, you don’t get a very good SDK if you don’t have a good API definition to start with. So, we’re going to talk about API design guidelines.
“You don’t get a very good SDK if you don’t have a good API definition to start with.”
Now, some of you may be thinking, we don’t need that. And actually I hear this a lot actually from the developers that we work with. But it’s important, because API design guidelines are going to help create consistency in your APIs.
And that consistency is going to help your users, the users of your APIs. Because they’ll look at the API and they’ll say, “Oh, yeah. I know what this is like. This is just like this other API. And they all work the same.” And it’s also going to benefit your developers. Because as they’re building the service, they’re going to be able to reuse a lot of the code. They’re going to be able to reuse the same frameworks. So it’s really important to have this kind of consistency, and it’s not hard either. So I’m going to talk little bit about some of the things that you can do.
You have to start with a set of guiding principles. So what we started with was we said, okay, we want our API’s to be useful. Okay. So we’re going to use that as our guiding light for how we create our design guidelines. We of course want to adhere to HTTP syntax or semantics. And there’s some details of that, that’s a little hairy. So we’ll talk about that a little bit. We want all of our APIs to have a low barrier to entry. So, that’s really important as you think about designing your API. Defensiveness or compatibility is very important.
You want to be building APIs that people can use and not give them a lot of rope to hang themselves. And you want to be able to release new versions of your API and not trip up your users, because of incompatibilities that get introduced. Of course, you need security and you also need longevity. You want something that’s going to have staying power that you’re going to be able to evolve and use.
Normally the references come at the end. But it’s really important to know that there’s lots of companies that have already done this stuff. Okay. Microsoft has done this and they’ve published it on the web. Google has done this and they’ve published it on the web. And this one in particular, is really great. This is a set of design guidelines that were published, that go through all whole bunch of different API design guidelines from different companies. So you can go in there and you can see what all these different companies have done in terms of creating API guidelines. And what’s even better is that, on this page there’s this design topics. And if you go here, you can come down here and see, “Oh yeah. All this different stuff here. What about methods, delete. Oh yeah.”
So you can go here and you can find out who’s doing things about forbidden methods alternative. Well, Cisco is, Adidas is, right? You can go in here and you can explore all these things. Find out what other people have done and then you can build on that and learn from that. So very important. Now, what we try to do is, we try to do design-first APIs. So, we really think about what is the service supposed to be doing? What are the entities? What are the operations? Just try to map that out as a design before you try to commit anything to code. Then when you do try to commit something to code, use a good specification format.
“We really think about what is the service supposed to be doing? What are the entities? What are the operations? Just try to map that out as a design before you try to commit anything to code.”
And we use OpenAPI, and very specifically OpenAPI 3. Now, OpenAPI 2, also called Swagger, was around for a long time. Lots of people used it. Lots of people are still using it, even though OpenAPI 3 has been out for over two years now. Move up to OpenAPI 3. It’s important to get on the latest stuff. It’s important to have resource oriented APIs. And what I mean by resource oriented APIs is, your API should be operating on resources that are represented in your system. So you should have nouns that are these resources. And then you have verbs that are the operations. And what we do at IBM is we say, you look at the final segment, static segment in the API path. That’s the resource name. It’s actually the plural of the resource name. That’s the convention.
And then resources always have unique IDs, and they have a well-defined schema of their contents. So doing this in a regular repeatable way, will make your API much more understandable, much more accessible to your users. So here’s an example. This is an authors resource. This in particular is author number 18345 and it has a set of properties in it. There’s actually a Schema that defines what these properties are, what their types are, what their values can be, things like that.
The resource format, there’s been lots over the years. We’re hopefully past the XML days now. Most people are using JSON. But JSON is very flexible and that’s not always friendly for some languages, for some users. So you have to be careful about some things. So for example, JSON is unordered. So if you’re looking at an object and you see the order, like we just looked at this one, right? You would think, well the ID is an important property. It identifies this thing. But when it comes across the wire ID might be the last thing that comes through. So you just have to be sensitive to that. Also the JSON names are case sensitive. So be aware that if your language munches around with the names as it might, you might have collisions. And you should be using well-defined types.
So in an OpenAPI you can actually say, type is anything, format is anything. They don’t restrict it. But if you don’t use well-defined types, your users don’t know, “Well, what is that type really?” You need to be able to be clear. OpenAPI actually does list a set of well-defined types. And what we try to do is restrict ourselves to those, so it’s very clear when we say type integer format in 64, 64 bit integer. When we say type number, format double, then it’s a double.
JSON would let you mix types in array. So you can have a JSON array that has a string, and then a number, and then a float. That’s not very friendly to some languages. So we avoid that. And you can choose to do what you want. But this is something that I just want to make you aware of, can really trip up some applications, in some use cases. There’s also the value, node, in JSON, which is tricky. Because you can pass a property name with a node. Now what does that mean? Does it mean something different than if the property wasn’t there to begin with? Some people think it does. We try to say, “No, don’t do that.” If the property is not provided, then just don’t provide it. Don’t specify the property with node. And again, this is hard earned experience from doing some of these in various languages.
In the end, you want to create some clear guidelines on what kind of JSON that you’ll allow and what you won’t allow, so that your library developers and your service developers, can do the right thing.
Naming conventions. This is probably the place where fights break out. And I’m not going to tell you what the right naming convention is. But it’s important to have naming conventions for the reason that I raised earlier, is that these names are actually case-sensitive. So you could, if you didn’t follow a set naming convention, wind up with collisions of your property names or your parameter names that then your users would be able to resolve. What we do is we say we’re going to use snake case for parameter and property names. So every place that we have a parameter or property, it might be snake case. We use camelCase, and specifically lower camelCase, for the first characters lower case. For operation IDs, these get turned into method names in our SDKs, and that’s generally the convention for method names.
We use upper camelCase for schema names, and again, because these get turned into classes and that’s a very typical convention for class names. And we don’t use kebab case anywhere. Just that’s our convention. As far as operations go, if you’re doing a resource oriented API, you’re going to be doing something like “CRUD”—create, read, update and delete.
Typically, what you will see is something like create with POST, read with get, update with PATCH and delete with delete obviously. There are some other styles though. And in fact often what we do, is we do both create an update with POST. And I’ll tell you why in just a little bit. But the point here is, get a convention and establish that and then use that consistently and your users will thank you for it.
So this is regarding HTTP semantics. You want to make sure that you’re following the conventions and the semantics for HTTP verbs. So GET and HEAD should both be safe and item potent. So they shouldn’t be changing any resources on the server. They should only be returning things. They should also ignore request bodies. So you should not have a GET operation that accepts a request body. And in fact, in OpenAPI 3, I think this is explicitly disallowed.
POST is unsafe. It’s non item potent. And so what that means is that if you issue it twice in a row, it might wind up with different results. And GET was item potent because you could issue it over and over and it will get the same thing. You can use POST for creation of support and resources. So if you POST to slash books, you’ll get a book back, an instance of a book with a particular resource identifier. You might also use it for modifying an existing resource. And so you could do a POST to books with the ID, with a request body that had parameters in it or new properties in it. And you could modify it that way.
PATCH, is another way to update a resource. It has many of the same properties as POST. The thing with PATCH that it has very specific format for how it expects it’s request bodies to come, either as a JSON PATCH or as a JSON merge PATCH. And that can be difficult for some applications. And that’s one of the reasons why we use POST rather than PATCH. And then there’s PUT, which I didn’t mention it at all. It’s another HTTP verb. People do use it. It’s unsafe, it’s item potent. The thing about PUT is it completely replaces a resource. It’s not an update in the typical sense, right? You can’t just do pieces of it. PUT, the semantics of that are that it’s going to completely replace the resource. If that’s what you want, that’s great. But that’s not something that we find a whole lot of use for.
Okay. Another thing is having a standard error model. Now, a lot of your operations will have cases where they have errors that they need to return. Your users are not going to want to know 10 different error formats, error model formats for 10 different operations, or 100, or 10000, right? So you should have a standard error model that all of your operations return. And they can fill in the fields differently or however they need to, but the field should be named the same. They should have the same types. And so we have an error model where we have a code, we have a message, and then we have some target information and you can add other fields into this as well. But there’s a certain set of standard fields that are always in the error models.
“You should have a standard error model that all of your operations return.”
Furthermore, we take the convention that what you put into your error model, what you put into your error response is basically programmatic information, that is to say information for the developer. This isn’t necessarily a message that you’re going to turn around and display to the end user. This is something so that a programmer can look at this and try to figure out what went wrong with the API. And that gets you out of the hassle of worrying about localization and all those kinds of things. Collections are another place where it’s really important to standardize. How do you actually return a list of things? And the way that we’ve chosen to do it is we have a property at the top level of an object. So we never return a bear array. We always return an object with a property in it that has that array.
And the reason for that is because, well, you’ll see in a minute. We actually want to put stuff next to that property. But even if we didn’t have that there, this would allow for extensibility. So we have a property at the top level. We always call the property the same name as the resources that is returning. So here, if we get authors, that properties called authors, and then it has a list of authors within it. Pagination is something that’s important and you want to do this consistently so that your users are able to understand this. It’s a complicated thing. So you want to make sure they don’t have to learn five different ways of doing it. And there are a couple of ways that you see commonly. One is called offset and limit pagination. And this is where your resources are numbered. You say, I want to go to this point in my list, and then I want to have this many come out.
The problem with offset and limit is that it can be fragile. So there’s another flavor which is called token based pagination. With token based pagination, the server responds with a token that can then be passed on a subsequent request that will then continue the listing for particular amount. You can still provide limit, for example, with this. Now the difficulty with token based pagination is, there are some server state that might have to be maintained in order to do this well, or the token can encapsulate all that state. But then the token can get very large. So either one of those is difficult from the server side to really get right.
So, this is just the summary, limit pagination is states lists, but it’s in precise. And token based pagination can be robust, but it’s more difficult and demanding to implement.
Versioning. So you want to make sure you have a very clear versioning strategy for your API. You want to know what’s going to be compatible and what’s not. So usually things that are compatible are things like adding new fields to models, adding new types of resources and new URLs, new paths, new operations. But things like removing resources, removing fields from resources, adding new required fields or making a previously valid value invalid. All those things are incompatible and you need to decide how you’re going to diversion these things.
There are a couple of different strategies. One is you can pass that version. You can pass a version number in a custom header. So you can pass that in a header and then that header has to come through on all your API requests. You could also use a query parameter. So to pass the API version in and then you would pass that in on all your operations. You could put it in the content type. This is, I suppose, another version of the custom header or using a standard header with a custom value. Or you could embed it in the path. And in fact, you can even mix some of these, because you can use a path flavor of version as your major version. And then you can use a parameter type version as a minor version. And that’s actually what we do in many of our services.
That’s it. Very short. And there’s actually lots more that we could go into on this. But I wanted to just give you the brief overview of doing API design guidelines. Like I said, there’s lots of resources out there on the web to learn more about it. So be courageous. Go out there, define your guidance.
View the full video and slides from Mike’s presentation.
Want to learn more about APIs?
Join us for our monthly meetup!
About Author
Leave a Reply