The API is Dead! Long Live the API!

A funny thing happened as I was noodling on this ZapFlash. I was all set to put the nail in the Application Programming Interface (API) coffin, continuing the discussion of just how awful Remote Procedure Call (RPC) interfaces are, and how we should avoid them at all costs. But then I ran across the “explosion of APIs” meme at ProgrammableWeb and elsewhere. After all, your head must be planted deeply in the sand not to notice the plethora of APIs at all our favorite Social Web sites like Twitter, Facebook, and hundreds of others. How can the API be dead when APIs are more important and plentiful than ever?

Maybe I’m just nitpicking. As APIs become less RPC-centric and more RESTful, the inflexible tight coupling of the past is giving way to a new golden age of API-ness, where anybody can connect any piece of software seamlessly and automatically to any Web site or app we might care to use. But upon closer reflection, my concern is not all nitpickiness. There are API best practices and API worst practices, just as there are good APIs and bad ones. Perhaps more to the point, most of the APIs out there, well, are among the bad ones. So, what makes for a good API today, with all the power and capabilities that modern technology approaches afford us? Why are so many people getting them wrong? And what can we do to steer everyone in the right direction?

Programmable Interfaces: The Never-ending Story

The real reason that APIs get under my skin is the “P” – “programmable,” as in programmatic. The core notion of an API, after all, is that you want one application to be programmable by something external to it, in the sense that you have an imperative control flow that goes from here to there and back to here.

The problem with such imperative programming in any distributed environment, of course, is that there is no end of problems that can arise between here and there. Network issues, incompatibilities, timing and sequence issues, the list goes on and on, leading computer scientists to the notion of functional programming. In the functional world, functions (which also go by the names procedures or methods) encapsulate the software at the remote location, forming black boxes with fixed inputs and outputs. Any local piece of software, therefore, can call a remote procedure and get the desired response, without having to worry about whether multiple calls to that procedure might interfere with each other or lead to other nasty surprises that a purely imperative approach might exhibit.

Here’s the rub: functional programming is nothing more than RPC, with all the tight coupling issues that come along for the ride. However, functional programming is also considered to be a type of declarative programming. Declarative programming has always been the ugly duckling of the programming language world, because with a declarative approach, you don’t specify the control flow. Instead, you describe the desired behavior you want the software to exhibit. And somehow, via some kind of behind-the-scenes hand waving, the software miraculously does what you want it to.

Functional programming is declarative in the sense that a remote procedure acts as a black box. Specify the operations, inputs, and outputs (its signature, in API-speak), and the inner workings of the procedure aren’t your problem. That is, unless you don’t understand what it’s supposed to do for you, or heaven forbid, its behavior changes.

REST to the Rescue?

Building abstracted interfaces that enable organizations to deal better with change is what SOA is all about, of course. Unfortunately, the tools we had at our disposal – namely, Web Services – didn’t go far enough. Yes, we moved from RPC-style Web Services to document-style, and that improved our loose coupling substantially. But even document-style Web Services still have operations, a holdout from the bad old functional programming days.

Enter REST. ZapThink has discussed the move away from RPC to RESTful interfaces before: in REST-Based SOA: an Iconoclastic Approach, but of course, we’ve been discussing the advantages of document style interfaces over RPC for several years now. But people still aren’t getting it. Not only are techies missing the point of REST, even when they build supposedly RESTful APIs, they’re failing to follow the REST constraints.

For example, many of these ostensible RESTful APIs don’t provide self-descriptive messages. With REST, every message from a resource should contain all the metadata necessary for any client to understand what it can do with representations of that resource. In the RPC days, procedural responses may have contained only data with no metadata. With Web Service interactions, Service responses contained some metadata in the form of the SOAP envelope, header, and the tagging structure in the body. But even with Web Services, the WSDL contract and policy metadata are external to the Service. You don’t expect to get the contract when you query a Service.

With a RESTful interaction, on the other hand, contract metadata may be returned in particular representations, whether it be media type metadata, schemas, or even less structured metadata. And of course, representations also include hyperlinks, which also provide contract metadata to the client.

Separation of resources from representations is another essential REST constraint – one that developers also frequently violate. If there are any instructions to the resource in a request other than one of the four operations of the uniform interface (GET, POST, PUT, and DELETE), then you are violating this constraint. For example, if your request is instructing a resource to update a record, you’re not being RESTful. It’s up to the resource to determine whether a POST or PUT should update a record, not the representation.

Finally, hypermedia as the engine of application state – the dreaded HATEOAS – is perhaps the most overlooked of the REST constraints, even though it’s the most important. The big picture of HATEOAS are the hypermedia applications that we’re trying to build, but even at the API level, HATEOAS is critical, in conjunction with the self-descriptive message constraint. In any truly RESTful interaction, the resource returns a representation that contains all necessary metadata, including hyperlinks that instruct the client what it can do next. Likewise, a RESTful client has no idea what a resource can do for it unless it follows such links.

“RESTful” Equals “Web-friendly”

What so many techies lose sight of is the fact that REST isn’t supposed to make the Web more like system integration; it’s meant to make system integration more like the Web. When you click a link in a Web page, you expect to go to another Web page, or perhaps a video or sound file or some other media representation. REST takes that simple interaction and abstracts it. Now you have an abstracted client (instead of a browser) supporting a request of a resource (the Web server capability, for example, the php script that generated the response) on an abstracted server (the Web server) that returns a representation (the resulting media file or Web page) as per the uniform interface (what the link you clicked was supposed to do). These abstractions let us apply simple Web behavior to all manner of system-to-system interactions. But never, ever lose sight of the fact that you want simple Web behavior from your application.

I’m sure many readers who made it this far are muttering to themselves that their RESTful APIs are truly RESTful. Well, maybe, but probably not, and here’s proof. Let’s take a popular, supposedly RESTful API as an example: the Yahoo! Maps PlaceFinder Geocoding API. You’ll notice on the documentation page a “BASE URI,” which it expresses as http://where.yahooapis.com/geocode?[parameters]. There are two problems with this expression. First, it’s not a hyperlink (it just looks like one). Second, the URL has the question mark in it, requiring the user to know what parameters might be valid.

Let’s say instead we turn this expression into a true hyperlink, leaving off the parameters since we don’t know what they are: http://where.yahooapis.com/geocode. Go ahead,  click on that link, I dare ya. Sure enough, you get an error message. True, it’s XML formatted, but it’s missing something absolutely essential: a hyperlink.

In fact, Yahoo! made several serious mistakes: first, the base URI isn’t a hyperlink, violating HATEOAS. Second, the metadata that tell you how to use the interface are separate from the interface, rather than  returned in the response from hyperlinking to the base URI, violating the self-descriptive messages constraint. Third, the URI requires parameters as part of the search query (the characters after the question mark), violating the separation of resource and representation. And finally, the error response it did return didn’t include a hyperlink, once again violating HATEOAS. Bottom line: there’s really very little that’s RESTful about this API.

While it may be amusing to pick on Yahoo!, it’s important to point out that they aren’t alone. In fact, they’re typical. It’s quite rare, in fact, for a supposed RESTful interface to be truly Web-friendly. After all, most techies don’t expect Web friendliness from APIs. APIs are, well, programming interfaces, while the Web consists of user interfaces. But in the REST world, programming interfaces are simply abstracted user interfaces. In other words, truly RESTful APIs aren’t really like the APIs we know and love, and have been working with for years. They’re fundamentally different.

The ZapThink Take

I know this ZapFlash is deeply iconoclastic, but that’s what you’ve come to expect from ZapThink, after all. However, the point of this article isn’t to scold everybody for doing REST wrong. The point is to help you think differently about what it means to program in a distributed environment. The point to the “programmable Web” isn’t to make the Web more programmable, it’s to make software more Web-like. If we can finally free ourselves from the last vestiges of imperative, RPC-style programming, even going so far as to steer clear of functional programming, and move to a fully declarative, document-centric paradigm, only then will we be able to achieve the resilience and power of the Web when we tackle system integration challenges.

This mind shift is at the core of REST. When Roy Fielding wrote his dissertation, he didn’t come up with REST and then build the Web following its constraints. On the contrary: he deeply understood what made the Web itself so special, and he sought to express that specialness as an architectural style. True, tackling system integration following true RESTful principles is difficult. But remember, building the Web was easy. Take TCP/IP, add a few simple standard protocols, code an HTML rendering app we called a browser, and the Web more or less took it from there. Almost twenty years later, the Web is unimaginably immense, and yet it still works just fine, thank you very much. Is it too much to ask for all of our IT systems to behave the same way?

Image source: cloudzilla