SOA: Integration vs. Business Process?

At the Integration Consortium’s recent Global Integration Summit, I sat next to a fellow at lunch who had been in the integration business his entire career. He began his career hand-coding integrations to IBM mainframes, and over the next twenty years, had connected one system to another using sockets, adapters, and several generations of middleware. It’s no wonder, then, that when the topic of conversation shifted to Service-Oriented Architecture (SOA), he looked at the concept through integration-colored glasses. From his perspective, SOA was about leveraging Web Services as standards-based adapters, and loose coupling amounted to little more than requiring various endpoints to support the same set of open standards.

In fact, ever since ZapThink coined the term “Service Oriented Integration” (SOI) back in 2002, there’s been unceasing confusion on just how Service-Oriented Architecture (SOA) and integration relate to one another. Several recent blog posts have refocused attention on this confusion, including ones from Anne Manes, Todd Biske and Loraine Lawson. Perhaps some of the confusion is over the definition of the SOI term, and what distinguishes SOI from Web Services-based integration, if anything. But the bigger controversy is over the larger question as to SOA’s fundamental purpose: is SOA’s purpose to solve integration problems, or is it more of a business transformation approach centered on implementing agile business processes?

Is “Service Oriented Integration” really Service Oriented?
ZapThink has been harping on the distinction between SOI and Web Services-based integration for more than six years now, so the fact that many pundits are conflating these two concepts is dismaying. The point we’ve been making over the years is that SOI requires SOA, in other words, SOI is an architecturally-driven approach to solving integration issues that can yield business agility, in particular, a flattened cost of dealing with business change over time. Web Services alone can reduce integration costs somewhat, but don’t provide the agility or flattened cost of change that SOA promises.

In some circles SOI means middleware-based integration that leverages Web Services, in other words, ESB-based integration. The problem with this definition is that much of the ESB-based integration today is in organizations that haven’t implemented SOA. ZapThink may be fighting an uphill battle here, pointing out that ESB-based integration without SOA should hardly be called Service Oriented. Be that as it may, arguments over definitions of terms is usually a waste of time, as the more useful arguments are over how best to solve problems.

In other cases, however, pushing for a redefinition of a term is an insidious way of influencing public opinion. SOA is too difficult, some vendors say, so buy my ESB instead. Or take Microsoft’s “Real World SOA,” which is their terminology for Web Services-based integration. Presumably you don’t need architecture in the real world! In other cases, redefining terms is little more than noise. For example, Yefim Natis from Gartner (yes, the Yefim Natis of “SOA 2.0” fame) recently claimed that “SOA is integration.” While there’s broad agreement that many SOA initiatives focus on solving integration-related challenges, no one who’s put any thought into this issue would actually say SOA and integration are the same thing.

Rethinking Integration in the SOA Context
All of these arguments over SOI, however, really miss the key point here, which is how SOA can help organizations with their integration challenges. Since SOA is architecture, it consists of best practices for organizing IT resources to better meet the changing needs of business. In other words, SOA requires us to rethink how we approach the problem of integration altogether, in the context of an architecture oriented toward Services rather than toward tightly-coupled interfaces. Instead of taking a “connecting things” approach to distributed computing, therefore, we need to take a “composing Services” perspective.

The difference between these two approaches goes to the heart of SOA. In the traditional, integration-centric world, the enterprise IT environment consists of a heterogeneous mix of resources that don’t automatically talk to one another. As a result, IT must devote substantial resources to connecting those resources to each other in order to provide value to the business. In other words, integration is something the techies do before the business can get the value they want out of the complex IT environment. SOA, on the other hand, focuses on building and using Services. Deployed Services, in essence, are ready to be integrated; if the organization implements the Business Service abstraction properly, then it’s possible to compose the Services to implement business processes.

Furthermore, while specific process requirements often drive the initial design of the Services, reusability of those Services is also a design priority for some of those Services. Remember, however, that Service reuse means composing Services into new Service-Oriented Business Applications that implement other business processes, often in ways that the original designer of the Services did not predict. Furthermore, business process requirements drive such composition. When the business composes Services to meet such requirements, they are actually performing the act of integration. Only now, integration is a byproduct of composition that takes place after the Services have been deployed, rather than something IT does to connect systems in order to deliver business value in the first place.

Integration vs. Business Process: Not a Battle at All
Once you understand that the core technical challenge of SOA is in building the Business Service abstraction which allows the business to compose such Services to support changing process needs, then the question of whether SOA is for integration or business process becomes moot. Instead, the focus of integration becomes Service composition. IT focuses on building and supporting Services, rather than connecting things. True SOI — that is, integration that is truly oriented toward Services — is a byproduct of Service composition.

One important caveat: part of the “building Services” challenge is in integrating existing legacy assets. After all, SOA is much more about leveraging legacy systems than it is about building green-field applications. As a result, legacy application integration and data integration are essential parts of the SOA implementation story. What’s changed here is the role such integrations play in the architecture. In SOA, they play a supporting role, part of the hard work that goes into building Services properly. In contrast, such integrations play a leading role in ESB-based integration. If you think that connecting things to your ESB gives you SOA, then you’ve fallen for the SOA straw man that gave rise to the “SOA is dead” meme. From ZapThink’s perspective, such integration isn’t Service-Oriented at all.

The ZapThink Take
Thinking of integration as a byproduct of Service composition supports not only the business agility driver for SOA, but also the business empowerment benefit that SOA promises. Because true SOI is business process-driven, it enables lines of business (LOBs) to take greater responsibility for such integration. If the IT organization has abstracted all of its capabilities as composable Business Services, then the LOBs can drive the integration of those Services via their Business Process Management efforts. And while LOB users will still likely call upon IT personnel to create the compositions themselves, they will be doing so at the behest of business process specialists who are focusing on solving process-centric business problems.

Of course, this process-centric view of Services raises the bar for the IT organization, since building the Business Service abstraction is a far more difficult challenge than simply implementing Web Services, or even implementing an ESB. If there’s one thing all the SOA pundits can agree upon, it’s that SOA is hard. Nevertheless, if you’re able to get the architecture right, then you’ll be able to shift the focus of integration from the tightly coupled, technology-centric world of connecting systems to the loosely coupled, business-centric world of composing Services.