Understanding the Real Costs of Integration

These days, it seems that everyone is pitching the merits of using Web Services to better integrate systems and businesses. “Drastically reduce integration costs, increase time-to-market, and improve business flexibility using Web Services!” claim many Web Services integration vendors. How much of these promises are hype, and how much is reality? In order to understand what impact Web Services will have on integration, it is important to understand its true costs, which means comparing different integration approaches and finding out whether Web Services can offer a positive return on investment. With this knowledge businesses can decide whether Web Services will truly save them money, or conversely, simply add to the chaotic proliferation of spaghetti code in the enterprise.

The Four Phases of Integration

The costs of a typical integration project go through four distinct phases: the initial setup costs, the cost of configuring and customizing the integration project, ongoing maintenance costs, and costs involved when any of the elements of the integration project change. ZapThink compares four approaches to integration: custom integration, traditional EAI and B2Bi approaches, Web Services “adapters” for integration, and loosely coupled Service-Oriented Integration, as shown in the chart below:

First, let’s take a look at the solid black curve, which represents the cost of custom integration. Custom integration involves dedicating current IT resources to the task of achieving some application goal that requires integrated results from multiple systems. Such custom integration involves the smallest up-front cost of the four integration approaches because the skills and tools necessary to complete the integration task are typically already in house. However, as the project progresses, it consumes an increasing amount of developer time, in proportion to the complexity of the integration task at hand. During the maintenance phase, then, things start to get costly. Traditionally, companies spend most of their time and money maintaining existing integration projects and then dealing with the changes that occur in those systems as business requirements and technology change. With custom integration, the costs for both maintaining and changing systems can become exorbitant since developers must recode all applications that are impacted by changes.

The second approach to integration lumps together traditional EAI (Enterprise Application Integration) and B2Bi (business-to-business integration) approaches, as shown by the dashed green line in the graph above. Traditional EAI and B2Bi solutions have sought to solve many integration headaches by presenting an architecture that efficiently manages and maintains connections among systems and between enterprises. The primary downside to EAI is that up-front costs are much higher than custom integration, as shown in the first column of the graph. In a typical EAI solution, end-users must spend from tens of thousands to millions of dollars on software licenses and server systems prior to completing any integration. The actual integration project itself costs many times more than the initial costs and can easily dwarf the costs of custom integration projects.

However, EAI and B2Bi achieve their the real win during the maintenance phase of an integration project. As long as the endpoints, business processes, or for that matter any major business assumptions, don’t change, then the costs of running an EAI solution over the long haul can be substantially lower than that of custom integration projects. However, the hidden cost of EAI is when things do change. When systems, business processes, or major assumptions change, EAI system costs can spike, as shown in the fourth column of the graph. In fact, we say that EAI systems “pour concrete on business processes,” since they tend to solidify existing processes rather than enable an IT environment that allows companies to deal easily with change.

Enter Web Services. By enabling standards-based approaches to integration, many people believe that Web Services will turn the EAI/B2Bi cost curve on its head. However, ZapThink believes that blindly applying Web Services to integration is a mistake. Just because Web Services promise dramatically reduced integration costs, that doesn’t mean that any arbitrary use of Web Services will achieve this effect. After all, just because a new technology has promise doesn’t guarantee that it will be applied correctly.

In particular, we are talking about vendors that provide SOAP interfaces to various software components with Web Services “wrappers.” The critical issue is the difference between using Web Services in a loosely-coupled, Service-oriented manner, as opposed to misusing Web Services in a tightly-coupled, point-to-point manner. Vendors that pitch their wares as “SOAP wrappers” or “Web Services adapters” offer a dramatically short-sighted view of integration that seeks to lower only the initial costs without solving any of the fundamental problems of the traditional EAI/B2Bi approaches. In fact, the blue dashed line above shows that only initial and configuration costs are saved while long term costs of change mirror those of traditional EAI/B2Bi costs.

While it is true that the use of standards-based technology lowers initial and perhaps even configuration costs, these tightly-coupled Web Services adapters are just as brittle as the EAI adapters of old. Instead of having to reconfigure 300 DCOM or CORBA calls when a system changes, integrators will have to reprogram 300 SOAP calls. All they have done is play a shell game, moving the pea from proprietary to standards-based adapters. This approach doesn’t do squat to loosely couple the systems or change their level of granularity. In fact, all that is accomplished is that developers now pour standards-based concrete over existing business processes — wrapping EJBs, COM, or CORBA in SOAP is simply not going to get us anywhere. In fact, the Web Services wrapper approach may actually increase the long-term cost of integration, because of the inefficiencies inherent in XML when compared to binary message formats, as the blue dashed line on the graph shows.

In stark contrast to the “Web Services wrapper” straw man is the loosely coupled Service-Oriented Architecture (SOA) approach to integration, which we call Service-Oriented Integration (SOI). While tightly coupled SOAs have been around for a while — both CORBA and DCOM have elements of service-oriented architectures — Web Services technologies allow architects to build standards-based, loosely coupled SOAs that expose business functionality at varying levels of granularity. The real costs in building and integrating such SOAs is in the system re-architecting. In an SOI solution, it is not sufficient to simply wrap an existing API with a Web Services interface. It is too easy (and almost lazy) to create a one-to-one mapping between system APIs and Web Services interfaces. Rather, businesses must spend time analyzing their business processes and creating business Services at varying levels of granularity, perhaps even requiring the orchestration and choreography of multiple layers of Web Services to accomplish a single task. This support for multiple levels of granularity enables the SOA to support frequent changes in the underlying systems, as well as changes to business processes and underlying business assumptions, without the need to make interface changes that break the loose coupling of the Services. The real win with SOI, therefore, is in the dramatic reduction of cost at the maintenance and change phases of integration, as shown by the dotted red line in the graph.

The ZapThink Take

ZapThink encourages both vendors and users of Web Services integration technology to take a close look at the mistakes of the past. What are the true costs of integration, and how do Web Services seek to reduce those costs? Are implementations
of Web Services simply “old wine in new bottles,” with interfaces every bit as brittle and tightly-coupled as in the past, or are they really implementing Service-Oriented Integration among Services at many levels of granularity? Clearly, “Web Services Integration” does not equal “Service-Oriented Integration.” We are not doomed to repeat the mistakes of the past, as long as we understand the appropriate use of the Web Services technologies behind Service-Oriented Integration.