Why is SOA Taking So Long?

At a recent conference, ZapThink spoke with an enterprise architect who bemoaned the fact that his organization was ready to get started on their Service-Oriented Architecture (SOA) initiatives back in 2003, but now it’s two years later, and they have made surprisingly little headway. Compounding his frustration is the pressure he is getting from his colleagues who want updates on his progress. Unfortunately, this architect is not alone. Many organizations are finding it more difficult than they had expected to put SOA into practice.

At this point in the evolution of the market, organizations seem to understand what SOA is and why they should do it, but there’s still broad confusion about how to implement SOA in a way that guarantees business benefit and keeps the ball rolling. Some firms find that some combination of organizational, technological, or architectural issues bring their SOA initiatives to a standstill. Other companies lack sufficient buy-in from business executives to take SOA past the pilot stage. In fact, we have identified several specific roadblocks that are preventing companies from making more progress with their SOA initiatives. Here are the worst of today’s SOA roadblocks:

  • True interoperability is still largely out of reach. Today’s vision for SOA depends upon loosely coupled interactions among heterogeneous systems, and that vision depends upon standards-based interoperability. Cross-product, cross-platform interoperability is the raison d’etre of Web Services, after all. Nevertheless, in spite of all the work in standards bodies and the Web Services Interoperability Organization (WS-I), companies report that there are still small, but significant differences in the implementations of even the most basic of Web Services standards — SOAP and WSDL. And if vendors can’t get their SOAP and WSDL to interoperate, what hope is there for the plethora of follow-on standards that are in the works? To get past this roadblock, companies must push their vendor suppliers to be compliant with at least the most basic of Service standards. If the vendors won’t participate, replace them — there are more suppliers out there willing to get your business than you might think. 
  • Few people know how to write a contract or a policy. ZapThink recently asked a few dozen enterprise architects if they had any idea what went into a Service contract, the critical metadata document that abstracts the interface between Service providers and consumers. A few had implemented basic contracts with WSDL, but none had yet ventured beyond WSDL’s limited scope to additional metadata specifying service levels, security, or other policy information. In fact, it seems that very few people know how to turn human language-based policies into metadata that software can understand. The developing standard WS-Policy only provides for policy containers, not the policies themselves, but hardly any company has adopted WS-Policy yet anyway.
  • Companies lack architects with proper skills. The discipline of software architecture is still broadly misunderstood across enterprises. Compounding this lack of understanding is the fact that there are many different types of architectural specialties, and most architects are only proficient with one or two. To be a Service-oriented architect, an individual needs a solid grasp of enterprise architecture, as well as a strong familiarity with technical architecture, information architecture, business process architecture, and data architecture. People with all of these skills are simply as rare as four-leaf clovers. Organizations need to realize that in the future, the architect will be more valuable than the developer.
  • Standards are incomplete, too limited, or poorly conceived. Several standards bodies and their vendor-laden membership have put together elaborate roadmaps of proposed standards, covering integration, management, security, and more. To be sure, they’re making good progress on assembling the standards on these roadmaps, but there’s still a long way to go before we have a coherent, complete set of interoperability standards for SOA. More troubling, however, is the fact that some standards are too limited, and even worse, poorly conceived. WSDL, for example, is a standard that suffers from being too limited, as it fails to specify service-level requirements or contractual limitations on the Service consumer. Among the poorly conceived standards are UDDI and WS-BPEL. UDDI suffers from several years of changing priorities, and WS-BPEL combines two ill-fitting precursor specifications and still lacks human workflow capabilities.
  • Corporate politics. For many organizations, the relationship between lines of business and IT is strained at best, and antagonistic at worst. All too often, business management sees IT as a high-risk money sink that places limitations on the ability of the company to execute on its strategic goals. In all fairness, many IT people are willing to admit that the history of IT in their organizations is littered with cost overruns, failed projects, and plenty of activities that don’t help the bottom line. Nevertheless, this deep-seated lack of trust between business and IT places any SOA initiative into a political morass, as IT leaders must make the case for a powerful, yet hard to explain architectural approach that is still emerging and full of hidden risks. Companies implementing SOA must seek to reduce these tensions by showing early, consistent business benefit from their implementations. A 22-month SOA project might be 18 months too long. Be short, iterative, and high-value and you won’t have business management blocking your further progress.
  • Product selection is either too fragmented or too integration-centric. No software or hardware product will give an organization an SOA (or any other architecture, for that matter), but any SOA initiative will require new products to handle the new needs of loosely coupled, composite, metadata-intensive Service-orientation. While a number of companies have entered the fray with specialized SOA products, the problem is that you need to assemble several different products from a range of vendors to get the necessary capabilities, and those vendors are still struggling with interoperability, as we pointed out above. Coming to the rescue are the large vendors, who each have a shot at providing a soup-to-nuts SOA offering. Yet these vendors find themselves updating their older integration middleware products, squeezing them into their new role as SOA infrastructure, regardless of how suitable they are for SOA. Over time, the market for SOA solutions will increasingly consolidate. Those companies that have planned their architecture first, and product selection second will be much better off and able to move forward with SOA more smoothly.
  • Skepticism, selfishness, and stubbornness. It is human nature to resist change. While it clearly makes sense to be realistic about the limitations and problems with SOA, there will always be people in any organization whose skepticism goes beyond reason to emotional issues of platform choice, vendor bias, or personal preference. Furthermore, too many people resist new approaches, not because they truly see anything wrong with the new approach, but simply because they’re comfortable with the skills they have, and they don’t want to have to learn anything new. Other people see their scope of influence as their own personal fiefdom, and when someone outside their group starts talking about shared Services or federated policies, they raise their own personal drawbridge and send the archers to the walls. Companies that wish to have a prayer of making consistent progress with their SOA projects should identify not only the champions who will further the cause of Service-orientation, but also those roadblockers that only wish to stop the progress of something they see as threatening.

The ZapThink Take
Like any good coach, when something is preventing team members from reaching their full potential, it’s our job to recognize the problem, point it out, and help them solve the problem so that they can get back on track. If the SOA team is dropping the ball, the coach is going to yell — but not because the coach thinks the team can’t perform, or that the game cannot be won. On the contrary, the coach yells at the team precisely because tough love is just what the team needs at the difficult times that all teams go through. Therefore, while the roadblocks listed above are problematic, you can overcome each and every one of them. The secret is to take things one step at a time. ZapThink espouses taking an iterative approach to SOA as a key best practice, not only because requirements are constantly in flux, but also because there are so many pitfalls like the ones above on the road to successful enterprise SOA implementations.

Architects not up to speed? Train them. Standards not mature enough? Work around them. Too many selfish, stubborn people in your organization? Convince or reassign them. Vendors giving you the runaround? Pressure or replace them. None of the roadblocks above are preventing you from taking the first steps toward putting together a high-level SOA plan, or assembling a few loosely coupled Services. Roadblocks may slow you down, but there’s no question that for the vast majority of organizations considering SOA today, the fundamental business benefits of SOA outweigh the disadvantages. Be patient, be persistent, and be ready to learn new things, and you will eventually be successful with your SOA initiatives.