Finding the Real Barrier to SOA Adoption
There have been countless many articles focused on establishing a business case for Service-Oriented Architecture (SOA), developing a prospective Return on Investment (ROI), and proving to the business why SOA will provide short-term benefits, address pressing problems, and provide a long-lived answer to the persistent problems of IT’s ability to meet the changing needs of the business. However, with all this business goodness, you would think that SOA investment would go unimpeded. After all, if SOA can prove its value to the folks who hold the pocketbooks, then why aren’t we seeing unfettered SOA spending reminiscent of the dot.com-boom days when the promise of IT was as great as what SOA promises now?
Part of the reason for SOA’s continued measured uptake is that SOA is all about architecture, and doing architecture well is really hard. Despite how some vendors may portray it, you can’t just buy a product and expect it to miraculously create the Services you need and the agile architecture and organization to support them. At best, you can get the tools and infrastructure needed to create those Services. On the contrary, experience has shown that successful SOA adoption requires companies to spend more time changing the way they do IT more so than just the technology they use to implement their short-term needs.
What ZapThink is finding is that the primary barriers to SOA adoption do not come from business management, which by and large realize the benefits of an agile, reusable, and loosely coupled architecture (even if they don’t call it that), but rather from within the IT organization that resists the movement to SOA for a wide range of reasons — many of which have little relevance to the needs of the business. Even when a business has approved the investment of significant sums in their SOA projects, ZapThink has found that in many cases, their own IT organization can and will sabotage those efforts, slowing the SOA drive to a crawl.
IT — The Primary Barrier to SOA Adoption?
It might not seem obvious that an organization’s own IT department might be the single biggest obstacle in the movement towards SOA adoption. After all, the origin of many SOA concepts arises from the experience of IT practitioners and technologists. Yet, too many people in the IT organization conceive SOA as a technology concept only, and as such think of SOA as just a set of technologies and infrastructure for exposing, securing, running, and managing Services. With those technology blinders on, these IT practitioners see SOA as nothing more than Web Services and standardized middleware. Of course, they are wrong. The critical flaw in thinking is confusing the technology that sits beneath the Services level of abstraction and the mechanism by which Services are accessed with the architectural approach that aims to decouple the implementation from the consumption and focus on sustainable architecture that allows for continuous change — an approach that is completely technology agnostic.
Successful SOA adoption requires a cultural shift in the way IT is done. The Service-oriented movement to agility and loose coupling demands a shift from traditional, waterfall styles of development (design-build-test-deploy-manage) to iterative approaches to continuous Service modeling. The drive towards loose coupling demands a heterogeneous approach to computing that reduces dependence on single-vendor platforms and stacks. The move away from point-to-point integration to compositional, process-driven applications that consume Services from a broad array of assets across the enterprise requires development and management approaches based on Service domains rather than system-specific silos.
In a Service-oriented environment, governance moves from a nice-to-have afterthought to an absolute requirement prior to the consumption of the first Service. Security considerations shift from network boundaries to become context-specific, message-driven, and federated. And of course, the modus operandi of IT as a whole moves from focusing on the short-term project management to meeting the long-term sustainable needs of the business as it changes. All of these big movements require significant change and thus strike fear in the hearts of IT managers who find it easier to adopt one technology fad after another. It is precisely because SOA requires a fundamental change to the way IT is done that many see it as a threat.
Thriving on Complexity and Fear of the Unknown
One of the easiest ways to explain IT resistance to SOA is that it represents a number of unknown factors, and things that are unknown are the cause of fear. Few people know how to properly budget for SOA, or how to properly govern SOA initiatives, or how to deal with perceived performance and security issues. Because SOA raises as many new problems as it solves, many shun it altogether. Some reason, “why bother with this new SOA thing when at least you understand how the old tightly-coupled integration thing works?” Of course, if companies solely managed by fear, we’d probably still be riding in horse-driven carriages. Innovation requires change, and change does not come without uncertainty.
Some in IT resist SOA because it seems to make obsolete their current skills. If someone is a mainframe expert, say, and SOA allows non-experts to build new applications that leverage all the functionality that previously could only be accessed using system-specific knowledge, then it makes sense that they would oppose the SOA movement. There’s a certain job security that comes from understanding complicated things. SOA serves to simplify things in certain ways, especially those aspects of IT that are tightly-coupled and inflexible. It is natural to expect some to want to protect their jobs, but this is a solely selfish reason to oppose SOA. If the best interests of the business demand innovation that obviates the need for system-specific knowledge of things complex, it is hard not to see that opposition as selfish.
Further evidence of this “what’s too simple must not be too good” line of oppositional thinking can be seen in criticism of the common analogy of comparing SOA with LEGO ®. The appeal of the LEGO metaphor is that implies that complicated systems can be abstracted as simple blocks that business organizations can compose with little concern for the underlying complexity. It is for this primary reason that some individuals within the IT organization and industry pundits criticize this comparison. To them, the LEGO metaphor represents a gross oversimplification of the actual complexities of IT.
The reasoning goes that it is a dangerous line of thought to suggest to the business that they should think that composing IT systems is as simple as child’s play with blocks. However, this line of thinking thrives on maintaining the complexity status quo. The reason why the LEGO metaphor is so appealing is that business desperately craves a gross oversimplification of the challenges of IT. Most business folks are already painfully aware of the complexity and inflexibility of IT, trying to convince them of that is pointless. There’s no need to tell business that they should only think of IT in complicated terms. Indeed, what is needed is a gross oversimplification because there’s no reason for the overly complex state of today’s IT. Of course, an easy read into why some in IT resist this simplification is because complexity gives many in IT their jobs — not just the employees of the organization, but also vendors, consultants, and even analysts. Those that thrive in environments of complexity have the most to lose when the movement is towards simplification.
Architects? We Don’t Need No Stinkin’ Architects
Too many in IT also believe that architecture is not needed as a central and separate role from development or project management. These individuals feel that architects are theorists that pontificate from their Ivory Towers and make unfeasible recommendations without having to consider short-term time and budget limitations. For these folks in IT, the pervasive belief is that there’s time to do things over, but not do them right. After all, the business has never before invested in proper architecture and design, so why should they now?
This short-sighted defeatism is most evident when you hear SOA opponents decry that any attempt to establish architecture as a project-independent activity is doomed for failure because the business simply won’t budget for anything that requires investment beyond the needs of a single project. Of course, this attitude spells instant defeat for any attempt at reuse or agility that is the primary effort of SOA. Companies looking to successfully adopt SOA and realize its primary benefits need to find the defeatists and convince them the business will continue to invest in architecture and its long term benefits, even if cheaper, but less agile, alternatives exist.
Organizational Resistance
Many companies have sabotaged their own SOA efforts because the IT organization is simply not set up for the SOA effort to succeed. In particular, many companies don’t have the culture of sharing that is required to realize the SOA benefit of reuse. These organizations not only represent their IT systems as silos, but also segregate their management teams in system-focused silos as well. There is no incentive to share Services in an organization that separates budget and responsibility for individual projects in silos. Trying to build a shared Service that cuts across the domains of multiple systems as well as organizational hierarchy is potentially doomed for failure when issues of budget and control can’t be rectified. Such organizations not only need to adopt SOA from a technology and methodology perspective, but should also Service-orient their organizational structure.
The organizations also show their dysfunction by segregating the architecture role and management into groups that have no control, authority, or budget to mandate or enforce architectural recommendations or decisions. Architecture’s sole function in the organization is to be the strategic management of business requirements to IT capabilities, and so architecture teams must have supervision, control, and responsibility for the outcome of the SOA efforts of the whole organization. Yet, too many firms leave project managers in charge of their own “architecture” leading to the tangled rat’s nest evident in most firms today.
Indeed, many firms aren’t actually doing architecture, but rather they are doing archeology, digging through their IT environment to discover long lost civilizations of technology developed by their organization and then promptly buried, only to be discovered later when something fails catastrophically. A functional IT organization empowers architecture groups with budget and authority. As further evidence of that, most CIOs are not performing the role they should be — as strategic managers of the architecture of the organization. CIOs should think like chief architects, but many are either glorified project or IT operations managers. The valuable CIO is one that sees and plans the strategic value of IT, leveraging SOA and enterprise architecture as the central mission of the IT organization as it provides continued benefit to the business as an asset, rather simply fighting fires and responding tactically, inflexibly, and imprecisely to the needs of the business, thus treating IT as a cost center.
ZapThink Take
The industry has focused primarily on convincing the business to adopt SOA, and to a certain extent, has been successful in addressing the business case and ROI of SOA. Yet, convincing the business is not enough to guarantee the lasting success of SOA. Companies need to address the latent resistance, hostility, resentment, and fear in the IT organization that will effectively prevent SOA adoption and success.
In our book Service Orient or Be Doomed, Jason Bloomberg and I wrote about the “Return of the Luddites”:
For virtually every SOA project, organizational, human issues are far more difficult to resolve than technical ones. People are only just so flexible, after all, and groups of people tend to be less flexible than many of the individuals within the group. There are many reasons for this human inflexibility–fear of the unknown, resistance to change, limited attention spans, and our core personal motivations to follow enlightened self-interest and avoid discomfort and risk.
SOA champions, therefore, must also take on the role of change agent, requiring them to work with people on an individual basis to address the issues that make people reluctant to change. An organization that plans to fully embrace Service Orientation will have to fully embrace the organizational and cultural changes that are required.
Couldn’t have said it better.