ZapThink’s 2005 SOA Scorecard and 2006 Predictions
Whether it’s holiday fruitcake or Grandma’s sweaters, we all have our traditions when it comes to celebrating the holidays and ringing in the New Year. ZapThink’s traditional end-of-year ZapFlash focuses on revisiting our predictions and analyses over the past year and pontificating about the future of Service-Oriented Architecture (SOA) and Web Services. Our predictions are not always on the money, of course, but we’re not worried, since we bet our company on SOA back in 2001 when we first started talking about leveraging the nascent Web Services movement for architectural purposes.
It seems just yesterday that we spent our time introducing SOA and differentiating it from Web Services and other technology trends. As recently as 2003, we cautioned technology vendors to be judicious about their discussions of SOA, since at that time, only certain architect audiences had heard of the term. Now, of course, every IT department is all abuzz over SOA, and every software vendor has slapped SOA stickers on their products, regardless of how relevant the products are to helping companies implement the architecture. Yes, ZapThink predicted the rise of SOA, so any minor imprecision in our annual auguries tends to pale by comparison. On that note, let’s take a look at how well we did with last year’s predictions, and boldly make some new ones for 2006.
ZapThink’s 2005 SOA Scorecard
Our first prediction last year was that the Web Services Interoperability Organization (WS-I) would have to get going or get out. We cautioned that the WS-I would have to publish profiles that could solve a range of interoperability problems or risk fading from relevance. Unfortunately, the latter has taken place — the WS-I has made surprisingly little progress in 2005, and has become increasingly irrelevant in our conversations with architects who are looking for true interoperability. Even though there’s still interest and support for the WS-I’s mission, and there’s hope for their possible return to relevance at some point in the future, for now they have faded from view, and have become little more than a footnote in the general discussion about the advancement of SOA and Web Services.
2005’s second prediction was that companies would make more money on SOA professional services in 2005 than on SOA products. We predicted a dramatic growth in SOA consulting, but the reality is that growth has been more gradual than we expected, due to two key factors. First, SOA has been taking longer to become established in the enterprise (see our ZapFlash Why is SOA Taking So Long? for a discussion), and second, seasoned Service-oriented architects willing and able to be consultants are in short supply (a trend we covered in another earlier ZapFlash). So, even though the demand for SOA consulting has grown significantly, and the desire for professional services firms to offer such expertise has likewise increased, market forces have moderated the growth in this area.
Our third prediction centered on the burgeoning, yet mystifying Enterprise Service Bus (ESB) market. We pointed out that there was no clear definition for ESB, and suggested that the term would either have to resolve into a single definition or disappear from the SOA lexicon. It’s true that the definition of ESB has solidified to some extent, but unfortunately, there is still far too much confusion in the marketplace about what an ESB is, and why you would want one.
We place the blame for this continuing confusion squarely on the heads of many (but not all) of the vendors hawking ESB products. While some vendors with ESB products have scalable, truly Service-oriented product architectures that properly abstract the underlying transport, other companies pitching so-called ESBs have done little more than take an older product (such as hub-and-spoke EAI products, traditional message buses, message brokers, or application servers), and simply added Web Service interfaces to it in order to brand it an ESB, for no other reason than customers were asking them for an ESB and they needed to sell their customers something. Now, we’re not going to name names, but you know who you are! This shell game of selling the same product with a different name to meet new, evolving needs doesn’t fool anybody for long. SOA success requires new approaches, not simply the limited ones of the past foisted once again on a confused market.
Checking how we did, it looks like we were generally right on the predictions, but wrong on timing. In particular, it looks like some of the issues around organization, skills development, and architectural practices are delaying SOA adoption, so we should expect to see continued progress on our 2005 predictions through the coming year. Nevertheless, that won’t stop us from plowing right ahead with our new predictions for 2006!
Prediction #1: The Rise of “Next Generation” SOA Projects
In the past year, it was hard to find a single company that hadn’t at least considered SOA. Indeed, most of the firms we spent our time with had already made some progress with at least one SOA pilot project. Unfortunately, many of these projects haven’t taken advantage of the SOA best practices that are only now becoming clear. For example, we’ve seen SOA-like implementations based on serialized Java objects, Services with metadata stored in a proprietary format or even a format meant only for human consumption (such as text documents or spreadsheets), and Services scattered around an organization with no proper governance in place to promote reuse. Indeed, many companies starting down the path to SOA have yet to realize the goals of lower cost of integration, increased reuse, and reduction of overall IT redundancy.
Our prediction is that many such organizations will chalk up their current projects to “lessons learned,” and put together the next major version of their SOA that takes advantage of the wealth of best practices that are now becoming clearer, including the proper use of a registry/repository, effective Service lifecycle management, and a comprehensive IT governance framework. These next generation SOA projects will return certain courageous enterprises to technology leadership positions, while it seemed that they had stalled with their earlier attempts at implementing the architecture. The good news is that by clearing away some of the poor implementations of the past, architects can build SOA implementations that better meet the needs of their organizations moving forward.
Prediction #2: The Rise of the Service Consumer
In the past five years, ever since SOA shifted from obscurity to curiosity and now ubiquity, the focus of most vendors, end-users, and yes, even analysts, was on the production side of Services. We focused on how to build and expose Services using a wide range of runtime infrastructure, but we paid little attention to the software that would consume those Services. Well, the tables will finally turn in 2006. With the emergence of technologies such as Asynchronous JavaScript and XML (AJAX) for building rich Service consumers, companies will now be able to focus on the plethora of opportunities that Service consumption can foster. In 2006, we will start to see evidence that the conversation will shift from a one-sided discussion of who builds and provides the Service, to a dialogue between the parties responsible for Service provision, and those who care just as much about how to consume those Services.
Prediction #3: Shakeout and Maturation in the SOA Governance Space
Possibly the most dramatic trend in the SOA world in 2005 was the rise of SOA governance solutions, products, and other offerings. Companies have long struggled with corporate governance in general, and IT governance in particular for several years now, driven by the new regulatory environment and the compliance issues it generated. The movement to SOA both introduces new approaches to IT governance, as well as requiring a new set of technologies, practices, and infrastructure that itself requires governance. This two-sided opportunity has led to the rise of a variety of SOA governance solutions, often from vendors and consulting firms with very different strengths and approaches.
Complicating the situation is the fact that SOA governance doesn’t stand alone as a market niche, but rather overlaps other markets, including the registry/repository, metadata management, business activity monitoring, and Service lifecycle management market segments. Such a convergence of capability is the recipe for significant consolidation resulting in a shakeout of the market that will lead certain vendors to merge or get acquired, while others might find themselves out of business entirely. This sort of rapid consolidation has happened many times in the SOA marketplace, including the native XML data storage space in 2003, the Web Services management space in 2004, and most recently, the XML acceleration appliance market in 2005. We predict that 2006 will be the year the SOA governance space takes its turn.
Prediction #4: The rise of SOA Compliance Solutions
While governance is a broad concept that concerns all the policies a company might want to create, communicate, and implement, regulatory compliance dictates specific policies and procedures that firms must follow, or risk penalties so severe that executives might actually go to prison. If that weren’t bad enough, enterprises are finding that there are so many regulations they must comply with that simply figuring out how the various regulations fit together is a Herculean task in itself. Companies are finally realizing that the only way to get a handle on this tangled morass of regulation is to consider all specific initiatives the business must complete to comply with every applicable regulation.
The only hope that such companies have for actually implementing such a holistic compliance solution that will (a) actually work, (b) not bankrupt the firm, and (c) allow the company to continue to meet customer needs is to leverage SOA to build a flexible, governable approach to IT. Our prediction, therefore, is that there will be a dramatic increase in the number and capability of SOA-based compliance solutions coming to market in 2006. Some of these solutions may be in the form of composite applications, while others may simply be sets of Services. Either way, SOA will become increasingly critical to many enterprise’s compliance initiatives.
The ZapThink Take
SOA has finally appeared on almost every company’s slideware. Whether you are a vendor selling software, a consulting firm selling professional services, or an end-user plotting your overall IT direction, SOA is your best chance for long term success. Up to this point in time, people relegated SOA to a corner of an existing problem area, such as integration or application runtime infrastructure. Companies are now finally realizing that SOA is not a technology, but rather a key part of a vision for how they can organize the relationship between business and IT to allow them to become more agile and responsive to business change.
The trend toward SOA-based compliance solutions, in fact, is actually part of a broader trend toward SOA solutions of many flavors. Now that the SOA market is maturing, the center of the action is moving away from the nuts and bolts and toward the business. SOA makes it possible to sell Services or composite applications that implement agile business processes specific to a particular industry or business problem. As a result, many industry-specific vendors are now leveraging SOA to take their solution offerings to the next level. Compliance solutions promise to be the most explosive of these offerings because of the urgency that companies currently have, but other solution-based SOA offerings won’t be far behind.