Solving the Very Large Messaging Problem in the Enterprise
Companies are increasingly seeking to tie together their disparate enterprise using the promising, but emerging technologies of XML, Web Services, and Service-Oriented Architectures. These approaches promise significant business agility in the face of IT heterogeneity. However, these benefits come at a price: performance and efficiency. As the network traffic increases due to the increasing size and volume of messages, both XML and non-XML based, existing corporate IT infrastructure will be taxed to its limit. General-purpose application servers, network equipment, and messaging infrastructure will be increasingly devoted to simple message parsing, handling, and routing functions, while precious few resources will be left to execute the core business logic so important to companies.
Research shows that the quantity and size of these metadata-laden messages won’t be decreasing soon. Developers and specifications bodies continue to tax messaging systems with additional layers of headers and metadata meant to abstract underlying infrastructure. Increasingly large message size, along with a general increase in message volume, combine to create the challenge of Very Large Messaging (VLM).
Previous approaches to solving distributed messaging problems, including messaging middleware, ESB, and application servers, were not designed to handle the challenges of VLM. Emerging approaches such as hardware appliances and binary XML, may solve part of the overall VLM problem, but fail to provide a comprehensive approach that targets all systems, networks, and processing infrastructure that runs within the corporate IT environment. Further, while these approaches may remove some of the overhead of message parsing, they offer no direct benefit to assimilating and utilitizing these messages within applications.
As a result, new approaches are needed to deal with messages being exchanged on the network that are exceeding the capabilities of the general purpose hardware and software that is now being applied to the problem. How can efficient content-level message processing be distributed to all the nodes in a corporate network? How can dumb networks be made more intelligent through the ability to process data and metadata it formerly ignored? In this paper, approaches to the “Very Large Messaging” (VLM) problem, and potential optimal solutions that hope to break the stalemate in network processing of data are presented.