I read everywhere that you need an ESB to truly implement SOA.
Having read many articles about what an ESB is, I am still unsure whether it is a product or a pattern.
If it is a product does SAP have one ?
Software vendors will tell you that ESB is a product that can be pulled off the shelf, installed, configured, and deployed as a common infrastructure in supporting all SOA initiatives. These products provide a consistent environment for interfaces, connectivity, orchestration, security, QoS, monitoring, scalability, availability etc. Without doubt the biggest selling point is the ease in which existing business processes can be webservice enabled through the use of wizards.
Some engineers and architects on the other hand will tell you relying on single product for integration creates a monolithic architecture investing heavily in legacy. Encapsulating integration into a single point of failure will eventually lead to irreparable hemorrhage. If you have implemented SOA correctly you will have de-centralized integration and the ESB will be a vendor agnostic distributed peer-to-peer communications backbone, in which case ESB is an infrastructure pattern.
Product or pattern even after reading the wiki definition there seem to be no clear distinction.
Contrary to the more classical enterprise application integration (EAI) approach of a monolithic stack in a hub and spoke architecture, the foundation of an enterprise service bus is built of base functions broken up into their constituent parts, with distributed deployment where needed, working in harmony as necessary – wikipedia
Recent trends in integration
The following illustrates trends in integration architecture – from point-to-point, hub and spoke and now ESB.
- Point-to-Point – data flows directly between systems
- Hub-and-Spoke – the hub is the central point and accepts requests from multiple applications that are connected as spokes. The hub and spoke simplifies integration by providing various adapters and the ability to map different data formats and protocols.
- ESB – applications publish and subscribe services to and from the bus through local adapter and integration engines, all service follow the same standards. Any new system can plug into the bus, as long as it meets the standards
In point-to-point integration, decisions are made bilaterally and processing carried out locally, integration starts of simple and gets complicated very quickly.
In hub-based integration, decisions are made independently, and processing is carried out centrally, this approach introduces complication by adding another place for development, proprietary development methods and another runtime component.
In ESB based integration, decisions are embodied in standards, and processing carried out locally like point-to-point. The connect everything to everything approach can be flawed if coding of services is done without business requirement or business value, it is difficult for a bus to handle complex, secure and enterprise SLA based applications.
Is SAP XI/PI an ESB? traditionally XI has been seen as a hub-and-spoke middleware.
ESB architecture is far ahead of this Hub and Spoke concept, its not only limited upto integrating various applications in distributed environment but also have provide an Event Driven Architecture. I hope, may be the revolutionary changes in XI/PI make it possible to consider it as an ESB. SDN – Is PI/XI an ESB?
The changes made with PI 7.1 are the introduction of the Enterprise Service Repositroy (ESR), adoption of webservices and standards like WS-RM, distribution of adapter engines and local point-to-point integration scenarios.
With XI/PI the functionality of the adapter engine can be deployed as separate service containers as needed, while working in harmony as necessary. Orchestration is now completely separated from transformation and can be deployed with or without PI/XI. However like many SAP products PI/XI is deployed all or nothing meaning you get functionality you will never need. The engineers and architects will argue that the framework bloat and component dependence approach of SAP and other vendor disqualifies them being true ESB by nature. Administrators and vendors might discuss otherwise and tell you that central integration is needed for complex mission critical scenarios. I guess that is why there is no consensus on what is ESB, as for Event Driven Architecture (EDA) that starts with a galaxy far far away.
rsol Weblog 
Loose coupling, scalability, reuse and minimal disturbance are the key ingredients to heterogenous integration. A lot of big companies are looking to SOA WS-* and ESB as a silver bullet to solve these problems, but in the ongoing buy vs build debate REST and dynamic languages seem to be winning.
I am a big fan of RoR, flex etc. they are fantastic frameworks and lead to the development of very innovative solutions. I have only limited experience working with them, developing POC’s and prototypes. What I find with these frameworks is that they are very domain specific, they do normally do one thing really well (say rapid web development or rich internet applications). If you want to add additional capabilities not available or limited in that framework you have to discover and adopt additional libraries and frameworks. From a support, maintain and extend point of view this adds a lot of complexity not found in a pre-packaged solution.
Likewise with REST I find it to be domain specific (HTTP), the ad-hoc approach is a quick win for integration and I can see its merits for solving one time developments and those long winded go nowhere discussions between silos. Abstraction is the key for me, from the little exposure I have had to RESTful services I find them simple, fast to deploy and powerful, yet very ambiguous (something I could probably get over), without a standard structure and the ability to generate a proxy a lot of code is needed for parsing, this is time consuming, inefficient and prone for error.
Regardless of whether it is REST or SOAP or whatever else, having a clearly defined, easy to find unambiguous interface is easier to support and maintain, using an SLA driven contract first approach (documented agreement not wsdl) promotes confidence between partners and also promotes re-use.
PS. I find Greg the architect humorous but his views are a little lopsided for me also.
When considering your question “…I am still unsure whether it is a product or a pattern” I would say that the ESB can be applied as a product or a pattern depending on the scope and granularity of the problem space you are addressing. For example – if I have 2 applications A, and B, and I want to create an ESB between these apps as well as any new apps I choose to add in the future, then the product-ESB model works fine. What you’d effectively have is a centralised messaging/switching broker and service endpoints representing the application adapters. I don’t believe there is much benefit of aspiring to an ESB in this context given there’s little more involved than a disciplined use of WSDL and MOM. However this model doesn not scale well when looking at an Enterprise SOA implementation (i.e. 000’s of applications with logical groupings, abstract SOA endpoints, and a highly distributed and heterogeneous technical landscape exists). In this situation the application of an ESB pattern to reign-in the complex problem-space can be highly effective, whilst not precluding localised ESB-product adoption for localised integration requirements. In summary – ESB is a pattern, ESB is a product, and an integration/SOA challenge can involve both depending on scope. I am speaking from hard-fought experience here.
REST is just another distraction from the ESB product/pattern debate. REST does offer an alternative mind-set in terms of how data and/or function can be exposed within an integraiton problem space. REST does not solve the issues associated with large-scale IT mobilisation within an Enterprise SOA transformation program, but can play into the implementation as a technical principle.