Step 2: Federated Data Access. Consistent and reliable data access that spans multiple data sources/systems has been a pressing challenge that many enterprises struggle with. When solved correctly, it allows for single-source-of-truth of data which can be used for broader information. For example, if you wanted all the information about your current customers today, how many systems would you have to access? In this model we need to ensure that the data is federated how it is accessed so that it can be re-used as a single service. That service can be passed into the ESB to provide greater re-use across the organization. Because the data sources stem from multiple project locations it is extremely important to ensure that the data is consistent, accurate, and up-to-date.

AquaLogic Data Services Platform supports Federated data access has optimized connectivity for AquaLogic Service Bus and is the backbone of Information as a Service (IaaS) architecture.

Step 3: Enterprise Wide Business Processes. Remember that your business doesn’t run in a silo, even if your IT is managed that way. Business processes often go across lines of business and may break down the barriers between organizational projects. If we’re only thinking of BPM as human-centric processes, we don’t have any challenges here for federation; unfortunately most business processes in organizations actually do touch automated system-centric processes. These integrations between the Business-centric and the IT-centric collide. For example, billing processes need to often access a host of CRM and financial systems as well as an approval workflow process. To solve this both BPM and ESB need to be integrated together to provide service-enabled business processes to span this federated model. These integrations need to support reliability guarantees and be integrated into the design-time components of the two solutions.

AquaLogic Service Bus and AquaLogic BPM today support tight integrations (both design-time and run-time) for better managing broad federation of business processes that leverage highly federated services underneath.

Step 4: Management and Security of Composite Services. The complexity of federation is no different than the complexity of composite services across an organization. We need to be able to consistently manage how services behave as they are messaged between ESBs, Data Services, BPM, and other service applications. We need to be managing the Service Level Agreements (SLAs) and be able to enforce their explicit behavior as a policy. For example, when data is extracted from a system, it might need to be scrubbed for privacy considerations – let’s say we want to hide someone’s full social security number when a call center is verifying the last four digits. These types of fine grained entitlement policies can help to better secure services and provide governance on the run-time behavior.

AquaLogic SOA Management provides management of services beyond simply an ESB, but across projects and across domains. Together with AquaLogic Enterprise Security can provide fine-grained entitlements for greater control of data and information.

Step 5: SOA Governance for better control and visibility of Federated deployments. The original objective of Federation was so that you can get better control and visibility of the deployments. Because services don’t always go through a single ESB or even a single big bus ESB, there is an even greater level of abstraction that is required. In this case the Service Component Architecture (SCA) can help. By leveraging a Repository/Registry that supports SCA, you can ensure that composite deployments can be visualized, managed and deployed across multiple projects and domains. For example, one team develops inventory services, another team develops billing services; consequently a third team can leverage the two to create new services around order management. Because these services are located in a single repository/registry, scorecards about the performance characteristics, dependencies on who is calling them, can lead to faster-time-to market of bringing these new services into production and ensuring that they continue to be leveraged effectively.

AquaLogic Registry Repository helps better control federated integration implementations of ESB, Data Services, BPM and provide out-of-the-box capability for SCA for better managing composite services.

What are some of the steps in Federation you are still struggling with? Feel free to send over your comments!

References

• How to look at ESB Federations – Irene Rusman
• United Federation of ESB – Richard Manning

Hotter than a Hedge Fund: SOA Integration

Are you looking for places to duck and hide in what looks to be an inevitable recession?  

Can’t figure out if you need to invest in ETFs, stocks, mutual funds, bonds, or dare I say it Gold? Are you simply putting your cash under your mattress?

Ever think to invest in SOA?

Recently on an interview on SearchSOA.com, Randy Heffner was asked if he thought the SOA curve would continue its upswing responding that the potential down-turn might actually spur further SOA adoption.

"There are conditions under budget stress that actually encourage the use of SOA," he said. "For example, one benefit of SOA is that it extends the life of legacy applications. Say we were going to rewrite this application and spend $X millions, but we figured out we didn't have to because with a fourth of the money we could get where we needed to by SOA-enabling a legacy application."

Perhaps SOA Integration can be your oasis that you are looking for. Getting those pesky legacy apps sorted out without spending so much to rewrite them, and that’s just the beginning!

Make sure you don’t miss these types of opportunities by looking on Arch2Arch on more about SOA, SOA Integration and SOA Governance. 

See also Joe McKendrick’s reaction: http://www.soainaction.com/blog/2008/03/survey_soa_isnt_just_surviving.php

Solve your Agileocrity Part II: The SOA Integration Anti-patterns

I presented recently at the Arch2Arch Summit in San Diego a presentation on SOA & BPM: Better Agility through Business Integration. During the talk we discussed in detail some key things to avoid in your SOA Integration Architecture. These anti-patterns come especially from Architects like you who are trying to improve their architectures and build them purpose built with agility in mind.

Anti-Pattern #1: Service-Enabled Spaghetti

We’ve seen spaghetti diagrams before. I admit it – I’ve generated at least a dozen of them. However this is a slightly different takeaway. Simply putting a web service interface to all your systems doesn’t get you the benefits of an SOA. Why is that? Because the returns of SOA are only realized when you can properly sort through the spaghetti, when you can control it, when you know how people are re-using those services, and when they truly insulate from change, and not just add to confusion. Without an organized, highly controlled, architecture, you’ll cause yourself more headache than if you had simply left your systems alone.

Recommendation: Build a SOA Reference Architecture to properly classify the intent and purpose of services. Leverage SOA Governance to be the watch-dog on what you build. Make sure you build it with an intent and purpose to be flexible.

Anti-Pattern #2: Accidental Integration Architecture

This particular pattern was referenced in the SOA Integration Blueprint Article.

Historically, integration started as a pattern to build better transparency between consumers and producers of information across multiple channels. You can think of EAI as one of the earliest forms of integration focused around application integration, ERP modernization, and B2B (a specific type of multi-enterprise integration). As SOA was recently introduced, integration evolved to meet other, more specific needs such as service mediation for web services. On the journey to integration, enterprises also recognize that data services as well as real-time events needed to be treated uniquely within other integration scenarios.

All these integration patterns are "correct," but, unfortunately, many companies struggled to adopt all of them or simply favored one but not another. The Accidental Integration Architecture emerged simply as a means to solve multiple types of integration between consumers and producers.

Recommendation: Look at Integration as components within a broader solution framework. Eliminate redundancies in connectivity, security, management, and QoS by considering a single unified solution which meets multiple integration entry points.

Anti-Pattern #3 The SOA Silo

In this example, each project does the “correct” implementation, yet in a silo. ESBs are implemented properly, reference architectures (within the project) are consulted, and solution frameworks consider multiple entry points. However it is still wrong. Why?

There are 4 problems with this approach: 1) There is no unified management, control and visibility across the project Silos. 2) The re-use across the ESBs hasn’t been considered. It might require a federation architecture for services that are needed to span the organization. 3) There might be security, privacy issues for data that spans multiple projects. 4) Duplicates of data models where data isn’t single-sourced.

Recommendation: Look at Governance to better manage interactions across the silos where a federated registry can deal with the complexities of multiple run-time domains. SOA Management can help SLAs that span across the business. Look at ESB federation if interactions needs to span across data-centers, domains, projects and business units. Look at identity and security federation requirements. Look at Data Services or Information as a Service approaches to manage data federation.

In summary, these patterns are most likely not new, they are iterations on previous anti-patterns we have seen before in Integration. Leveraging SOA Integration together with SOA Governance and the right types of reference architecture we can start to see some of the benefits getting more agile systems. 

Appliances or Software-based ESBs? The Million Dollar Question

Take a look at this article on SOA World Magazine by Shiva Bhajekar: Appliances or Software Based ESBs: The Million Dollar Question

Talks about some of the most important questions that Enterprise Architects have been wrestling with:

• When should you look at appliances or software ESBs?
• What are the differences?
• What are their strengths and weaknesses?
• Do I need both?
• And if I implement both is there a recommended architecture for that?

“You may start with the software ESB when service brokering and integration is the primary driving factor for a project, especially when services use is internal within an organization. For a smaller deployment in an organizational unit, it is possible to stick with just a software ESB, with the more traditional network firewalls providing the missing gatekeeping functions.

Start with the hardware appliance when perimeter security is a concern, the service consumers are in the extranet or a different organization, and the services have already been enabled for reuse. For small organizations where integration needs are very simple, it might suffice for service brokering as well. Ultimately the lack of the appliance's flexibility and an inability to support complex integration patterns will result in experiencing the need for a software ESB.”

Below is an except of the Reference Architecture to help you if you are leveraging both. This relates back to our SOA Integration blueprint we talked about in an article we published last week.

Questions or comments on the article? We’d love to hear your feedback!

SOA Integration: Solve your Agileocrity

In today’s dynamic business climate, inflexible infrastructure can result in lost customers, lost revenue, late entry into emerging markets…oh wait…you’ve heard this before!

Why haven’t you fixed it already? You are so not agile.

Your Excuses: Your boss says for you to do more with less; getting over SOA troughs of disallusionments; new buzz words like EaaS (Everything as a Service); anything with the word ‘federated’ in it; virtualization; events; there is a recession (maybe); and your taxes are due.

Don’t panic. Breathe. Breathe in SOA Integation.

SOA Integration is about getting more flexible. It clears the path for your business by transforming brittle IT systems, applications, and data sources into highly flexible, reusable services that can be shared across the enterprise. Service enable your business logic why don’t you! 

Now that you are breathing easier, we believe that SOA Integration is about the unification of these integration elements into these 4 important categories.

• Service Integration – An ESB provides enterprise-wide IT-agility and flexibility.
• Data Services – Better information. Better decisions your company can make by better connecting multiple system views as a service.
• Process and App-Integration – All apps are not the same, you’ll need some heavy-lifting.
• SOA Connectivity – the last mile is the most important mile of your SOA.

Leveraged together with BPM, the combined approach enables better Business Integration and revolutionizes your IT’s ability to meet what the business wants. 

Controlled by a SOA Governance Framework, the combined approach gets you closer to enforce what you actually intended to integrate.

Rise above the mediocrity: get yourself acquainted with SOA Integration using some of these great resources below!

• Taming the Challenges of Business Integration – Mike Gilpin from Forrester talks about real-world SOA and trends in Integration
• Business Integration with SOA, a Revolution in Business Agility – BPM and SOA are better together!
• Blueprint for Successful SOA Integration – All you need to start thinking about key requirements for SOA Integration.
• BEA SOA Reference Architecture – Let someone else do the hard work for you.

AquaLogic Service Bus 3.0: Setting the New Benchmark for Enteprise-class ESB Performance

AquaLogic Service Bus once again proves that it is the enterprise-class leader in ESB technology with its refined foundational capabilities in 3.0. These include:

• Performance improvements
• Optimized parallelism
• Partial parsing of headers
• Large message streaming

Together these new techniques provide AquaLogic Service Bus customers ways of lowering their ESB hardware requirements, at the same time future-proofing their implementations for the large enterprise-wide deployments.

Performance Improvements

In our standard routing of HTTP/SOAP messages benchmark, AquaLogic Service Bus easily beats the 500 million messages a day benchmark. The benchmark is run for 2-CPU dual core (2 GHz) Intel Xeon server running Red Hat Linux; average message sizes are 5K. For even smaller messages, we see over 700 million messages a day or roughly 9,000 messages a second.

This is approximately 12% faster than AquaLogic Service Bus 2.6 with some additional noteworthy improvements below which give an additional bump for cases that involve parallelism, partial parsing, or large message processing.

Incidentally this is also beats the numbers from this performance study on WS02 and Service Mix with a significant advantage.

Parallelism

• Benchmark involves AquaLogic Service Bus invoking 3 Business Services in parallel (each with 20ms latency).
• The baseline is calling the 3 Services sequentially in AquaLogic Service Bus

Using the new Parallel action, we see a 62% reduction in response time. This is ideal for cases where processing of messages requires steps with latency and optimizations of parallelism can be implemented.

Partial Parsing Optimizations for Content Based Routing

Improved content based routing is enabled through partial parsing of the SOAP header. This is enabled by using StAX extract the SOAP header. This is an advanced technique of parsing large XML documents which doesn’t require a large memory footprint. Using this technique shaves over 3 times the performance for processing larger XML messages. SOAP header based routing is just one place where we gain from using partial parsing.

We also gain from partial parsing in the following scenarios:

• SOAP Header based routing (irrespective of whether we enable streaming) or any SOAP header processing (e.g. adding a new header)
• Routing based on any part of the payload (not just SOAP header) when streaming is enabled. This is a big plus for content based routing of larger payloads. This is assuming the payload is not transformed as that requires the entire payload to be parsed. 

 Streaming Mode

This new feature in AquaLogic Service Bus allows messages to be kept in a serialized format instead of a parsed XmlBean format for optimized large file handling. With this new feature, paging support is included which allows for the serialized message buffer to be persisted either in-memory or to the disk. The message is buffered in order to allow retries. Streaming allows for significantly larger messages (>500MB) to be transformed easily and ultimately, this helps the ESB scale by reducing the memory pressure. Reducing the memory pressure means less Garbage Collection and that may lead to performance improvements.

Feel free to take your own test drive of the ESB, check out: AquaLogic Service Bus 3.0.

References

Also look at other tests of other open source ESBs here:

• WSO2 – http://wso2.org/library/1721 – round1
• WSO2 – http://wso2.org/library/2259 - round 2

Look out Godzilla here comes AquaLogic, "SOA Field" hits the Theaters on YouTube

In this clever spoof on Clover Field, watch as innocent bystanders flee in horror from competitive threats while a brave enterprise Architect (played by Annie) attempts to shield herself using what looks like… BEA AquaLogic?!

Save yourselves not just from what’s looming outside that window, but you too can be a hero to your enterprise. It just takes a few clicks!

Special thanks to Ian Wikramanayake who directed the film and Bob Rhubart our avid screenwriter.

Ed the Enterprise Architect

I’ve written many blogs about myself – my favorite subject – but never do I write about you! How selfish of me.

Who are you again?

Well perhaps I’ve lost touch with you over the years. You’re Edward. You’re an Enterprise Architect. That’s with a capital A along with a capital E for Enterprise.

And not just any architect will do. You have a diversity of IT & development experience under your belt, but at the same time, you’re always thinking how to make your business more efficient. You’re the champion that seeks to raise the bar in the organization in both IT as well as setting the right alignment between IT & the various line of businesses in your organization. You manage and choreograph enterprise-wide architectures; you make infrastructure recommendations that will either make or break your company. Let’s just say that you’re the one person in the company that knows how to use PPT, excel, and still can write code (but choses not to).

You’re probably one of the few people who goes home at night thinking about big problems no one else can grasp; they’re technical challenges but at the same time they’re rooted in business problems as well. In fact, the technical challenges are probably the least of your headaches right now. It is a lonely world championing problems that few can relate to.

Lately you’re losing sleep over these types of problems:

• I get SOA, but how do I get folks in the organization to embrace it?
• How do I manage the chaos I inherited?
• Consolidate vs. End of Life vs. simply walk away.
• Security is something no one thinks about.
• Are my applications compliant to the policies I set? – Oh wait is there actually a policy framework?

Yes, all your multiple advanced degrees in Masters Computer Science, MIS, MBAs can’t help you. You turn to your Magic Eight ball and no luck. You’ve got few people you can even turn to on these challenges. I’ve got some unsolicited advice for you Ed and here goes:

Architecture comes at you fast, remember to take a moment and smell the infrastructure you got; leverage a small piece of what you have and turn it into something re-usable. Do that one small thing and those around you will thank you for it.

Feel free to give me a shout, or turn to other Architects here in Arch2Arch, even if Ed is not your name, I’d love to hear from you!

JBI is still dead

About a year an a half ago I wrote a highly infamous blog stating that JBI is dead. At the time of my blog I had some suspicions of the limitations of the standard. As it turns out, my suspicions are still warranted.

From polling our customers, partners and folks out there like you, we’ve heard that JBI 1.0 is not ready for prime time. The JBI 1.0 standard focuses primarily on a few vendors and doesn’t deliver on the promise of ESB-portability or ease of creating SOA. As a result, it contributes little to the usefulness to business integration, one of the real pain points for our customers. There have been a few articles that disagree with me although the disagreement is not with me on the 1.0 standard, but rather on the future of what JBI potentially can deliver in 2.0. A great quote from Mark Little, “JBI isn’t dead but pining for the Fjords”. Well possibly. But let’s take an objective look at it.

What is actually included in JBI 2.0? Below is an excerpt from the JCP’s plan with my annotations in red.

Enhancements in JBI 2.0 will include:

• Enhancements to facilitate the use of JBI in clustered or distributed environments, principally with respect to administration rather than the clustering/distribution mechanism itself

Many of the open ESBs which are based on non-J2EE container architectures and only JBI ones, are finding themselves unable to meet scalability requirements. J2EE offers clustering/distribution options, so it is not likely we’d see rapid adoption of new standards in this area.

• Enhancements to clarify and enhance the use of JBI in a SOA-based approach to the creation, deployment and runtime support of Composite Applications.

Composite applications are where SCA has a sweet spot; the ability to design, deploy SCA is where everyone is heading. Do we need another level of abstraction for deployment objects?

• Enhancements to support requirements stemming from WS-Policy.

Great idea; wrong standard for it. Policy in general has become one of the most hot topics in SOA Governance. It requires adoption by not just the ESBs but by the repositories, management vendors, and SOA Governance solutions. A JBI standard-only for policy would be quite limiting for SOA Governance.

• Enhancement to support Web 2.0 technologies and usage models.

Well that’s a different standard isn’t it? It is called AJAX and REST? Do we need another level of abstraction for the binding here?

• Introduction of a Message Exchange handler/interceptor model.

Makes perfect sense, the ESB vendors are noticing that the JBI standard isn’t ideally situated for ESB approach.

• Enhancements to facilitate performance optimizations by component and container implementers.

Ok I am not sure how you are going to make a standard here to make competitors beat each other in performance; what would be more useful is a standard way to measure performance in general among ESBs.

• Improved alignment with Java EE (e.g. use of transactions).

That would be simply adopt the J2EE standard.

• Recoverability of Message Exchanges.

Recoverability is again something handled quite well in today’s J2EE containers. We don’t need to reinvent this wheel.

• Improved readability of the specification to clarify the needs of container, component and application developers.

Keep it simple. Totally agree. JBI 1.0 had millions of marketecture interpretations – but few cases that met the promises of vendor neutrality and openness.

• Alignment with the Service Component Architecture (SCA) specifications (see www.osoa.org) with the goal of making JBI 2.0 a standard Java runtime for SCA.

Alignment is another word for … borrow… or perhaps recognize that the standard is much better at managing complex composite applications across not just JAVA but across all.

• Enhancements to support full compatibility with OSGi, without necessarily requiring OSGi.

Makes sense, no comment here. Although again, sounds like another unnecessary level of abstraction. 

• Technical issues stemming from implementation experience using the JBI 1.0 specification (e.g. life-cycle of components, error handling, interop profiles, examination of the utility of WSDL definitions for non-Web Services deployed components, component attributes, threading, NIO use, classpath or endpoint activation)

Bug fixing. Yes Good idea.

The recurring theme here: JBI is pulling itself back to life on the other standards which have become more popular and widely adopted, especially SCA (Service Component Architecture). I think this is especially important. JBI yearns to be like SCA in the way it meets interoperability above and beyond simply Java-based bindings. Do you need both or just one? If you’d like to see the relationship between JBI and SCA you should definitely read this great blog by Mike Edwards. You should definitely make your own determination if you think this standard has life left in it. Will the ubiquity of SCA, BPEL, J2EE hang make JBI the forgotten standard of the past? Feel free to send me your comments; even if you think JBI still has some air left in it. I’d love to hear from you.

BEA’s AquaLogic Service Bus 3.0 will take advantage of the SCA standard along with BEA AquaLogic; today it supports interoperability to BPEL and leverages one of the most widely adopted J2EE platforms on the market.

Dell going Green against Big Blue

At Oracle Open World last week Michael Dell out trumped Ellison in his keynote with a visionary and quite re-volutionary message about what a company ought to be doing with its capital. It is clear Dell is rising about their typical competition and going after a much larger message.

Yes Dell has found such a higher message by going green and done some great marketing around what they have implemented in terms of environment friendly IT. They point at that by 2011 70% of companies will experience power/brown outs and yes you better your virtualization act together quickly.

Hey I think BEA has a pretty good story around Virtualization!

The key Dell message to get you there: Simplify IT in all caps it looks like SIMPLIFY IT, again something we have heard before at BEA. No surprise there. In fact we have been saying that the only way to simplify IT is through SOA & BPM, which aligns the IT with the business objectives, their processes and their needs; not just for IT’s sake.

But hey, this is Michael Dell talking, not me…so I digress.

Dell also discussed the advantages of Software as a Service (SaaS). For a good explanation go read Nick’s blog on SaaS. They showed a few demos that talked about what they could do there, although it wasn’t that interesting of a demo, they simply built a “Image” hosting environment for storing your images. It took me 10 minutes to figure out that they were talking about VMWare images and not Picture images. Agree with the SaaS message. Yes! But next time pick a better demo to illustrate the point.

Dell concluded the keynote by claiming to be Carbon Neutral by 2008 (although I am not sure what that means given all the Hummers sitting in their Texas parking lot). But joke aside the important part is that they are trying.
 
Finally, Dell left the audience with a striking movie clip with upbeat and energizing trance like music with images from Age of Aquarius, saying that the time for the me too generation needs to move over for the Re-generation: Re-build, Re-cycle, Re-engineer.

It was catchy, and I have to admit it was memorable. Count me in for the Me too Generation. Oops I mean count me too for the Re-generation. But before you do that make sure you go read about BEA Virtualization and my blog on how ESBs are in fact the ultimate in Green.

ESB vs. Hummer H2

 
Hi. I see that you drive a Hummer. Aren’t you concerned about the environment? 
I can see quite high above the environment.   

Do you ever try taking a Bus?
No I like this car. It has 8 cupholders and a place to put my tree-cutting chainsaw.

No I mean a Bus. Enterprise Service Bus?
Oh that. Well no.  
 
Why not. Everyone else has one?
Well I am happy travelling point to point. It gives me more freedom.
 
Oh I see! You mean you don’t have a central Repository/Registry controlling your navigation?
Umm no. I don’t have a navigation system. My wife helps me navigate.
 
So you always know where you are going, and what route to take?
Actually yah. Usually. It is ok I am not looking for the fastest route.
 
Yes, but don’t you have RASP? Where are your ilities?
Ummm. I don’t know what you are talking about. Raspilities?
 
You know. Ilities. Reliability, Scalability, Security, Possibility
Those sound expensive. Does it come with the bus ticket?
 
Hey now you understand! So lets see your WSDL!
My Wiz What?
 
You know. Descriptions of your service consumers and service producers. I need those if I can get you to your destination
This is exactly why I drive.
 
Did you know that 1 of every messages not sent on a bus are lost or stolen?
That is why I have insurance
 
No insurance can protect you against what may happen in an unmanaged SOA.
Well I’ll just take my chances.
 
Do you ever consider going green?
My Hummer is already painted green
 
Well you should really put your services on a virtualized stack so that you can go green.
Oh… I already own VMWare. I just bough it yesterday at 90 bucks a share!
 
I don’t think you get it. You really should take a SOA Assessment immediately. Your short sighted thinking is really causing all of us harm as well as polluting the environment. And you really should try public transportation like an Enterprise Service Bus!
 
In the interest of full disclosure: Dain currently takes the local subway each day to get to work, and to save the environment, lowers his carbon footprint by driving a Mini Cooper. He also inserts himself quite regularly onto AquaLogic Service Bus.

Tastes great but less functionality - Light ESB

So why is it we can never get this definition right?

I’ve seen countless number of blogs (SearchWebServices) on the mythical light ESB. But I don’t agree with many of the statements on what a light ESB is; and hence many of the conclusions that are drawn.

• Is a Light ESB an ESB that is easy to install? No, that’s just nonsense. In which case an iPod is an ESB.
• Is a Light ESB an ESB that doesn’t run on an application server? No. In fact by embedding it’s own built-in application server functionality, it is in fact much heavier
• Does a heavy ESB includes governance functionality? No, I would say if there is no way to manage it, it is not really an ESB in the first place.
• Does light ESB mean open source? Not necessarily. Because many of the open sources are just as heavy weight in terms of memory and in terms of how they perform in a run-time environment.
• Does Light ESB have functionality removed? No, That’s how many use the term, the question is only what functionality would you take out? Routing, Transformation, IDE, Security, Monitoring, Scalability, Performance, Standards, Extensibility. They are all important. You can’t do Service Mediation
• Is a Light ESB an appliance? Well not all rhombuses are squares as the saying goes. This too is what I would say for the appliance. Just because you put your ESB in a square box (or is it a rhombus?!); doesn’t make it lightweight. Especially if you require multiple boxes + some software to do what a single ESB can do.

So how does an ESB get lighter?

• Lower memory footprint for the entire ESB, as well as overall SOA Integration solution.
• Support real-time systems with deterministic garbage collection
• Supporting time and event driven computing in ESB environments.
• Optimzed design time plugins for Eclipse which support better tooling integration for the entire SOA & BPM stack
• Optimizing it for secure, reliable transports with co-located integration capabilities as plugins: ERP Connectivity, Integration, BPM, Data Services.
• Build it with policy in mind for SOA Governance. And leverage a separate Registry/Repository component for visibility and control at all levels.
• Support all aspects WS* which are by nature light already.
• Don’t force Java on everything that the Bus does
• The same solution should be interchangeable for both small and large scale environments to scale out as your business grows.

Today the market needs a fully functional bus, but one that can grow and evolve as enteprise needs for SOA evolve and mature, but that starting point is an ESB. Call it what you want, light or heavy you decide. I like to call it AquaLogic Service Bus.

The New Seven Wonders of ESB

I am here in beautiful Barcelona, for BEA World’s annual worldwide conference. Take a look at some of the cool blogs about the buzz at the BEA World show day 1 (Quinton Wall). One of the buzz items: ESBs have come quite a bit since their inception. Solutions based on ESB technology is emerging as a de-facto standard fabric for Service Oriented Architecture. They have become one of the most universally accepted solutions. Because of this I think we updated definitions about what ESBs are and where they are heading.

3 years ago when ESBs first started taking off we saw something like this which helps give a picture of how far we’ve come [this is from an actual email I saved, minus the annotations in red]

• Step 1: Connect .NET client with Java Web Services to AXIS. (Hey weren’t those supposed to inter operate?)
• Step 2: Validate, Transform, Route, Operate (I vetoed the VTRO because it was too inflexible)
• Step 3: Repeat steps above with JMS (Because without WS-RM there was no reliability!)
• Step 4: Add both Transport and Message Security with Identity propagation (no comment. This actually was a good idea and still is)
• Step 5: Just Add Monitoring (without interfering with the message or adding latency!)

Complete these five steps in under about 15 minutes (without writing a line of code) and you’ve entered into the ESB playing field.

Well now it is near the end of 2007 and we’re starting to see a new level of sophistication in what’s expected out of ESBs. I have compiled a list of the “New Seven Wonders of ESB” as an homage to the new 7 wonders which was dreadfully needed since no one could find, visit, or remember the original 7 wonders.

ESB Wonder #1: “Smart Connectivity”– ESB technology has been universally accepted, now the pressure is on to broaden their purpose to deliver on the original promises of SOA. For instance, ESBs have the flexibility to extend their reach into services, WS-*, JMS, File, FTP/S, MQ, but is that broad enough? What about legacy connectivity? What about out-of-the-box packaged app connectivity for SAP, Siebel, Peoplesoft, Oracle? What about custom/proprietary connectivity. Aren’t adapters dead already?

ESB Wonder #2: “Unified Integration” – And since ESBs are becoming the fabric for SOA, shouldn’t that fabric be woven seamlessly through all business integration technologies: ESBs + BPM, Data Services, EAI, AI, Semantic Platforms, and other emerging forms of integration? Without ESBs, these integration technologies rely on point-to-point integration which creates issues with future process changes & business agility.  

ESB Wonder #3: “SOA Governance Ready” – ESBs also need to monitor traffic “on the bus”. Is this SOA management? No. There is additional down-stream services managed holistically apart from the bus. AquaLogic Service Bus today works closely with AquaLogic SOA Management (for those of you that heard my round-table today!); I call this “managed-readiness”. But even more wondrous, ESB is a participant in the Governance lifecycle; policies, service metadata, prescribed assets can be centrally managed by a Repository/Registry. In summary, ESBs can become SOA Governance-ready when they are managed-ready and participate in SOA Governance.

ESB Wonder #4: “Industry focus” – Take a look at my blog post on AquaLogic Service Bus, Financial Services Edition, which addresses how the financial industry is adopting ESB to support financial payments & settlements standards. We’ll start to see more ESBs like this one that embrace the industry “flair” to build greater value than generic ESBs.

ESB Wonder #5: “No Boundaries” – ESBs are emerging beyond simple small project silos, they are traversing boundaries, bridging communities of interest, and connecting customers to partners, building increasing value along the way. We’re starting to hear companies both adopt big busses & little busses working together to provide a a Service Network (See Bill's blognote on Paul Patrick's Keynote)

ESB Wonder #6: “A new kind of ESB Virtualization” – Related closely with Wonder #6, we are starting to see companies adopt ‘virtualization’ strategies (not to be confused with service virtualization. The reason: save on operation costs as well as boost performance and lower latency. Undoubtedly, new ESBs will be on the forefront to run on virtualization engines. 

ESB Wonder #7: “Step beyond SOA” Which I would vote as the most significant wonder. ESBs are beginning to move beyond WS-* and into newer as well as even more traditional technologies. Ironically ESBs need to take a few steps back to take giant leaps forward. For instance Web 1.0 and basic events need to be supported before Web 2.0 or complex event processing. The increasing trend towards increased business visibility, near real-time response opportunities & threats and situational awareness will drive the need for CEP enabled ESBs to address the handling of low-latency, high volume events. The increased evolution of Web2.0 and enterprise mash ups (and even event aware mash ups) will drive the need for light weight (for REST based web services) ESBs 
 
As ESBs mature, ESBs will take on new forms to enable greater customer benefits. BEA is using AquaLogic Service Bus as its foundation core for SOA and quickly becoming a key to unlock what we see beyond SOA: Dynamic Business Applications - take a look at Jon's blognote on Alfred's keynote. Over the coming months, we’ll be announcing new editions to the AquaLogic Service Bus family to address many of these new wonders!

BEA Announces AquaLogic Service Bus - Financial Services Edition

I am here in beautiful Barcelona, for BEA World’s annual worldwide conference. There was a significant event this week, which marks a trend in what is happening with both ESBs, and SOA implementations in general. And surprisingly it has absolutely nothing to do with WS-* standards.

It has to do with “twenty-Oh-twenty-two” as it is lovingly called or ISO 20022. Financial Industry has always been the forefront of messaging infrastructure (long before the birth of SOA or ESB); what are those electronic payments anyway but simple, standards based messages? But typically these solutions are brittle as a concrete column in the financial district when it comes to change. Recently we’ve seen the financial services industry move aggressively to adopt SOA for their payments infrastructure to accomodate that change, as well as to reduce costs, reduce risk, and keep up with industry regulations.

For instance, take a look at SEPA (Single Euro Payment Area) which mandates that all payments in Europe single Euro standard by 2010; this is just the beginning. We’ll start to see multiple compliance initiatives like this one in both North/South America as well as Asia Pacific Countries.

Is this the tipping point for revolutionary new ESBs? Well it is one of the first ESB solutions to have an industry-focused edition. The trend here is that ESBs have evolved beyond their basic capabilities and can extend their reach to very industry specialized capabilities. 

For example, these are some of the new offerings for BEA AquaLogic® Service Bus, Financial Services Edition:

    *  An integrated design environment that can enable users to customize and
       define financial payment messaging standards including: SWIFT, SEPA and
       other payments standards;

    *  User-defined business logic associated with their custom SWIFT network
       validation, exception management, auditing and reporting;

    *  SWIFT connectivity through transports embedded out-of-the-box with BEA
       AquaLogic Service Bus, for lowest latency, highest performance and
       greatest flexibility;

    *  SWIFT message categories including customer payments, financial
       institution transfers, treasury markets (foreign exchange, money
       markets and derivatives and precious metals), collections, securities,
       documentary credits and guarantees, travelers checks, and cash
       management.

I would expect to see many more industry-focused solutions like this one, as well as new editions for AquaLogic Service Bus that address unique use cases for very specific and complex pain points. It will ultimately make SOA-adoption much more significant. And maybe it isn’t a fancy WS-* standard, but I wouldn’t scoff at twenty-O-twenty-two.

Mashing Up The Bus: Getting Ready for the Convergence of SOA & Web 2.0

Regarding Anant’s recent article on Mashups on “Are You Ready For Mashups”, he talks about something that we continue to hear: Web 2.0 technologies are converging to SOA and Enterprise IT technologies. This is real — and we better get ready!

So I thought I would write a short blog on an exercise I recently did while getting ready. The problem I wanted to solve: render individual AquaLogic Administrative dashboards into a consolidated view [Hey isn’t that a Mashup?]. For a simple example, I put a single Dashboard snippet view from AquaLogic Service Bus into AquaLogic User Interaction. You can almost think of this as a “Hello Enterprise Mashup World” example for combining elements of AquaLogic into Web 2.0 technologies. Granted, there are much more challenging things I could have incorporated, like using REST services together with Pages, Ensemble & Pathways, but I thought for all those Service Bus fans out there in the world, I would show something cool.

What this sample looks like… And this is just the beginning!

Requirements. Install these products First.

• AquaLogic Service Bus 2.6
• AquaLogic User Interaction
• Kapow WebClipper BEA Edition 6.2
• A Web Server – Any will do, you can even use Tomcat

Simple Instructions:

1. Start up the ALSB samples domain.
2. Build the Kapow webclip using Kapow by clicking the appropriate tags that you want to render in your ALUI application. For mine I did a simple login script and then click the Pipeline alerts tab. You can use the sample I provided if you would like to skip this step of creating a Kapow web-clip. You can also do something similar which is to use AquaLogic Pathways to extract data that you need. It so happened that this example the webclipping was all I needed, but I could have used Pathways to retrieve the % of errors or warning results into a web service.
3. Deploy the WebClip onto an app server. You can follow the example listed in Kapow documentation for using the rstl-demo as a sample on Tomcat.
4. Create an iframe to reference the Kapow webclip. You’ll need to do this to render it correctly in ALUI. You can use my example attached.
5. Using ALUI you will need to create a remote web service that points to the iframe on the Tomcat server.
6. Using ALUI now create a web portlet that points to the remote web service, almost there…
7. Load the portlet into a ALUI home page.

Now let us run it!

• You’ll need to restart the tomcat server after you deployed. WebLogic doesn’t have this problem by the way…
• Deploy the Robo Server for Kapow, this serves the webclip to the app server so that you can display the webclip appropriately.
• Make sure you send some messages through the ALSB sample app so you can see some pipeline alerts show up on the dashboard. The sample domain in ALSB doesn’t set Pipeline alerts by default, so you can add some of those to make your dashboard as pretty as mine.

So what is the point of all this?

Well, I’ve got a live, constantly refreshing piece of the AquaLogic Service Bus Dashboard sitting in a view in ALUI that can be delegated for a community of Operations and Architect. And by the way, did I mention that it refreshing in real-time, even while sitting in ALUI?

The point is this: it shows you just the beginning of what is possible when you start to take our Enterprise suite of products in AquaLogic, like Bus, BPM, Data Services Platform, Enterprise Security and shows you how you can extend the power of these products when you enable them for Web 2.0 in a collaborative environment. It just so happens that this aleady exists today with BEA.

Your Homework Assignment

• You can embellish this example using some of BEA’s new Web 2.0 products together with ALUI under Pages, Ensemble & Pathways – some cool resources there you can leverage.
• You can also extend this example to other consoles that exist in BEA.
• Share with us how you think this can be extended.

Good luck. And you better get ready. I know I am!  

File Attachment: ALSB.clip (4 KB)

File Attachment: iframe_alsb (81 bytes)