Figure 1. Dependency navigation in BEA AquaLogic SOA Management (click for a larger version of the image)
Published on dev2dev (http://dev2dev.bea.com/)
http://dev2dev.bea.com/pub/a/2007/06/understanding-soa-management.html
See this if you're having trouble printing code examples
by Dain Hansen
06/12/2007
Why should you be interested in SOA management? First ask this question: Do you know today where your services are and what they're doing? Are they contributing to a more optimized agile and responsive business? Or are they contributing to a burden that is impossible to manage? Getting your arms around SOA requires getting your arms around SOA management. This article defines what challenges exist with SOA today, and how SOA management can address many of those challenges. In addition, this article addresses key capabilities to look for in a SOA management solutions, and looks briefly at BEA's SOA management product offering.
Do you really know where your services are? Do you know how many services in your enterprise are in production? How many of them are running at the level of quality you expect? Are any of the services improperly secured or exposed directly to customers or partners? Perhaps there are services you didn't even know you had? Do you have anything to protect against when your services behave incorrectly? How are you monitoring your services today?
These are many of the questions that you should ask in architecting today's SOA. Surprisingly, many organizations can't answer these questions, despite the fact that they depend on these services for mission-critical business applications.
So let's review a little bit of the fundamentals. What is SOA, really? SOA—or Service Oriented Architecture—enables enterprises to rapidly exploit new business channels by the improved reusability of services. SOA management then attempts to get its arms around an SOA and manage it both in terms of visibility and control. Without management, there is no guarantee that an SOA lives up to its expectation. Ultimately, SOA may even jeopardize an organization as it depends on these services for mission-critical applications.
What makes managing these services in an SOA so difficult? First, services in an SOA come in all shapes and sizes (and standards), from granular to coarse-grained services, from traditional non-XML, event-based queues, to more innovative XML based WS-* compliant services. Services are unique in the platform they run on; some may be Java EE-based, .NET, or other legacy or proprietary platforms.
Second, SOA may increase in complexity when implemented across multiple divisions and projects. And even when this isn't intended, some SOAs are forced into a distributed environment simply because of the complex nature of the enterprise. For example, services may be designed by one department for reuse across other departments, or services published by partners may be leveraged internally, and vice versa. Without the ability to centrally manage these services in a distributed environment, any service that changes or fails can have significant consequences on the business's bottom line.
Third, SOA systems evolve quickly. They grow in scale and complexity as new versions are published, additional applications are woven into the SOA landscape, and new customers and partners participate. A change to any service can set in motion a ripple effect across the entire network, potentially crippling an entire system.
Because SOA is heterogeneous, distributed, and highly fluid, it is hard to get your arms around it using traditional approaches. In fact, traditional IT management frameworks may be incomplete, and new approaches centered on SOA are necessary to give better visibility and control to an organization.
Traditional IT services management frameworks include capabilities to manage applications and are really good at getting better visibility to the generic behavior for applications. Is an application up or is it down? Unfortunately, an SOA is not so boolean. A mission-critical service can be completely broken from the perspective of the consumer, but the applications are alive and running.
In addition, traditional IT services management is important as a basis for a strong set of SOA applications, but SOA management will help manage and optimize this behavior back to the line of business. SOA is also not always about the applications: services may live across wire message protocols or events. They can be processed by intermediaries, and then end at a destination in an area you can't see, for example with a partner or a customer.
Ultimately, SOA management needs to coexist with IT management strategies. Standards in JMX, SNMP, WS-Man, or WSDM can help provide coexistence between the two. For example, SNMP trips fired by SOA management can be consolidated and managed leveraging an ESM such as an HP OpenView or BMC Patrol.
The original reasons for implementing an SOA were to simplify the complexity of your IT. Unfortunately, giving visibility to only a handful of IT operations staff will not take advantage of your original SOA objectives. SOA management can help fulfill that need by bringing the right information to the right people; this information can be collected in the form of service metrics, scorecards, and policy information, and then exposed to architects to better optimize the end-to-end architecture.
The benefits of implementing SOA management will ultimately create broader visibility and control for your organization. This can yield innumerable benefits to a company and lead to greater agility of the enterprise—the original promise for implementing SOA.
In fact, one of the real reasons for implementing SOA management is to enable SOA governance. For example, if you associate runtime scorecard data to services, you can make better decisions about which services you need to optimize as part of your governance lifecycle.
Some additional benefits for SOA management include:
These factors enable more efficient cost savings, risk reductions, and business revenue opportunities. This not only justifies the ROI for SOA management solutions, but it also justifies the investment for SOA in general.
Every environment in SOA is unique, but there are consistent requirements in SOA management that every organization can utilize for its specific needs. These are key elements in SOA management today; they are critical for optimizing how visibility is collected across systems and how control is implemented. They include:
Let's look at each of these in a little more detail.
Because the nature of SOA is to liquidate application silos into multiple services scattered across a network, distributed systems increase the risk of failure exponentially. Production of SOA systems, 24x7, requires comprehensive, system-wide tracking and operational visibility to minimize failures and downtime.
The visualization needs to include enough information—both the logical and the physical service network end-to-end. For example, you may want to see throughput and availability for each service node so that you can rapidly identify any operational issues with a particular service. You'll need the ability to zoom into the network to graphically assess service dependencies and outage impact as well as drill down to the end point itself to understand any of the relationships associated with that service. Figure 1 shows an example of this.
Figure 1. Dependency navigation in BEA AquaLogic SOA Management (click for a larger version of the image)
The more fragmented an organization, the stronger the need for unified monitoring across the service network. A great example is a type of company that has made multiple acquisitions with multiple packaged application software such as order management or inventory systems. The company wants to consolidate but doesn't yet understand the actual runtime impact of the service behavior across systems. Managing these complex relationships of these services can be implemented only by a SOA management system that has the ability for a company to then start to recognize where consolidation can begin to take place and make future optimizations.
Finding out what has actually been implemented, as opposed to what was intended, has often been an a roadblock for organizations seeking to gain the full potential of an SOA. Discovering the actual can lead to discovering potential issues from rogue consumption. You can also apply more rigorous governance policies by discovering rogue services that haven't gone through the official process. Integrating discovery capabilities directly into SOA management gives a more accurate representation for an organization's architects, IT, and operations.
In SOA management you'll also need to model and represent all services in your SOA as well as autodiscover services wherever they may reside. These services may have deployment characteristics. In addition, you'll need to autotrack these changes to deployments in multiple run-time environments.
Here is a great example of service and infrastructure discovery in action: a company wants to get better governance processes in place for services across the company. The company has no means or processes to find these services across a complex organization with multiple projects and divisions. By implementing SOA management, these services can be detected when they are either hidden or are possibly even rogue services. Unbeknownst to the company, these services may even be used by their customers without any protections put in place. Once they are detected by a SOA management solution, they can then be fed into a governance process, or policies can be applied to increase levels of message security enforcement to better protect these services.
Often visibility (the read-only view) of an SOA is not enough; rather, the capability to enforce policy is what will enable your organization to address SOA breakages, as well as optimizing it for better responsiveness.
A key enabler of SOA management is the use of policies for declarative specification of system characteristics for a service or a set of services. Policies can represent various attributes of a system ranging from process and function to security, performance, and robustness requirements for the infrastructure on which the system executes. Policies can also be a mechanism for control, for enforcing change to new attributes of your SOA. Changing a policy can be a means to communicate change swiftly and also accurately.
Systems are made more adaptive by specifying more of their behavior as policy (rather than procedural code) because policies are more concise, easier to understand and verify, and much simpler to change than code.
For example, an IT organization might decide to change its user authentication from entering username and password to supplying a certificate. In a policy-based world in which the security policy is separated from the application, the IT staff would describe the change in a declarative fashion and run the application on infrastructure that would dynamically enforce the security policy provided. If the security policy changes, the code doesn't have to. The revised policy is provided to the system infrastructure, which dynamically adapts to implement it. It's easy to see the advantage of not requiring a system maintenance release to make such a simple change.
Another example is to declare routing behavior via policies; should a service become unavailable, rules declared in a policy can throttle or redirect requests to mirror services to accommodate the load. A simple change to the routing policy can alter the behavior of the system.
Service-level agreement management is crucial to any organization planning to make production use of enterprise-class SOA. When first starting out, success hinges on defining, tracking, and controlling appropriate service levels over the course of a pilot project. When implementing SOA systems, it's necessary to review and analyze quality-of-service (QoS) metrics in order to plan for growth, minimize risk, and justify additional investments.
With more complex SOA systems, companies need to evaluate service-level goals over extended periods of time, across continuous and discontinuous processes, and prioritize shared service resources by relevant business context, such as customer type, product line, or business unit.
To maintain the quality of your services, you've got to head off potential problems by proactively resolving any issues that arise. To do so effectively, you need a SOA management solution that provides full coverage for disparate formats, protocols, interface, and transports.
SOA makes it easy to deliver your business applications in the form of service composites or business processes that orchestrate multiple business services. However, with layers of services coupled together, it becomes more difficult to manually track the business flowing through a system. That's because pieces of information are scattered across multiple log files, across multiple servers across potentially different geographical locations.
Neither IT nor business teams have had the visibility they need to efficiently manage these composites or processes. In fact, they typically first discover exceptions through cryptic log entries or calls from vexed customers. What typically follows are resource-intensive fire drills and finger-pointing across various teams, increasing maintenance overheads incurred by IT. At the same time, disrupted or failed business transactions result in lower customer satisfaction, lost orders, lower revenue, and general inefficiencies.
To prevent their IT from turning into silos of complexity, enterprises need more granular visibility into these applications and business processes. Visibility into just the individual services is no longer sufficient.
As discussed earlier, these five elements in SOA management are important, but how does this work with governance? For SOA governance to be effective, SOA management extends the governance model into the runtime. This should leverage standards-based UDDI mechanisms to bidirectionally exchange design-time metadata and runtime information.
One of the important requirements is that SOA management needs to be built in to a governance lifecycle and then have the ability to bidirectionally exchange information with design-time governance mechanisms.
For example, once the service and its associated policies are deployed and synchronized within a registry/repository, SOA management automatically begins monitoring, managing, and enforcing policies and service compliance, regularly updating a registry/repository with scorecarding of services and runtime attributes of services it collects.
SOA management can also discover additional hidden or rogue services and track how services are being used, whether these are hidden or rogue services that are not necessarily catalogued within the registry/repository. SOA management can also help fulfill how services migrate from staging to production and correlate scorecard data back to the registry/repository for optimization of business processes.
Using SOA management together with the registry/repository can enable closed-loop governance where actual service details in runtime are collected and cross-referenced against intended design. This is one of the most important aspects of governance, which enables iterative optimizations and ultimately optimizes an SOA for better agility across the system.
The combination of SOA and policies helps to create truly adaptive systems. However, effective SOA management also needs to include optimizations that can instantly heal these complex service networks. For example, the ability to reroute a service based on a service failing is an important capability for SOA management. Service failover to an additional destination when the end-point destination has not just gone down, but has failed an important SLA, is critical for the success of an SOA. Figure 2 shows an SLA monitoring screen.
Figure 2. An SLA monitoring screen from BEA AquaLogic SOA Management (click for a larger version of the image)
As new technologies emerge in the space of virtualization, services can also be redeployed in real time based on event triggers in an SOA management. Look for such convergence of SOA management with virtualization—real-time techniques that are beginning to take hold in the industry.
BEA recently announced an answer for SOA management: AquaLogic SOA Management, the industry's most comprehensive platform for gaining visibility into an SOA and diagnosing SOA-based composite applications and business processes.
AquaLogic SOA Management tracks individual messages SOA-wide and automatically correlates them into their respective transaction or process flows. This unique vantage point offers a business-centric view into the system, providing central visibility into each message flow from start to finish. This eliminates the need to manually piece together how services behave together.
AquaLogic SOA Management addresses the key capabilities within SOA management—Service Network Monitoring, Service and Infrastructure Discovery, SLA Management, Exception Management, and Policy Enforcement—but it delivers these key capabilities together with a governance solution, which also includes AquaLogic Enterprise Repository and AquaLogic Service Registry.
AquaLogic SOA Management extends SOA management to include broader coverage within BEA Enterprise 360. For example, AquaLogic SOA Management can gain extended visibility within an AquaLogic Service Bus Proxy for services that may be connected to non-SOA-based entry points, like JMS, EJB/RMI, MQ, Mainframe, or even custom proprietary transports for legacy applications or packaged applications.
SOA management is critical for getting the full value out of your SOA. Meeting the promises of agility for your organization not only requires a rock solid foundation in SOA, but also a managed SOA. SOA management requires visibility to your services, discovery of hidden untapped services, protection of your end points, and the assurances your customers and partners demand. Together these provide critical insight across the complexities of your organization to unleash your SOA to its full potential.
SOA management is also a key actor in SOA governance; the real-time insight that SOA management provides enables optimizations as part of the SOA lifecycle. These iterative optimizations are a cornerstone to any successful SOA.
BEA leads the way with one of the best SOA suites on the market today. BEA AquaLogic SOA Management can help you answer the question, What's in your SOA?
Related blogs:
Dain Hansen is Director, Product Marketing Manager for SOA Integration at BEA. He is currently working on projects in SOA, including: SOA Management, Governance and has long running experience with ESB technology.
Return to Dev2Dev.