Paco Gomez's Blog

Algorithmic Trading with WebLogic Event Server

2008-01-30T08:14:11Z

In previous entries, I have discussed technical aspects of WebLogic Event Server and some exciting applications (1, 2). In the next entries I will write about Algorithmic Trading, probably the most common application in Complex Event Processing.

I've had the opportunity to work with the WebLogic Event Server Engineering Team to make available the Algorithmic Trading Demo on CodeShare. The purpose of this demo is to illustrate some techniques used in this field, also known as Algo Trading, and how they are implemented in WLEvS. Source code, application binary and a ready-to-run Event Server domain is available for download here.

Demo Description

The Algorithmic Trading Demo implements common indicators used in automated trading. The application receives market data through a socket connection and calculates the following results:

Percentage change. Detects when the price of a stock changes more than one percent.
Trend. Detects when the price of a stock fluctuates in one direction (up or down) more than two successive times.
VWAP (Volume Weighted Average Price), the ratio of the value traded to total volume traded.

These ratios are calculated and displayed in real time on a dashboard for a basket of symbols.

The dashboard also shows average latency and a distribution histogram of all latencies. The following is a screenshot of the dashboard:

How to Run the Example

The demo is distributed as two zip files:

"wlevs20_domain_algo.zip", it contains the WLEvS server with the demo deployed, ready to run.
“algo_trading.zip”, it contains the Algorithmic Trading application code and the data feed application that sends market data to the server. The application is distributed as an Eclipse project that can be imported directly into an Eclipse Workspace.

WebLogic Event Server 2.0 needs to be previously installed in order to run the example. We are assuming “c:\bea” as BEA_HOME. If WLEvS has been installed on a different BEA_HOME, see the specific notes in the steps below.

Follow these steps to run the example:

Unzip wlevs20_domain_algo.zip to %BEA_HOME%\user_projects\domains.
Unzip algo_trading.zip to %BEA_HOME%\user_projects\modules.
Start the data feed standalone application.
- Open a command window
- Change to %BEA_HOME%\user_projects\modules\algo_trading
- Run startDataFeed
- If BEA_HOME is not c:\bea, edit startDataFeed.cmd to specify a valid JAVA_HOME value.
Start the WLEvS server with the Algorithmic Trading application
- Open another command window
- Change to %BEA_HOME%\user_projects\domains\wlevs20_domain_algo
- Run startwlevs
- If BEA_HOME is not c:\bea, edit startwlevs.cmd to specify valid JAVA_HOME, USER_INSTALL, BEA_HOME and WLEVS_DOMAIN_HOME values.
Note: The data feed application can be started before or after WLEvS, with the same results.
At this point, the data feed application should be sending market data to the WLEvS server and the CPU utilization should go up.
On a browser window, open the following URL: http://localhost:9002/algo/dashboard.html. Push "start" button.

I will be providing more technical details about the demo in futures entries.

Technorati Tags: CEP, WLEvS, Algo Trading

GPS Tracking Demo for WLEvS

2007-11-28T17:52:18Z

In my last article, Location-based Information Processing with WebLogic Event Server and Web 2.0, I described a WLEvS application that detects the proximity of a vehicle to a location. I have posted the source code and binary of the demo on Dev2Dev's CodeShare.

The demo is now self-contained. It runs entirely on WLEvS and just requires an Internet connection to display the map mashup.

How it Works

WLEvS receives the GPS information stored on a file using a file adapter.
The processor determines if a position is near the target destination and generates an alert.
Positions and alerts are stored in memory on WLEvS application (Java Bean)
The same Java Bean is also a servlet that displays a JavaScript page. The page is a mashup of Yahoo! Maps and the data from WLEvS.

The dashboard is shown here:

Installation

Download the project from CodeShare and unzip the file. The code is distributed as and Eclipse project. It can also be built from the command line.
Copy "openmap.jar" to the WLEvS domain directory

Modify "startwlevs.cmd" to add the previous file to the bootclasspath:

set BOOTCP=-Xbootclasspath/a:openmap.jar
"%JAVA_HOME%\bin\java" %BOOTCP% ...

Start WLEvS
Update "setEnv.cmd" (JAVA_HOME and ANT_HOME) and "build\build.properties" (wlevs.home)
From another command window, build and install the application:
```
setEnv.cmd
build.cmd dist
build.cmd install
```
Open a browser window to access the dashboard. Here is the URL: http://localhost:9002/gps
Click on "Start Trip" to start the demo. Click on "Reset" to bring the demo to the initial state.

The Demo in Action

Monitoring JRockit Through a Firewall

2007-11-14T20:02:10Z

JRockit provides powerful monitoring capabilities with a very low resource consumption. That makes very common monitoring production JRockit instances using JRockit Mission Control (JRMC). It is also common to have a firewall protecting the production servers, sometimes in between the servers and the workstation where JRMC runs. In those scenarios, there are two ports that need to be open in the firewall: RMI Registry and RMI Server. The RMI Registry is typically 1099 but the RMI Server port is by default generated dynamically (and randomly) on the JRockit server and sent to the client (JRMC). Without knowing in advance what port is going to be used, it is not possible to configure the firewall to allow RMI communication to the server, therefore JRMC cannot connect to the JRockit instance.

A solution to this problem is to provide a configurable JMX Agent on the JRockit instance that uses a predefined port number for the RMI Server connection. The firewall can be configured to allow traffic through that predefined port. On the client, JRMC, we can specify the RMI port to be used to communicate with the RMI Server.

This solution doesn't require any changes to the existing application code. It can be used to monitor JRockit with JRockit Mission Control or any other JMX-compliant monitoring tool like JConsole or wlshell.

The attached jar file contains the ConfigurableJMXAgent, a JMX agent that can be used to specify the port number of the RMI Registry and RMI Server. Four system properties can be configured:

-Dconfig.registry.host=192.168.10.132
-Dconfig.registry.port=1099 
-Dconfig.rmi.host=192.168.10.132 
-Dconfig.rmi.port=9999

With these values, the firewall needs to allow traffic on ports 1099 and 9999. The URL to be used on the client, JRMC, is:

service:jmx:rmi://192.168.10.132:9999/jndi/rmi://192.168.10.132:1099/jmxrmi

The configurable JMX agent needs to be loaded by the JRockit server instance at startup. The following command line is provided as an example:

java -Xmanagement:class=com.bea.misc.ConfigurableJMXAgent 
     -Xbootclasspath/a:config-agent.jar 
     -Dconfig.registry.host=192.168.10.132
     -Dconfig.registry.port=1099
     -Dconfig.rmi.host=192.168.10.132 
     -Dconfig.rmi.port=9999 
     -jar C:\bea\jrockit-realtime20_150_11\demo\jfc\Notepad\Notepad.jar

The console will show the following messages:

ConfigurableJMXAgent: Created with params = [config.registry.host=192.168.10.132, config.registry.port=1099, config.rmi.host=192.168.10.132, config.rmi.port=9999]
ConfigurableJMXAgent: Create RMI registry on port 1099
ConfigurableJMXAgent: Get the platform's MBean server
ConfigurableJMXAgent: Initialize the environment map
ConfigurableJMXAgent: Create an RMI connector server
ConfigurableJMXAgent: Start the RMI connector server
ConfigurableJMXAgent: Started, use the following URL to connect =  service:jmx:rmi://192.168.10.132:9999/jndi/rmi://192.168.10.132:1099/jmxrmi
[JRockit] Local management server started.

The server is now ready. Start JRockit Mission Control and specify the connection URL as shown in the following image:

That should do it. Happy monitoring!

Links:
Configurable JMX Agent source code and JAR binary
BEA JRockit Product Center on Dev2Dev
wlshell

HTTP Adapter for WLEvS

2007-09-11T17:37:05Z

WebLogic Event Server provides an open and documented API to build adapters. Through an adapter, external applications can send events to WLEvS.

The HTTP Adapter for WebLogic Event Server allows sending events to WLEvS using HTTP Get or Post requests. Using this adapter, users can send events to WLEvS via a web browser, command line tools like wget, Service Bus (ALSB) or any programming language with an HTTP API, like Java, .NET, C, JavaScript, Perl, Phyton, Ruby, etc.

Along with the adapter, I have provided a modified version of the HelloWorld application to illustrate the use of the HTTP Adapter.

The HTTP adapter registers a servlet on the servlet engine included in WLEvS. It accepts HTTP get and post requests, extracts the parameter values and sends an event to the listener attached to it.

Here is a sample HTTP/Get request that sends an event. The URL is: http://localhost:9002/adapter?message=hello&number=123

Using a browser to send an event:

Here is another example using wget:

Here are the events processed by the HelloWorldHttp application:

Similar to the csvadapter, the HttpAdapter is configurable and generates any type of Event described in the EPN configuration file, for example:

The adapter factory registered in the HttpAdapter bundle:


  
    HttpAdapterProvider

The EPN configuration in the HelloWorldHttp application:


  
    com.bea.wlevs.event.example.helloworld.HelloWorldEvent

Some use cases and benefits include:

This adapter can be used during development and testing to generate any type of events without having to create a custom adapter. It is a bit more interactive than the csvadapter as users can type the event data in the browser and send it immediately
In addition to the browser, users can use the wget utility on the command line and create shell scripts
the http adapter can be used for performance testing with standard web testing tools like LoadRunner or Grinder
jetty, the servlet engine in WLEvS can be configured and tuned for multithread and performance
easy to integrate with programs via HTTP GET/POST API (Java, .NET, JavaScript, C, Perl, Phyton, Ruby, etc)
we can use ALSB in front of WLEvS and have, with little configuration, any protocol supported by ALSB (MQ, ftp, file, etc) be able to talk to WLEvS. We just create an ALSB business service that uses HTTP GET to talk to WLEvS

Some useful links:
HTTP Adapter for WLEvS
WebLogic Event Server Product Center
AquaLogic Service Bus Product Center

WebLogic Event Server Administration

2007-07-26T08:40:32Z

Configuration

The management tasks described here are performed using WLEvS version 2.0 and wlshell version 2.1.0. The example used is the HelloWorld application, included with WLEvS. By default, this domain has the JMX services enabled.

wlshell is a DSL (Domain Specific Language) for JMX. It is a scripting shell fully compliant with the JMX specification, providing access to MBeans in a convenient manner. Since both WLEvS and wlshell are based on standard JMX, the products interoperate directly out of the box, without any customization. WLEvS also includes another utility, wlevs.Admin, to access the MBeans and work with EPL rules.

On the machine running wlshell, it is not necessary to install WLEvS. The only requirements are the wlshell install files and a valid JDK or JRE (version 5 or 6). Please refer to the wlshell installation documentation and the section about using the JMXRemote connector. wlshell can run on the same machine as WLEvS or remotely on another server.

Connecting to WLEvS

We are going to start the HelloWorld WLEvS domain and wlshell. In wlshell, we use the "connect" command to connect to the WLEvS server, indicating the URL, user name and password, as shown here:

#connect
connect service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi wlevs wlevs

#change to wlevs domain
com.bea.wlevs:

#set keys
keys Type Name Application

#list current directory
ls

#show the explorer
explore

Here is a screenshot of those commands executed in wlshell:

We will be using the keys "Type, Name and Application" for the rest of the article, unless otherwise noted. The concept of "keys" is described in detail here.

The explorer is shown below, displaying the MBean server URL, available JMX domains and all the MBeans registered on the "com.bea.wlevs" domain. The explorer is very convenient to browse the MBeans and see what attributes and operations are available.

Managing Server Life Cycle

The ServerRuntime MBean provides an operation to shutdown the server. The command to shutdown the server is:

cd /ServerRuntime
invoke shutdown

or simply:

invoke /ServerRuntime/shutdown

]]> Managing EPL Rules

Each Processor inside an Application has one corresponding MBean, which exposes EPL Rules management operations. The following screenshot shows the MBean for the "helloworldProcessor" in the "helloworld" application:

The following screenshot shows a sample interaction with the EPLProcessor MBean to add and delete EPL rules:

The script used is:

app = helloworld
cd /EPLProcessor/${app}Processor/$app

get AllRules
invoke getRule helloworldRule
invoke deleteRule helloworldRule
get AllRules
invoke addRule helloworldRule "select * from HelloWorldEvent retain 1 event"
get AllRules

Accessing other MBeans

WLEvS provides MBeans for many other components like streams and application adapters. We can access their attributes and operations in the same way with wlshell.

As an example, the following script modifies dynamically the Message attribute of the "helloworldAdapter":

cd /HelloWorldAdapterConfig/${app}Adapter/$app
get Message
set Message "Hi! the time is now: "
get Message

This is the wlshell output when executing the previous script:

The change takes effect immediately, as seen in the WLEvS console output:

Conclusion

WebLogic Event Server provides all the management operations through the standard JMX interface. JMX-compliant tools, like wlshell, can be used to effectively administer WLEvS. In this blog entry I have described some basic management tasks, using MBean attributes and operations. Additional administrative tasks can be done on other MBeans using the same techniques described here.

References

WLEvS

wlshell

Configuring WebLogic Resources, a la wlshell

2007-07-20T08:00:43Z

In my previous entry, I discussed about how to connect to an Admin Server and how the MBeans are organized in three MBean servers. Here I will explain how to make changes in a domain, creating a Managed Server as an example.

All changes on the domain need to be done on the "Edit" MBean server and within the scope of a configuration session. The general process is as follows:

1. start a configuration session
2. make changes to the configuration
3. save changes
4. activate changes

The Configuration Manager MBean is used to manage a session. Since the name is quite long, I'll define a convenience variable with its name:

#keys
keys Type Name
#Configuration Manager
cm = edit.com.bea:/weblogic.management.mbeanservers.edit.ConfigurationManagerMBean/ConfigurationManager

The "keys" command is very important to be able to navigate the MBeans. There is a complete description of the command and the concept behind it in the wlshell documentation.

Also, I'm going to define some variables that I will use later in the script:

#domain
domain=myDomain
#admin server
server=AdminServer
#managed server
ser=myServer01

The first step can be done by calling the "startEdit" operation:

############################################
#start change session
wm = 60000
tm = 120000
invoke $cm/startEdit $wm $tm

Now I can make changes to the configuration. I'm going to create a Managed Server:

############################################
#make changes
edit.com.bea:
cd /Domain/$domain
invoke createServer $ser

cd /Server/$ser
set ListenPort 7111

You might remember that in WLS pre-9 I had to create an MBean to have a new resource, while here I use an MBean operation to create the resource, "createServer" in this case.

Finally, steps three and four:

############################################
#save and activate
keys Type Name
invoke $cm/save
invoke $cm/activate $tm

The new Managed Server will now appear in the "Edit" and "Domain" MBean servers.

The wlshell distribution contains additional examples using WebLogic 9 and 10:
- create and delete a DataSource
- create and delete a JMS Server and JMS Queues
- create and delete a Managed Server

I will cover some advance configuration topics in future entries, as well as other exciting new features of the shell.

WebLogic 10, as seen through wlshell

2007-07-11T22:19:36Z

In this first entry of the series, I will show how to connect and browse a WebLogic domain configuration.

WebLogic Server 9 and 10 changed internally the JMX implementation, which is now based on JMX 1.2 and JMX Remote API 1.0. But the change most obvious to users is the way how MBeans are organized and used to manage a domain.

As in previous releases of wlshell, users can still connect using a simple URL:

connect localhost:7001 weblogic weblogic

But the Administration Server has now three MBean servers: Domain, Edit and Runtime. When using the traditional URL (t3://host:port) wlshell connects to all MBean servers defined in that WebLogic server instance. It uses a prefix to differentiate between JMX domains defined in each MBean server. It prepends "domain.", "edit." and "runtime." to the names of the JMX domains defined in Domain, Edit and Runtime MBean servers respectively.

The explorer shows all the JMX domains available in the connection:

The JMX domain "domain.com.bea:" gives you information about what is configured on the WLS domain:

In the next entry, I will describe how to modify a WebLogic domain configuration.

Have fun!

Connecting ALSB to i5/OS and OS/400

2006-12-12T23:38:43Z

I Just posted the ifiveos ALSB Transport for i5/OS on Code Share.

ifiveos ALSB Transport for i5/OS and OS/400.

ifiveos is a transport extension that I've developed using the ALSB Transport SDK. It uses the IBM Toolbox for Java (or the open source version, JTOpen) to connect to i5/OS and OS/400.

The ifiveos ALSB Transport supports outbound synchronous calls from ALSB to i5/OS programs, written in RPG and other languages. Business Services based on ifiveos accept XML messages in the XPCML specification and return XPCML documents. XPCML is an XML format describing program calls, input parameters and return values.

Connecting ALSB with MQ

2005-12-02T21:18:26Z

BEA AquaLogic Service Bus (ALSB) provides interoperability with WebSphere MQ. For example, you can configure a Web Service Proxy in ALSB that sends messages to a MQ queue. Then you can have Web Service Clients using that Web Service Proxy to integrate applications with MQ, without having to install any MQ software on the clients. The Web Service Proxy can also be part of a composite application.

I have created a step-by-step guide to connect ALSB to MQ which can be found here: ConnectingALSBwithMQ

The two main steps are:

1 Bind MQ JMS objects to ALSB's JNDI tree
2 Configure the Business Service

I've also developed a small utility program, "ForeignJNDIHelper" to bind the foreign JMS objects to ALSB's JNDI tree. The utility can be downloaded here: foreignjndihelper

How JMX Saved My Day

2005-04-02T04:23:13Z

During one of my last proof of concepts, I was helping with the migration of a home grown portal to WebLogic Portal 8.1. One of the old features to reuse was usage tracking, integrating it with WebLogic Portal. The idea was to include a WLP behavior tracking tag in the portal JSPs and to persist the tracking data using the legacy code. The portal tag looks like this:

WLP offers a pluggable architecture for adding custom listeners to events. My custom listener would call the legacy code to persist the tracking information. The listener implements the com.bea.p13n.events.EventListener interface and it is developed as a utility class packaged as a JAR file in the APP-INF/lib directory.

The legacy code to persist tracking data was located under the WEB-INF/lib directory of the portal Web Application. For several reasons that code was packaged in this way and we couldn't change it. This packaging and the standard classloader hierarchy in an enterprise application made impossible to call the persisting function from the custom listener.

Having work with JMX extensively made me think I could use it to solve this dilemma. The JMX libraries are in the system classpath and the MBeanServer can be obtained from the JNDI tree. If any code running on the server can call MBeans, I could wrap the persisting functionality in a custom MBean and make it available for my custom event listener.

I developed the custom MBean and I registered on the MBeanServer from the Web Application, then I invoked one of its methods from the event listener loaded with the application classloader. The approach proved to be a good solution for the restrictions imposed by the classloaders, without having to configure custom classloader hierarchies. The added benefit was that the legacy persisting functionality was now manageable through JMX tools like wlshell.