Environment proving with WLST

Bookmark Blog Post

	del.icio.us
	Digg
	DZone
	Furl
	Reddit

Hussein Badakhchani's Blog | October 25, 2005   6:53 AM | Comments (7)

If you're responsible for delivering a environment how can you be sure that your delivery meets the recipients expectations? If you're on the receiving end how can you check that you have a been given an environment that meets your specifications; you did have some specifications didn't you?

Missconfigured JDBC connection pools, conflicting multicast addresses, incorrect listen ports, infinite timeout settings. These are just a few examples of kind of configuration cockups that can occur when project demands require fast turnaround of environments in multi-platform, multi-application enterprises. The impact of these errors can be trivial however, often it can lead to extended periods of pipe cleaning draining resources from infrastructure and application teams.

At this point advocates of Test Driven Development or Continuous Integration might pretend to know the answer, but in my experience use of these approaches does not extend well out of development environments simply because the maintenance overheads demand dedicated teams to keep the tests up to date or to service the titanic applications used to run them; to use this camps own vernacular, they are not "agile" enough.

I have been looking for a tool to use in order to proove WLS configuration for some time. What I want to be able to do is given a file that contains WLS configuration information, use that file to verfiy if a server is configured with those values. This sounds easy enough but when you consider how many parameters are involved, the number of different servers to test and rate at which parameters change its difficult to find a tool that is easy to maintain and does an effective job, especially if you don't want to spend any money on it...

After playing around with WLST for a few days and a crash course in Jython (Python) I managed to knock up a simple framework for environment proving driven by Ant. The main thing that impressed me about WLST was its use of Jython, anyway here's how it works:

To start with lets create our WLS server configuration file. This file defines what the value of the WLS parameters should be:

# WLS config for myserver
url=t3://localhost:7001
username=weblogic
password=weblogic
Domname=mydomain
ServerName=myserver

# '/'
root_AdministrationPort=9001
root_ArchiveConfigurationCount=5

# '/ServersConfig/ServerName'
serverConfig_AcceptBacklog=50

Notice the parameter names are prefixed with a token that indicates their position in the 'directory' of parameters provided by WLST. This is pretty ugly and is a shortcoming of my implementation, but hey, I've only been working on this for a couple of days! The property file is loaded by WLST and all the parameters become variables available to our script. Here is the ant tag responsible for running WLST:

<!--
 Check WLS configuration.
 -->
<target name="chkcfg" 
        description="Checks the configuration
        of the environment against its properties
        file under ${etc.dir}">
    <java classname="weblogic.WLST"
          classpathref="project.class.path"
          fork="yes">
        <jvmarg value="-Dpython.path=${lib.path} 
            -Dpython.cachedir=${lib.dir}/cachdir
            -Dpython.verbose=debug"/>
        <arg line="-loadProperties ./etc/${environment}
             ${wlsconfig.dir}/testconfig.py"/>
    </java>
</target>

The important thing to note here is the <arg> tag, as you can see the name of the property file to load is passed in as system variable to ant, this allows us to target different environments to test against. Now lets have a look at the Jython script that does all the hard work, make sure you read the comments:

# WLST Environment proving script.
# Hussein Badakhchani
# This is my first stab at using WLST and Jython to create an
# environment proving framework. Once its finished I will call it
# Jamshed.

from string import split

# Store our test results in erm.. results.
results = []

# We store the keys associated with different
# "directories" in their own lists
# e.g. these map to '/' and '/ServersConfig/ServerName'
rootKeys = []
serverConfigKeys = []

# A simple assert wrapper. Looks like we need to convert everything to
# String in order to ensure the equality is valid.
def assertEquals(value, expected, argName):
    try:
        assert str(value) == str(expected)
        return '%s %s PASS' % (argName, value)
    except AssertionError, e:
        return '%s %s FAIL expecting %s' % (argName, value, expected)

# Given a set of keys we fetch the values associated with those keys
# from our server (using WLST) and check that they equal the
# values defined in our property file.
def checkConfig(keys, prefix):
    for key in keys:
        value = get(key)
        results.append(assertEquals(value , eval(prefix + key), key))

# Loop through all the variables and grab those defined in our
# environment properties file, currently prefixed with 'root' or
# 'serverConfig'.
for a in dir():
   keyname = split(a,'_')
   if keyname[0] == "root":
       rootKeys.append(keyname[1])
   elif keyname[0] == "serverConfig":
       serverConfigKeys.append(keyname[1])

print 'starting the script .... '
# Connect to the server.
connect(username,password,url)

# '/' The properties can be viewed from 
# domain/configuration/general in the WLS console.
# Check the parameters under root
checkConfig(rootKeys, 'root_')

# This is cluncky it would be nice if WLST had the option of
# appending the full path to the parameter against the parameter
# read fom the property file, this way the script wont need to
# cd.
# TODO clean this up!
cd('/ServersConfig/' + ServerName)

checkConfig(serverConfigKeys, 'serverConfig_')

# Print the results out.
for result in results:
    print result

disconnect()

print 'End of script ...'
exit()

WLST sets up variables named after the keys in our property file and assigned them corresponding values. This script retrieves the keys, and after a bit of formating uses them to obtain the corresponding values on the WLS server, then using the Python assert keyword we check the values are what we expect them to be, simple! Here is some sample output:

/hoos/jamshed/bin > ant -Denvironment=myserver chkcfg
Buildfile: build.xml

chkcfg:

     [java] Initializing WebLogic Scripting Tool (WLST) ...

     [java] Welcome to WebLogic Server Administration Scripting Shell

     [java] Type help() for help on available commands

     [java] starting the script .... 

     [java] Connecting to weblogic server instance running at 
            t3://localhost:7001 as username weblogic ...

     [java] Successfully connected to managed Server 'myserver' 
            that belongs to domain 'mydomain'.


     [java] AdministrationPort 9002 FAIL expecting 9001
     [java] ArchiveConfigurationCount 5 PASS
     [java] AcceptBacklog 50 PASS

     [java] Disconnected from weblogic server: myserver
     [java] End of script ...

     [java] Exiting WLS scripting shell

BUILD SUCCESSFUL
Total time: 9 seconds

The code leaves a lot to be desired, the main areas I think are the directory changing and use of prefixes on the property keys, but the point is how easy it is to maintain these tests. Once the properties for a server have been captured (I used WLST to create them in the first place) changing them is easy and with a few changes to WLST itself to allow the option of prefixing of properties or even dumping the full path to an MBean parameter the script will become alot simpler.

I don't much like Junit and other cumbersome testing frameworks but using Jython it is easy to hook up existing tests with WLST if you must. Another tool I have subverted to environment proving is The Grinder, more on that in my next entry where I will show how I hook up all these tools and run them from ant. At that point I should have the whole framework ready for download if anyone wants it.

Comments

• For Hoos' regular readers, the above code snippit has caused more gnashing of teeth and frank discussion than is usual in an office.//

  for a in dir():
     keyname = split(a,'_')
     if keyname[0] == "root":
         rootKeys.append(keyname[1])
     elif keyname[0] == "serverConfig":
         serverConfigKeys.append(keyname[1])
  

  This is the python equivalent of filing you tax reciepts on the floor, then having your accountant rummage around looking for them at the end of the year. This, in Hoos' defence, is actually the fault of WLST. It should pass its variables around as a list or dictionary and not just dump them as named objects on the global name space.

  Posted by: simonvc on October 25, 2005 at 7:23 AM

• Its a fair comment. I think the offending class is weblogic.management.scripting.utils.WLSTUtil. It has a static method setProperties that seems to be responsible for enumerating over the the property file and making named objects out of them.

  If I get the time I might try a fix, anyone from the WLST team care to comment?

  Posted by: hoos on October 25, 2005 at 8:40 AM

• Hoos, great tool IMO. Thanks for sharing. Well, loadProperties is very similar to ant's LoadProperties. It will let you access property names as WLST variables if you know the variable name you want to access. Using dir() to get to all the variables and iterating over them to find the properties you have loaded is very inefficient, because a simple dir() in WLST is a dictionary of 650 objects (or more), ouch!. In your particular case since you would like to iterate over the properties that are loaded via the file, it would be better if the properties are loaded by the script that is going to use it. So, when you invoke WLST from ant, you would pass in the properties file location as an argument. and you can access this value in the script as shown. propFileLocation = sys.argv[1] Now you can load this properties file. # few imports from java.util import Properties from java.io import FileInputStream from java.io import File from java.util import Enumeration myProps = Properties() # load properties myProps.load(FileInputStream(File(propFileLocation))) # now iterate over these propertyNames = myProps.getPropertyNames() while propertyNames.hasMoreElements(): keyname = split(str(propertyNames.nextElement()),'_') if keyname[0] == "root": rootKeys.append(keyname[1]) elif keyname[0] == "serverConfig": serverConfigKeys.append(keyname[1]) Also check out WLSTAntTask at https://codesamples.projects.dev2dev.bea.com/servlets/Scarab/id/S112, this is an Ant wrapper over WLST that lets you invoke wlst from Ant more cleanly. You can also embed script snippets in your build file (check out the ReadMe in the downloaded zip file). This will be in the product for the next release of WLS. Hope it helps, -satya

  Posted by: sghattu on October 25, 2005 at 2:34 PM

• Re-posting since my earlier comment did not honor line breaks. Sorry about that.

  Hoos, great tool IMO. Thanks for sharing.

  Well, loadProperties is very similar to ant's LoadProperties. It will let you access property names as WLST variables if you know the variable name you want to access. Using dir() to get to all the variables and iterating over them to find the properties you have loaded is very inefficient, because a simple dir() in WLST is a dictionary of 650 objects (or more), ouch!. In your particular case since you would like to iterate over the properties that are loaded via the file, it would be better if the properties are loaded by the script that is going to use it. So, when you invoke WLST from ant, you would pass in the properties file location as an argument.

  arg line="${wlsconfig.dir}/testconfig.py ./etc/${environment}"/>

  and you can access this value in the script as shown.

  propFileLocation = sys.argv[1]

  Now you can load this properties file.

  # few imports

  from java.util import Properties

  from java.io import FileInputStream

  from java.io import File

  from java.util import Enumeration

  myProps = Properties()

  # load properties

  myProps.load(FileInputStream(File(propFileLocation)))

  # now iterate over these

  propertyNames = myProps.getPropertyNames()

  while propertyNames.hasMoreElements():

  keyname = split(str(propertyNames.nextElement()),'_')

  if keyname[0] == "root":

  rootKeys.append(keyname[1])

  elif keyname[0] == "serverConfig":

  serverConfigKeys.append(keyname[1])

  Also check out WLSTAntTask at https://codesamples.projects.dev2dev.bea.com/servlets/Scarab/id/S112, this is an Ant wrapper over WLST that lets you invoke wlst from Ant more cleanly. You can also embed script snippets in your build file (check out the ReadMe in the downloaded zip file). This will be in the product for the next release of WLS.

  Hope it helps,

  -satya

  Posted by: sghattu on October 25, 2005 at 2:43 PM

• Satya,

  Thanks for posting the code to your suggestion, we came to this conclusion in the office as well, looks like a short refactor is in order, incorporating your contribution!

  What would be really handy is if WLST could output all the properties of a WLS server in a properties file with the full "path" appended to them in conjunction to being able to call:

  get('full/path/to/MyParameter')

  That would make WLS configuration and comparisons and testing easy, as everything could be driven from WLST.

  Posted by: hoos on October 25, 2005 at 3:23 PM

• This may not be documented, but you can use get command to specify a path to the parameter. For example, to get to the ListenPort of SSLMBean from the root w/o doing a cd. get("serverConfig:/Servers/myserver/SSL/myserver/ListenPort"). Also, I just realised that you can use the 'find' command to dump all the properties in the domain. Example, find("") Please remember that this will take a while because it queries all the mBeans in the mbean server, calculates the path for each mbean and dumps all the attributes and its values.

  Posted by: sghattu on October 25, 2005 at 4:02 PM

• This sounds perfect, if I could post reward points onto blog entries you would get them! Perhaps thats feature Jon could look at :)

  Posted by: hoos on October 25, 2005 at 11:37 PM