How not to manage application configuration

Bookmark Blog Post

	del.icio.us
	Digg
	DZone
	Furl
	Reddit

Hussein Badakhchani's Blog | June 24, 2006   3:54 AM | Comments (6)

Having to manage an applications configuration via environment parameters is not an acceptable solution for enterprise applications. I am currently dealing with a few applications that "cleverly," change their behaviour based on environemnt parameters passed to the JVM.

Unfortuantely when this design decision was taken no one considered the effort required to manage the application configuration across multiple environments; the changing demands on those environments or simply how long it takes to implement a change to the configuration of a server (alter the remote start parameters and restart the cluster, up to 40 minutes on some of our servers).

Developers can get away with such poor mechanisms to configure applications when operational requirements are droppped or never considered in the first instance. If you are about to add some environment variables to your application I suggest you take a few minutes to consider the implications of what your doing and understand the consequences of having to manage and change these settings.

There is an API out there that can help managed your applications configuration, it is called JMX and while it has its flaws it will probably save the poor infrastructure and operations teams many hours of time wasted stoping and starting servers to "switch on debugging" or "use testing mode".

Comments

• There was a poll here on dev2dev the other day asking about what proportion of users use JMX, both custom and the generic admin beans. I cant remember the exact details, but i dont think many people were using custom beans at all.
  I think more people probably keep that sort of runtime config as a table in the database.
  I guess it comes down to "demarcation of responsibility".
  Who has access to what.
  If the devs/testers have access to the database keep config there. If they have access to stop and start weblogic i dont see the harm in keeping config there.
  If they dont have access to either, then i guess knocking up a WLST to push some JMX around is probably a good solution.
  

  Posted by: simonvc on June 26, 2006 at 1:31 AM

• There are several big problems with using environment properties to change application behaviour. These are:

  1. They are difficult to maintain, and can very easily be lost on the RemoteStart tab that like to keep the settings in a single text field (why not a text area BEA???).
  2. The settings can very easily differ between environments and cause erros to occur against one domain which are difficult to diagnose.
  3. In a clustered environment, inconsistency in configuration can quickly arise bewteen servers
  4. The properties are static, and servers must be bounced for the settings to take effect.

  That does not mean that you should not use them however, just that you must be cautious and think through the consequences before you do. I personally have no problems with their use for debug flags and testing modes. One particular example I recently saw was a flag that triggered a test mode in which the datasource names changed to target a test DB.

  I do however have serious objections to environment variables that are not temporary, and which in fact are required for the application to run. At a particular client they were setting application contexts as environment parameters ?!?!? It became a serious issue when a later release of the application was developed using a slightly different value for the environment parameter. The developers insisted that the application worked on their own domain, but live domain would not run the application. After much recrimination, and accusations of incompetence the environment variable difference was found.

  I have a couple of conditions under which I would use environment vaiables to define app behaviour:

  1. The default behaviour if the property is not specified must be the correct one (i.e. live behaviour). It shoudl never be critical for app .
  2. The environment variables should be temporary only, and the JVM Parameters carefully audited for the entire period that they are active.
  3. Logging must clearly show what the environment variable value is, and contextualise what that means for the application. This is probably the only documentation this flag will ever have
  4. In a 24/7 situation, static variables in general should not be applied, and his includes environment variables. Debugging on a 24/7 live service should be done dynamicaly using the database.

  Using JMX to make changes to your application on the fly, is not a bad idea, but I think that the overhead of JMX tooling their apps will put most teams off the idea, regardless of the merits down the road.

  Posted by: t_g_nielsen on June 26, 2006 at 3:12 AM

• And how exactly is change and configuration management (important ITIL process) to be implemented with the current crop of JMX tools and servers. I think you need to make a better distinction between transient and (relatively) persistent changes that are dynamic. It would also be a nightmare to have to reapply JMX attribute changes via a JMX front end everytime a server dropped and needed to be restarted. There are a large group of system and component level settings within most applications that do not need to be changed and when changed they are initialized only once to avoid the complexity of having to deal with global/process/method state conflicts. Personally I would prefer setting system properties than having to manually apply JMX attribute values or to script the settings during an appserver startup. By the way there are no guidelines even to what is the appropriate ordering attribute updates. Some attributes have a coupling and independent transactions (not resource transactions) are needed. The reasons for limiting the dynamicism of a system or component is that in the majority of cases the increased in complexity and reduction in performance overhead cannot be justified. Of course the usual suspects (logging, profiling, tracing...) should be made dynamic, Regards William Louth JXInsight Product Architect CTO, JInspired "JEE tuning, testing, tracing and monitoring with JXInsight" http://www.jinspired.com

  Posted by: wlouth on June 27, 2006 at 5:20 AM

• reduction in performance overhead
  
  change to
  reduction in performance optimizations
  

  Posted by: wlouth on June 27, 2006 at 5:21 AM

• I think the important point made in the comments so far is to differentiate temporary, static and dynamic configuration settings and use an appropriate mechanism to manage them.

  There seems to be a some consensus on the usual suspects that should be made dynamic thats good. In my post my main gripe is precisley with this sort of functionality, where you are "switching" funcionality. I'm not suggesting configuring the finer points of transaction handling using this method.

  I don't see why using JMX will force you to reapply changes everytime a server is restarted after all WLS is configured in exactly this way, all you have to do is store the change in say, erm, a config.xml file. A simple configuration mamangement component could handle this and be used in all your applications much like a logging component. In fact I think this would make a decent codeshare project.

  Posted by: hoos on June 27, 2006 at 9:25 AM

• 2 follow ups. 1/ Agree with Trevor: There are too many text fields in the weblogic console that would be better as textarea's and 2/ Agree with hoos, a generic JMX front end for anything would be a good codeshare project. (especially if you used the AJAX google web tool kit thingy to make it into a kind of Dashboard.)

  Posted by: simonvc on June 30, 2006 at 3:38 AM