Jboss ON and Reactive Alert Handling

Red Hat has recently released version 3.0 of JBoss ON, built on the upstream RHQ Project. There are several cool features in 3.0, but more than that, the feature set integrates more than ever before. This example shows how several subsystems can come together to handle some high level tasks.

The following demo features reactive alerting, meaning, the ability to automatically take corrective action in response to an alert fired in your environment. It combines the new Drift Management feature, for detecting configuration drift, Alerting to notify that drift has been detected, JON's CLI Scripting for reacting to the alert, and Bundle provisioning for correcting the situation. Also, the new JBoss ON GUI, written in SmartGWT, is on display.

You can watch the demo here, and then take a look at some of the details below.

RHQ Demo: File remediation using alert CLI notification, drift detection and bundles

The demo showed how you can monitor your system for file drift, and alert on that drift when detected. And then how that alert can in turn invoke RHQ CLI scripts to take action. The RHQ CLI is powerful, it offers a robust API and many built-in features, in a Rhino Javascript environment. In the example we use the following script, which calls into the RHQ server and leverages the remote API for bundle provisioning. Here is the script for reference:

//
// This script performs some mock remediation on Drift detection by performing
// a clean bundle deploy of the corrupted file.
//

// note, the 'alert' variable is seeded by the alert sender

// Log what we're doing to a file tied to the fired alert id
//
var e = exporter
e.setTarget( 'raw', 'c:/temp/alert-cli-demo/logs/alert-' + alert.id + '.log' )

// Dump the alert
//
e.write( alert )

// get a proxy for the alerted-on Resource
//
var alertResource = ProxyFactory.getResource(alert.alertDefinition.resource.id)

// Dump the resource
//
e.write( " " )
e.write( alertResource )


// Remediate file

// Find the Bundle Destination
//
var destCrit = new BundleDestinationCriteria()
destCrit.addFilterName('Alert CLI Demo')
var result = BundleManager.findBundleDestinationsByCriteria( destCrit )
var dest = result.get( 0 )

// Find the Bundle Version
//
var versionCrit = new BundleVersionCriteria()
versionCrit.addFilterVersion(1)
result = BundleManager.findBundleVersionsByCriteria( versionCrit )
var ver = result.get( 0 )

// Create a new Deployment for the bundle version and the destination
//
var deployment = BundleManager.createBundleDeployment(ver.getId(), dest.getId(), 'remediate drift', new Configuration())

// Schedule a clean deploy of the deployment. This will wipe out the edited file and lay down a clean copy
//
BundleManager.scheduleBundleDeployment(deployment.getId(), true)

e.write( " " )
e.write( "REMEDIATION COMPLETE!" )

The demo neglects to show the resulting log file. The log file gives you an idea of what the alert variable data looks like. The alert data, which is provided to the script automatically, allows the script writer to access lots of other information he may need. In this example, the resource info. Here is the log file generated by the demo:

Alert:
        acknowledgeTime: -1
   acknowledgingSubject:
        alertDefinition: org.rhq.core.domain.alert.AlertDefinition[ id=10001, name=Drift!, conditionExpression=ANY, priority=Medium, resourceId=10001 ]
  alertNotificationLogs: []
          conditionLogs: [org.rhq.core.domain.alert.AlertConditionLog[ id=10191, value=Drift Detected, extraInfo=[[Fri Jan 27 17:19:53 EST 2012] Drift detected in [1] files using drift definition [Alert CLI Demo]], org.rhq.core.domain.alert.AlertCondition[ id=10041, category=Drift Detected, name=Alert CLI Demo, comparator='null', threshold=null, option=null ] ]]
                  ctime: 1327702795100
                     id: 10181
recoveryAlertDefinition:
             recoveryId: 0
            willRecover: false

ResourceClientProxy_$$_javassist_138:
                       OSName: Win32
                    OSVersion: 6.0
                 architecture: x86
                     children:
                 contentTypes: {InstalledSoftware=Installed Software}
                  createdDate: Mon Jan 23 14:11:38 EST 2012
                  description: Microsoft Windows Operating System
                   freeMemory: 805.0MB
                freeSwapSpace: 3.9GB
                      handler:
                     hostname: jshaughnessy-PC
                           id: 10001
                         idle: 0.0%
                 measurements: [Wait Load, Used Memory, System Load, Total Memory, OS Name, Free Memory, Hostname, Architecture, Idle, Total Swap Space, Used Swap Space, User Load, OS Version, Free Swap Space]
                 modifiedDate: Mon Jan 23 14:11:38 EST 2012
                         name: jshaughn
                   operations: [viewProcessList, manualAutodiscovery]
          pluginConfiguration:
pluginConfigurationDefinition: ConfigurationDefinition[id=10136, name=Windows]
                 resourceType: Windows
                   systemLoad: 0.0%
                  totalMemory: 3.0GB
               totalSwapSpace: 7.4GB
                   usedMemory: 2.2GB
                usedSwapSpace: 3.5GB
                     userLoad: --no data available--
                      version: Win32 6.0
                     waitLoad: 0.0%

REMEDIATION COMPLETE!

The above script begins to help you build your own, but JBoss ON (and RHQ) include a more powerful base script for putting together a CLI-based provisioning solution. It can be downloaded in the GUI, via Administration->Downloads.

Enjoy!