MMC control for failing flows consistently

1. By Ramakrishna Narkedamilli

2. Mule Management Console ( MMC ) is a great tool to deploy Mule
applications and control. Using MMC you can perform many operations like
Deploy, Un deploy, Redeploy along with Monitoring of flows etc.
MMC provides options to monitor the failing flows and inform people about
failures. In MMC Alert mechanism you can create Alert definitions for
different types of exceptions and send mail to people using Alert
notifications.

3. Typically MMC will raise alerts for each exception it encountered on the
configured deployed applications. There is no intelligent mechanism on MMC
itself to raise the Alerts based on exception ratio ( Based on no of service calls
and no of failures among that )

4. Unfortunately there is no ‘out of the box’ MMC feature to solve this, so we have
to cater for it ourselves. Luckily Mule provides some APIs that give us good
statistical information. Using these APIs, we can write a small class that calculates
the ratio of the error messages coming in a flow compared to the messages that
were processed successfully.
To Monitoring the consistently failing flows from MMC and raising alerts on ratio
based instead of for each exception, we can add
1) Java class to calculate the ratio.
2) Spring code to call the Java class for a period of time Continuously.
3) Spring code to expose value as Mbean
4) Alert creation in MMC for the JMX value (Mbean)
Now Will see how to Implement this

5. Consider a simple web service like below

6. Now we will write Java code that calculate the failure ratio

8. The idea is to have the monitor() method execute every couple of seconds.
On each execution, we receive updated statistics of execution errors versus
completed executions. From these values, we subtract the same values
recorded in the previous run so that we get the values for just this period.
Once this calculation is complete, we divide the errors with the total
events to get a ratio of the two values. If the ratio exceeds a certain
threshold (by default set to 0.5), then the String class variable
constantlyFailingString is set to true.
Having the monitoring code ready, we need to instruct Mule to execute
this code every few seconds; 10 seconds for example . We also need to tell
ConsistentlyFailingMonitor which Flow to monitor. Spring comes in very
handy to do just that. Spring code we need add the in the simple web
service mule flow and it follows like

9. Below XML code shows the call of Java code for every 20000 milliseconds time

10. So at this stage, we have a class monitoring our flow for a stream of
consistently failing messages. This is all well and good, but we need a way to
raise an alert if our flow is failing. For this will use JMX attribute. The value of
ratio we stored in a variable that we need to expose as a Bean, SO MMC will
monitor bean and if it crossed the mentioned thresh hold value, it will raise
Alert.
To expose as Bean again we need to use Spring and the XML code follows like
below

11. To Review what ever done, it follows like
1) We created sample web service.
2) We created Java code to monitor the failures and calculate ratio.
3) Configured Spring code in Mule flow, to call the Java code continuously.
4) Configured Spring code to expose the Ration Value as bean.
After these steps, below steps need be done in MMC
5) Create alert definition for the JMX value and configure the threshold value.
Whenever the JMX value (Ration value ) crosses the threshold value, MMC will
raise Alert.
I hope this will help.

12. THANK YOU
By Ramakrishna Narkedamilli