Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Splunk for JMX


Published on

Presentation section from Splunk Live content

Published in: Technology
  • Be the first to comment

Splunk for JMX

  1. 1. Splunk for JMXDamien DallimoreDeveloper Evangelist
  2. 2. Copyright©2013,SplunkInc.Splunk for JMX2• ConnecttoanylocalorremoteJVMsJMXserver,Hotspot/JRockit/IBMJ9/OpenJDK/AzulZing• QueryanyMBeanrunningonthatserver• ExtractanyMBeanattributes(simple,compositeortabular)• InvokeMBeanoperations• Writeattributesandoperationresultsoutinadefaultkey/valueformat,orpluginyourowncustomformat,forSPLUNKindexingandsearching• TransporteventsoverSTDOUT(default),TCP,Syslog,SplunkRESTendpointordirecttofile.• DeclareclustersofJVMsforlargerscaleJVMdeployments• Runson*NixandWindows• OutoftheboxdashboardsforcommonJVMMBeans• FreelyavailablefromSplunkBase,allsourcecodeisonGitHub
  3. 3. Copyright©2013,SplunkInc.Connectivity Options3• Remote JMX interface– rmi (JSR160StandardImplementationandMX4JsJSR160Implementation)– iiop(JSR160StandardImplementationandMX4JsJSR160Implementation)• Direct Process attachment– ConnectdirectlytoalocallyrunningJVMprocess• MX4J HTTP connectors (requires MX4J in the target JVM also)– soap,soap+sssl– hessian,hessian+ssl– burlap,burlap+ssl
  4. 4. Copyright©2013,SplunkInc.Setup and Configuration4• The main goalof the app was to make it as simple and intuitive as possible to connect toyour JVMs and start Splunking JMX data• Enableyour targetJVM’s remoteJMX interface, test connectivitywith JConsole• InstallSplunk for JMX• ExtractSplunkforJMXtarballtoSPLUNK_HOME/etc/apps• RestartSplunk• Atthesetupscreen,choosea scriptedinputforyourplatform(Nix/Windows)• Setup your JMX configurationfile• Thedefaultconfig.xmlfileispreconfiguredforcommonJVMMBeans• BrowseyourJVM(usingJConsole)forotherMBeansthatyouwishtopollandconfigurethese• Youcanhaveasmanyconfigfilesasyourequire,andyoumightsettheseuptofireoffatdifferentscheduledfrequencies
  5. 5. Copyright©2013,SplunkInc.Configuration Examples - Simple5
  6. 6. Copyright©2013,SplunkInc.Configuration Examples - Wildcards6MBean Object name format “domain:key=value,key2=value2”Around 25KBytes per dump on Hotspot JVMsDon’t know the domain or properties name or have many Mbeanswith the same attributes ?* and ? wildcards are supported in the Mbean name
  7. 7. Copyright©2013,SplunkInc.Configuration Examples - Clusters7• Define clusters of JVM’s that share thesame MBean definitions• Note , in these examples, for brevity I amusing “dumpAllAttributes” , but inproduction you’d want to pick andchoose specific MBean attributes youare interested in, and perhaps splitdefinitions over multiple files run atvarying frequencies
  8. 8. Copyright©2013,SplunkInc.Configuration Examples - Operations8• Invoke JMX operations that return a value or simplyperform some action on the target JVM• Operation definitions can take parametersUse Case 1 : your developers might code a JMXoperation that returns a CSV or JSON formattedsnapshot of some metrics for SplunkingUse Case 2 : dynamically trigger HPROF dumps.The “”Mbean exposes a “dumpHeap” operation
  9. 9. Copyright©2013,SplunkInc.Configuration Examples - Connecting9• IP Address with credentials• Hostname• Static Process ID• Process ID lookup from file• Process ID lookup from command output• Raw JMX Service URL• MX4J HTTP Connector
  10. 10. Copyright©2013,SplunkInc.Custom Formatters/Transports10• The Splunk for JMX configuration is user extensible• You can code and configure your own Formatters and TransportsFormatters• Takes the raw MBean polled output and formats it for Splunking• A Java implementation of the "com.dtdsoftware.splunk.formatter.Formatter" interface• If the optional formatter declaration is omitted, then the default formatter will be usedTransports• Takes the formatted output and transports it to a destination• A Java implementation of the "com.dtdsoftware.splunk.transport.Transport" interface• If the optional transport declaration is omitted, then the default transport(STD out) will be used
  11. 11. Copyright©2013,SplunkInc.Formatter Examples11
  12. 12. Copyright©2013,SplunkInc.Transport Examples12
  13. 13. Copyright©2013,SplunkInc.Deployment Architectures 113• Simplest scenario• Monolithic Splunk installation• Splunk for JMX polling 1 or more remote/local JVMs viathe remote JMX interface• There is support for many target JVM’s in theconfiguration schema but to really scale out, you needa more advanced Splunk architecture
  14. 14. Copyright©2013,SplunkInc.Deployment Architectures 214Splunk UF running locallywith target JVMSplunk IndexerClusterSplunk Search HeadPoolLoad Balancer• Run Splunk UF locally with targetJVM.Can connect use remote JMXinterface or direct process attachment.• Each tier scales out horizontally.• Can overcome firewall issues that aresometimes inherent with Java RMI• Deploy Splunk for JMX components andconfigurations with Splunk DeploymentServer, Puppet or Chef.
  15. 15. Demo
  16. 16. Copyright©2013,SplunkInc.Contact me16Email : ddallimore@splunk.comTwitter : @damiendallimoreSkype : damien.dallimoreGithub : damiendallimoreSplunkbase : damiendSlideshare : : :