Getting Real Time
                                Value From Data

                                Eddie Satterly
                                Chief Big Data Evangelist, Office of CTO
                                esatterly@splunk.com
Copyright © 2012, Splunk Inc.                      Listen to your data.
What Does Big Data Look Like?
                          Volume | Velocity | Variety | Variability | Value


                                                                            GPS,
          Machine-generated data is one of the                             RFID,
             fastest growing, most complex                            Hypervisor,
         and most valuable segments of big data                     Web Servers,
                                                                Email, Messaging
                                                           Clickstreams, Mobile,
                                                      Telephony, IVR, Databases,
                                                  Sensors, Telematics, Storage,
                                            Servers, Security devices, Desktops




Copyright © 2012, Splunk Inc.                     4                       Listen to your data.
Big Data Ecosystem



Copyright © 2012, Splunk Inc.            Listen to your data.
Big Data Technologies
                         TeraData                 Hadoop                 Cassandra
                         Greenplum                                       CouchDB
                                                                         MongoDB




 RDBMS                      SQL &              HDFS Storage +               NoSQL                    Real Time Indexing
Sharding                 Map / Reduce           Map / Reduce
                                                                        Map / Reduce


   Relational Database                  Distributed File System       Key/Value, Columnar or    Temporal, Unstructured
   (highly structured)                     (semi-structured)          Other (semi-structured)      Heterogeneous


                                                                  6
Splunk as a Data Platform

            GPS, RFID, Hypervisor,
                                             Ad hoc       Monitor       Custom     Report and      Developer
            Web      Servers,   Email,       search       and alert   dashboards    analyze         Platform
            Messaging, Clickstreams,
            Mobile, Telephony, IVR,
                    Data
            Databases, Sensors,
            Telematics,       Storage,
            Servers, Security devices,
            Desktops, CDRs               Splunk storage




                                                                                                     Data
                                                                         Data
Copyright © 2012, Splunk Inc.                         7                                 Listen to your data.
Hadoop Connect – Read/Write HDFS

                                                 Search



         Data


                                Splunk storage




                                                          Data
Copyright © 2012, Splunk Inc.                     8              Listen to your data.
HadoopOps
        splunkHadoopOps                                                                                          splunkHadoopOps
    Forwarder Package on every host      Add      Collect & Index   Distributed   Monitor &     Rich UI            Dashboards, alerts and
                                      Knowledge        Data           Search       Alert      Framework
                                                                                                          notifications, powered by splunk search




                Host




         Operating System


           Infrastructure



Copyright © 2012, Splunk Inc.                                                                                            Listen to your data.
Cassandra Connect – Read / Write

   •   V1 will provide remote             •   Future version will add
       lookups to Cassandra                   support for other Key Value
   •   Will enable Splunk searches            stores
       to write results to                •   Private Beta now and
       Cassandra                              released to Splunkbase as
   •   Will allow Column Family               template app in Feb 2013
       creation from Splunk UI


Copyright © 2012, Splunk Inc.        10                       Listen to your data.
Sources                   Splunk - The Big Data Platform




Copyright © 2012, Splunk Inc.                11            Listen to your data.
Questions ?


                                  bigdata@splunk.com

Copyright © 2012, Splunk Inc.               Listen to your data.

Paris live eddiesatterly_022013

  • 1.
    Getting Real Time Value From Data Eddie Satterly Chief Big Data Evangelist, Office of CTO esatterly@splunk.com Copyright © 2012, Splunk Inc. Listen to your data.
  • 2.
    What Does BigData Look Like? Volume | Velocity | Variety | Variability | Value GPS, Machine-generated data is one of the RFID, fastest growing, most complex Hypervisor, and most valuable segments of big data Web Servers, Email, Messaging Clickstreams, Mobile, Telephony, IVR, Databases, Sensors, Telematics, Storage, Servers, Security devices, Desktops Copyright © 2012, Splunk Inc. 4 Listen to your data.
  • 3.
    Big Data Ecosystem Copyright© 2012, Splunk Inc. Listen to your data.
  • 4.
    Big Data Technologies TeraData Hadoop Cassandra Greenplum CouchDB MongoDB RDBMS SQL & HDFS Storage + NoSQL Real Time Indexing Sharding Map / Reduce Map / Reduce Map / Reduce Relational Database Distributed File System Key/Value, Columnar or Temporal, Unstructured (highly structured) (semi-structured) Other (semi-structured) Heterogeneous 6
  • 5.
    Splunk as aData Platform GPS, RFID, Hypervisor, Ad hoc Monitor Custom Report and Developer Web Servers, Email, search and alert dashboards analyze Platform Messaging, Clickstreams, Mobile, Telephony, IVR, Data Databases, Sensors, Telematics, Storage, Servers, Security devices, Desktops, CDRs Splunk storage Data Data Copyright © 2012, Splunk Inc. 7 Listen to your data.
  • 6.
    Hadoop Connect –Read/Write HDFS Search Data Splunk storage Data Copyright © 2012, Splunk Inc. 8 Listen to your data.
  • 7.
    HadoopOps splunkHadoopOps splunkHadoopOps Forwarder Package on every host Add Collect & Index Distributed Monitor & Rich UI Dashboards, alerts and Knowledge Data Search Alert Framework notifications, powered by splunk search Host Operating System Infrastructure Copyright © 2012, Splunk Inc. Listen to your data.
  • 8.
    Cassandra Connect –Read / Write • V1 will provide remote • Future version will add lookups to Cassandra support for other Key Value • Will enable Splunk searches stores to write results to • Private Beta now and Cassandra released to Splunkbase as • Will allow Column Family template app in Feb 2013 creation from Splunk UI Copyright © 2012, Splunk Inc. 10 Listen to your data.
  • 9.
    Sources Splunk - The Big Data Platform Copyright © 2012, Splunk Inc. 11 Listen to your data.
  • 10.
    Questions ? bigdata@splunk.com Copyright © 2012, Splunk Inc. Listen to your data.

Editor's Notes

  • #4 Further along that trend, folks typically start in any one of these solution areas and as they move up the adoption chain, they see the value of getting all the data into one place to solve problems across all of IT.Splunk can provide that single pane of glass and act as your platform for machine data.Customers start by using Splunk Enterprise to address one specific solution area. Then they leverage it and their machine data to solve other pressing problems over time.Consequently, Splunk Enterprise has many critical uses across IT and the business: Application Management: provide end-to-end visibility across distributed infrastructures; troubleshoot across application environments; monitor for performance degradation; trace transactions across distributed systems and infrastructure.Development: accelerate development and test cycles; support advanced development methodologies like agile, continuous; integrate enterprise applications with SDKs and a robust API; build enterprise applications that leverage Splunk software.Infrastructure and Operations Management: proactively monitor across IT silos to ensure uptime; rapidly pinpoint and resolve problems; report on SLAs/track SLAs of service providers.Security and Compliance: provide rapid incident response, real-time correlation and in-depth monitoring across data sources; statistical analysis for advance pattern detection and threat defense.Web and Business Analytics: gain visibility and intelligence on customers, services and transactions; identify trends and patterns in real time; fully understand the impact of new product features on back-end services.
  • #7 Highlight Splunk differentiators