Operation Real-Time: Analyzing Big Data Now

Like this? Share it with your network

Share

Operation Real-Time: Analyzing Big Data Now

  • 987 views
Uploaded on

The Briefing Room with Robin Bloor and Acunu ...

The Briefing Room with Robin Bloor and Acunu
Live Webcast Mar. 5, 2013

The closer you get to the point of action, the better control you have of your business. Being able to respond within seconds instead of minutes can maximize margin, minimize risk, and create opportunities that would have otherwise required guesswork. Until recently, that kind of situational awareness called for massive financial investment and serious know-how. Today, companies can tackle vast amounts of data in real-time, thanks in part to the continuing wave of open-source database innovation.

Check out the slides from this episode of The Briefing Room to hear Chief Analyst Robin Bloor, PhD, explain why real-time analysis of Big Data is possible today. He'll be briefed by Tim Moreton, CTO and Founder of Acunu, who will discuss his company's Cassandra-based Analytics offering. He'll explain how Acunu facilitates both real-time and historical analysis of Big Data, thus enabling rapid development and deployment of highly focused business applications.

Visit: http://www.insideanalysis.com

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
987
On Slideshare
785
From Embeds
202
Number of Embeds
2

Actions

Shares
Downloads
23
Comments
0
Likes
0

Embeds 202

http://rebeccajozwiak.com 102
http://insideanalysis.com 100

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Briefing Room
  • 2. Welcome Host: Eric Kavanagh eric.kavanagh@bloorgroup.comTwitter Tag: #briefr The Briefing Room
  • 3. Mission !   Reveal the essential characteristics of enterprise software, good and bad !   Provide a forum for detailed analysis of today s innovative technologies !   Give vendors a chance to explain their product to savvy analysts !   Allow audience members to pose serious questions... and get answers!Twitter Tag: #briefr The Briefing Room
  • 4. MARCH: Operational Intelligence April: INTELLIGENCE May: INTEGRATION June: DATABASETwitter Tag: #briefr The Briefing Room
  • 5. Operational Intelligence !   Real-time, dynamic business analytics !   Visibility and insight into data, streaming events and business operations !   The ability to make decisions and act quickly !   Automated alerts and/or responseTwitter Tag: #briefr The Briefing Room
  • 6. Analyst: Robin Bloor  Robin Bloor is Chief Analyst at The Bloor Group robin.bloor@bloorgroup.comTwitter Tag: #briefr The Briefing Room
  • 7. Acunu ! Acunu offers a Cassandra-based real-time analytics platform !   Its platform allows Cassandra users to build and extend business applications without being a database expert ! Acunu Analytics provides the ability to leverage customizable and re-usable analytic apps on top of its analytics layerTwitter Tag: #briefr The Briefing Room
  • 8. Tim Moreton Tim is an expert in distributed file systems. Tim was previously a senior member of the technical team at Tideway (now BMC), where he led the creation of solutions for managing data centres at Fortune 500 clients. Previously he was CEO of a consultancy delivering data solutions for the aviation sector. He holds a PhD in Computer Science from Cambridge University.Twitter Tag: #briefr The Briefing Room
  • 9. Real Time Analytics for Apache CassandraMonday, 4 March 13
  • 10. New Big Data Sources, New Big Data Applications Crowd Generated Machine Generated 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html Mobile RFID Tags Web Logs Phones 2Monday, 4 March 13
  • 11. New Big Data Sources, New Big Data Applications Crowd Generated Machine Generated 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html Mobile RFID Tags Web Logs Phones Operational Intelligence ! Dashboards Real-time Alerting Decisions 2Monday, 4 March 13
  • 12. New Big Data Sources, New Big Data Applications Crowd Generated Machine Generated 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html Mobile RFID Tags Web Logs Phones Operational Intelligence Offline Exploratory Analytics ! ? Dashboards Real-time Alerting Unstructured Machine Data Decisions Warehouses Learning Mining 2Monday, 4 March 13
  • 13. New Big Data Sources, New Big Data Applications Crowd Generated Machine Generated 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html 02:44:02 241.24.41 0.0.1 GET /index.html Mobile RFID Tags Web Logs Phones Operational Intelligence Offline Exploratory Analytics ! ? Dashboards Real-time Alerting Unstructured Machine Data Decisions Warehouses Learning Mining Query speed Query richness Data freshness, response time Complex analysis, comprehensive datasets 2Monday, 4 March 13
  • 14. Big Data Technology Timeline Relational databases ✓ Rich, fast queries ✗ Not economically scalable for volume, velocity, variety 3Monday, 4 March 13
  • 15. Big Data Technology Timeline Relational databases ✓ Rich, fast queries ✗ Not economically scalable for volume, velocity, variety BigTable, NoSQL DBs MapReduce, Hadoop ✓ Horizontally scalable ✓ Horizontally scalable ✓ Millisec query latencies ✓ Very rich queries ✗ Spartan K-V queries ✗ Jobs take mins or hours 3Monday, 4 March 13
  • 16. Big Data Technology Timeline Relational databases ✓ Rich, fast queries ✗ Not economically scalable for volume, velocity, variety BigTable, NoSQL DBs MapReduce, Hadoop ✓ Horizontally scalable ✓ Horizontally scalable ✓ Millisec query latencies ✓ Very rich queries ✗ Spartan K-V queries ✗ Jobs take mins or hours Hadoop-EDW Hybrids Impala+Trevni, Drill, Pivotal-HD ✓ Rich queries ✓ Very rich queries ✓ “API real time" ✓ “Exploratory real time” 3Monday, 4 March 13
  • 17. Apache Cassandra Ideal building block for real-time analytics • Multi-master: highly available by design • Multi-data center optimised • Very high write performance • Atomic counters • Numerous large production deployments But building analytics on a key-value interface is hard • Awkward data ingest interfaces • Spartan query semantics • Brittle data modeling leads to lack of flexibility 4Monday, 4 March 13
  • 18. Apache Cassandra Ideal building block for real-time analytics • Multi-master: highly available by design Virtual nodes CQL v2 • Multi-data center optimised • Very high write performance • Atomic counters • Numerous large production deployments But building analytics on a key-value interface is hard • Awkward data ingest interfaces • Spartan query semantics • Brittle data modeling leads to lack of flexibility 4Monday, 4 March 13
  • 19. Continuous OLAP Cubing: Fresh, Instant Answers dashboard queries programatic interface API roll-up cubes event stream Ingest Processing event store 5Monday, 4 March 13
  • 20. Acunu Analytics Simple, high-velocity Pre process event data event stream ingest ‣ Transform, filter, split streams ‣ RESTful HTTP Beautiful, modular, ‣ Integrate other data sources live BI dashboards ‣ JSON-based ‣ Apache Flume ‣ Rapidly visualize results ‣ MQ sources ‣ Rich, flexible widgets ‣ Drill-down to raw events 0101 0 1 0 1 1 0 10 0 1 0010110011010010100110011011 ‣ Create custom 10 1 10 1 1 monitoring apps 0 00 01 010 1 1 1 10 01 010 10 00 0 010 101 00 11 11 0 01011 10 01 00 10 0 10 01 10 1 01 11 10 10 01 10 10 10 01 0 10 01 01 01 010 0101 10 0100101 1010101 01011 0 1 00 01 10 1 10 100101 01010 110 10 10 Drive Applications 101 10 01101001 ‣ AQL queries, JSON results ‣ Instant results enable reactive feedback loop Aggregate queries templates, Historic context continuously evaluated Threshold Alerts ‣ Rich aggregate analytic operators ‣ Raw events and cubes stored in ‣ Comparisons vs !! ! ‣ Probabilistic functions Cassandra historic baselines !! ! ‣ Joins, limits, group bys ‣ Link aggregates to raw events 6Monday, 4 March 13
  • 21. How Acunu Analytics is Used • Analytics of Telco telemetry data • Monitoring of large-scale compute and cloud infrastructures • Continuous analytics for high-tech manufacturing production lines • Real-time financial tick data analytics • Powering social media analytics • Real-time user engagement and advertising analytics in large web • Funnel analysis, instant user journey optimization in social gaming 7Monday, 4 March 13
  • 22. How Acunu Analytics is Used • Analytics of Telco telemetry data DevOps Analytics: • Monitoring of large-scale compute • Unified real-time infrastructure, application and business metrics and cloud infrastructures • Live customer supply and driver • Continuous analytics for high-tech availability powering in-app features manufacturing production lines • Push new builds with confidence • Real-time financial tick data analytics • Faster issue resolution times • Powering social media analytics • Real-time user engagement and advertising analytics in large web “ We feed in our data and just ask questions. We get immediate results. Its resilient and very flexible • Funnel analysis, instant user journey ” and fits into our service- optimization in social gaming based architecture. 7Monday, 4 March 13
  • 23. Thank You.Apache, Apache Cassandra, Cassandra, Hadoop, and the eye and elephant logosare trademarks of the Apache Software Foundation.Monday, 4 March 13
  • 24. Perceptions & Questions Analyst: Robin BloorTwitter Tag: #briefr The Briefing Room
  • 25. The Bloor Group
  • 26. The Dawn of Operational Intelligence We have noticed an emerging trend towards building business intelligence capabilities that are deployed and used in a real-time manner. The term that has emerged to describe such capabilities is: OPERATIONAL INTELLIGENCE The Bloor Group
  • 27. Criteria It involves immediate The OI capability is fed by analysis of the (event) streamed data or current data to derive useable data at low latency intelligence OPERATIONAL INTELLIGENCE It is actioned immediately: It is integrated, to some either for automated use degree, with other BI or to inform operational capabilities and other staff “just in time” applications The Bloor Group
  • 28. OI Implementation The Bloor Group
  • 29. The Discovery Side Of The Coin The Bloor Group
  • 30. The Bottom Line They should not OI capabilities be The business are only just “architecturally” value of OI is emerging divorced from usually dramatic other BI The Bloor Group
  • 31. !   How close to actual real-time does Acunu achieve?!   Is Acunu tied entirely to Apache Cassandra or is it likely that it will be added to other data platforms?!   There are many analytics approaches and algorithms. What is the breadth of Acunu’s capability?!   How rich is the historic context? The Bloor Group
  • 32. !   In your view, is the “age of the data warehouse” over?!   Do you have a cloud offering?!   Which sectors/businesses are currently in Acunu’s “sweet spot”?!   Which companies/products do you regard as competitors/ partners? The Bloor Group
  • 33. Twitter Tag: #briefr The Briefing Room
  • 34. Upcoming TopicsApril: INTELLIGENCEMay: INTEGRATIONJune: DATABASE www.insideanalysis.comTwitter Tag: #briefr The Briefing Room
  • 35. Thank You for Your AttentionCertain images and/or photos in this presentation are the copyrighted property of 123RF Limited, their Contributors or Licensed Partners and are beingused with permission under license. These images and/or photos may not be copied or downloaded without permission from 123RF Limited.Twitter Tag: #briefr The Briefing Room