Submit Search
Upload
Big Data: Introducing BigInsights, IBM's Hadoop- and Spark-based analytical platform
•
48 likes
•
35,654 views
Cynthia Saracco
Follow
Technical introduction to IBM's BigInsights platform for managing and analyzing Big Data.
Read less
Read more
Technology
Report
Share
Report
Share
1 of 49
Download now
Download to read offline
Recommended
introduction to data mining tutorial
introduction to data mining tutorial
Salah Amean
Machine Learning
Machine Learning
Rahul Kumar
Machine learning
Machine learning
Rajesh Chittampally
Machine Learning
Machine Learning
Kumar P
Lecture1 introduction to machine learning
Lecture1 introduction to machine learning
UmmeSalmaM1
Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
Lukas Masuch
Machine learning ppt.
Machine learning ppt.
ASHOK KUMAR
Lecture6 introduction to data streams
Lecture6 introduction to data streams
hktripathy
Recommended
introduction to data mining tutorial
introduction to data mining tutorial
Salah Amean
Machine Learning
Machine Learning
Rahul Kumar
Machine learning
Machine learning
Rajesh Chittampally
Machine Learning
Machine Learning
Kumar P
Lecture1 introduction to machine learning
Lecture1 introduction to machine learning
UmmeSalmaM1
Deep Learning - The Past, Present and Future of Artificial Intelligence
Deep Learning - The Past, Present and Future of Artificial Intelligence
Lukas Masuch
Machine learning ppt.
Machine learning ppt.
ASHOK KUMAR
Lecture6 introduction to data streams
Lecture6 introduction to data streams
hktripathy
Big data ppt
Big data ppt
Thirunavukkarasu Ps
Machine learning
Machine learning
Navdeep Asteya
Machine Learning and Real-World Applications
Machine Learning and Real-World Applications
MachinePulse
Big data mining
Big data mining
Thadsanamoorthy Kajavathanan
Computer vision
Computer vision
Mahmoud Hussein
Big Data Analytics
Big Data Analytics
Ghulam Imaduddin
Machine Learning
Machine Learning
Vivek Garg
An introduction to Machine Learning
An introduction to Machine Learning
butest
Big data Presentation
Big data Presentation
Aswadmehar
Introduction to Deep learning
Introduction to Deep learning
Massimiliano Ruocco
Machine learning vs deep learning
Machine learning vs deep learning
USM Systems
The 7 steps of Machine Learning
The 7 steps of Machine Learning
Waziri Shebogholo
Machine Learning Tutorial | Machine Learning Basics | Machine Learning Algori...
Machine Learning Tutorial | Machine Learning Basics | Machine Learning Algori...
Simplilearn
Lecture1 introduction to big data
Lecture1 introduction to big data
hktripathy
Introducing Deep learning with Matlab
Introducing Deep learning with Matlab
Massimo Talia
General introduction to AI ML DL DS
General introduction to AI ML DL DS
Roopesh Kohad
Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
Mohammad Junaid Khan
Big data
Big data
Pooja Shah
Traffic Prediction for Intelligent Transportation System using Machine Learning
Traffic Prediction for Intelligent Transportation System using Machine Learning
OmSuryawanshi9
MOBILE BI
MOBILE BI
Raminder Pal Singh
Overview - IBM Big Data Platform
Overview - IBM Big Data Platform
Vikas Manoria
NoSQL overview implementation free
NoSQL overview implementation free
Benoit Perroud
More Related Content
What's hot
Big data ppt
Big data ppt
Thirunavukkarasu Ps
Machine learning
Machine learning
Navdeep Asteya
Machine Learning and Real-World Applications
Machine Learning and Real-World Applications
MachinePulse
Big data mining
Big data mining
Thadsanamoorthy Kajavathanan
Computer vision
Computer vision
Mahmoud Hussein
Big Data Analytics
Big Data Analytics
Ghulam Imaduddin
Machine Learning
Machine Learning
Vivek Garg
An introduction to Machine Learning
An introduction to Machine Learning
butest
Big data Presentation
Big data Presentation
Aswadmehar
Introduction to Deep learning
Introduction to Deep learning
Massimiliano Ruocco
Machine learning vs deep learning
Machine learning vs deep learning
USM Systems
The 7 steps of Machine Learning
The 7 steps of Machine Learning
Waziri Shebogholo
Machine Learning Tutorial | Machine Learning Basics | Machine Learning Algori...
Machine Learning Tutorial | Machine Learning Basics | Machine Learning Algori...
Simplilearn
Lecture1 introduction to big data
Lecture1 introduction to big data
hktripathy
Introducing Deep learning with Matlab
Introducing Deep learning with Matlab
Massimo Talia
General introduction to AI ML DL DS
General introduction to AI ML DL DS
Roopesh Kohad
Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
Mohammad Junaid Khan
Big data
Big data
Pooja Shah
Traffic Prediction for Intelligent Transportation System using Machine Learning
Traffic Prediction for Intelligent Transportation System using Machine Learning
OmSuryawanshi9
MOBILE BI
MOBILE BI
Raminder Pal Singh
What's hot
(20)
Big data ppt
Big data ppt
Machine learning
Machine learning
Machine Learning and Real-World Applications
Machine Learning and Real-World Applications
Big data mining
Big data mining
Computer vision
Computer vision
Big Data Analytics
Big Data Analytics
Machine Learning
Machine Learning
An introduction to Machine Learning
An introduction to Machine Learning
Big data Presentation
Big data Presentation
Introduction to Deep learning
Introduction to Deep learning
Machine learning vs deep learning
Machine learning vs deep learning
The 7 steps of Machine Learning
The 7 steps of Machine Learning
Machine Learning Tutorial | Machine Learning Basics | Machine Learning Algori...
Machine Learning Tutorial | Machine Learning Basics | Machine Learning Algori...
Lecture1 introduction to big data
Lecture1 introduction to big data
Introducing Deep learning with Matlab
Introducing Deep learning with Matlab
General introduction to AI ML DL DS
General introduction to AI ML DL DS
Support Vector Machines ( SVM )
Support Vector Machines ( SVM )
Big data
Big data
Traffic Prediction for Intelligent Transportation System using Machine Learning
Traffic Prediction for Intelligent Transportation System using Machine Learning
MOBILE BI
MOBILE BI
Viewers also liked
Overview - IBM Big Data Platform
Overview - IBM Big Data Platform
Vikas Manoria
NoSQL overview implementation free
NoSQL overview implementation free
Benoit Perroud
IBM WATSON
IBM WATSON
suchitralotus
IBM Watson
IBM Watson
Harshdeep Singh
IBM Watson Analytics Presentation
IBM Watson Analytics Presentation
Ian Balina
Ibm's watson
Ibm's watson
Hdavey01
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
Tony Pearson
Ml, AI and IBM Watson - 101 for Business
Ml, AI and IBM Watson - 101 for Business
Jouko Poutanen
Big Data & Analytics Architecture
Big Data & Analytics Architecture
Arvind Sathi
Bring IBM Watson to your telephone
Bring IBM Watson to your telephone
Brian Pulito
IBM Watson Overview
IBM Watson Overview
Penn State EdTech Network
Viewers also liked
(11)
Overview - IBM Big Data Platform
Overview - IBM Big Data Platform
NoSQL overview implementation free
NoSQL overview implementation free
IBM WATSON
IBM WATSON
IBM Watson
IBM Watson
IBM Watson Analytics Presentation
IBM Watson Analytics Presentation
Ibm's watson
Ibm's watson
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
IBM Watson: How it Works, and What it means for Society beyond winning Jeopardy!
Ml, AI and IBM Watson - 101 for Business
Ml, AI and IBM Watson - 101 for Business
Big Data & Analytics Architecture
Big Data & Analytics Architecture
Bring IBM Watson to your telephone
Bring IBM Watson to your telephone
IBM Watson Overview
IBM Watson Overview
Similar to Big Data: Introducing BigInsights, IBM's Hadoop- and Spark-based analytical platform
IBM Smarter Analytics
IBM Smarter Analytics
Adrian Turcu
Get Started Quickly with IBM's Hadoop as a Service
Get Started Quickly with IBM's Hadoop as a Service
IBM Cloud Data Services
Making Hadoop Ready for the Enterprise
Making Hadoop Ready for the Enterprise
DataWorks Summit
IBM CDS Overview
IBM CDS Overview
Jean Tan
OC Big Data Monthly Meetup #6 - Session 1 - IBM
OC Big Data Monthly Meetup #6 - Session 1 - IBM
Big Data Joe™ Rossi
SD Big Data Monthly Meetup #4 - Session 1 - IBM
SD Big Data Monthly Meetup #4 - Session 1 - IBM
Big Data Joe™ Rossi
Analyzing Big Data - Jeff Scheel
Analyzing Big Data - Jeff Scheel
Kangaroot
Big Data: InterConnect 2016 Session on Getting Started with Big Data Analytics
Big Data: InterConnect 2016 Session on Getting Started with Big Data Analytics
Cynthia Saracco
Iotbds v1.0
Iotbds v1.0
Roy Cecil
The sensor data challenge - Innovations (not only) for the Internet of Things
The sensor data challenge - Innovations (not only) for the Internet of Things
Stephan Reimann
IBM Industry Models and Data Lake
IBM Industry Models and Data Lake
Pat O'Sullivan
Has Your Data Gone Rogue?
Has Your Data Gone Rogue?
Tony Pearson
IBM Cloud Storage - Cleversafe
IBM Cloud Storage - Cleversafe
Michael Beatty
Using real time big data analytics for competitive advantage
Using real time big data analytics for competitive advantage
Amazon Web Services
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
Amazon Web Services
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
Amazon Web Services
Getting started with Hadoop on the Cloud with Bluemix
Getting started with Hadoop on the Cloud with Bluemix
Nicolas Morales
Insights into Real World Data Management Challenges
Insights into Real World Data Management Challenges
DataWorks Summit
IBM Software Day 2013. Smarter analytics and big data. building the next gene...
IBM Software Day 2013. Smarter analytics and big data. building the next gene...
IBM (Middle East and Africa)
Big and fast data strategy 2017 jr
Big and fast data strategy 2017 jr
Jonathan Raspaud
Similar to Big Data: Introducing BigInsights, IBM's Hadoop- and Spark-based analytical platform
(20)
IBM Smarter Analytics
IBM Smarter Analytics
Get Started Quickly with IBM's Hadoop as a Service
Get Started Quickly with IBM's Hadoop as a Service
Making Hadoop Ready for the Enterprise
Making Hadoop Ready for the Enterprise
IBM CDS Overview
IBM CDS Overview
OC Big Data Monthly Meetup #6 - Session 1 - IBM
OC Big Data Monthly Meetup #6 - Session 1 - IBM
SD Big Data Monthly Meetup #4 - Session 1 - IBM
SD Big Data Monthly Meetup #4 - Session 1 - IBM
Analyzing Big Data - Jeff Scheel
Analyzing Big Data - Jeff Scheel
Big Data: InterConnect 2016 Session on Getting Started with Big Data Analytics
Big Data: InterConnect 2016 Session on Getting Started with Big Data Analytics
Iotbds v1.0
Iotbds v1.0
The sensor data challenge - Innovations (not only) for the Internet of Things
The sensor data challenge - Innovations (not only) for the Internet of Things
IBM Industry Models and Data Lake
IBM Industry Models and Data Lake
Has Your Data Gone Rogue?
Has Your Data Gone Rogue?
IBM Cloud Storage - Cleversafe
IBM Cloud Storage - Cleversafe
Using real time big data analytics for competitive advantage
Using real time big data analytics for competitive advantage
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
TiVo: How to Scale New Products with a Data Lake on AWS and Qubole
Getting started with Hadoop on the Cloud with Bluemix
Getting started with Hadoop on the Cloud with Bluemix
Insights into Real World Data Management Challenges
Insights into Real World Data Management Challenges
IBM Software Day 2013. Smarter analytics and big data. building the next gene...
IBM Software Day 2013. Smarter analytics and big data. building the next gene...
Big and fast data strategy 2017 jr
Big and fast data strategy 2017 jr
More from Cynthia Saracco
Using your DB2 SQL Skills with Hadoop and Spark
Using your DB2 SQL Skills with Hadoop and Spark
Cynthia Saracco
Big Data: Getting off to a fast start with Big SQL (World of Watson 2016 sess...
Big Data: Getting off to a fast start with Big SQL (World of Watson 2016 sess...
Cynthia Saracco
Big Data: SQL query federation for Hadoop and RDBMS data
Big Data: SQL query federation for Hadoop and RDBMS data
Cynthia Saracco
Big Data: Querying complex JSON data with BigInsights and Hadoop
Big Data: Querying complex JSON data with BigInsights and Hadoop
Cynthia Saracco
Big Data: Using free Bluemix Analytics Exchange Data with Big SQL
Big Data: Using free Bluemix Analytics Exchange Data with Big SQL
Cynthia Saracco
Big Data: Big SQL web tooling (Data Server Manager) self-study lab
Big Data: Big SQL web tooling (Data Server Manager) self-study lab
Cynthia Saracco
Big Data: Working with Big SQL data from Spark
Big Data: Working with Big SQL data from Spark
Cynthia Saracco
Big Data: HBase and Big SQL self-study lab
Big Data: HBase and Big SQL self-study lab
Cynthia Saracco
Big Data: Getting started with Big SQL self-study guide
Big Data: Getting started with Big SQL self-study guide
Cynthia Saracco
Big Data: Big SQL and HBase
Big Data: Big SQL and HBase
Cynthia Saracco
Big Data: SQL on Hadoop from IBM
Big Data: SQL on Hadoop from IBM
Cynthia Saracco
Big Data: Explore Hadoop and BigInsights self-study lab
Big Data: Explore Hadoop and BigInsights self-study lab
Cynthia Saracco
Big Data: Get started with SQL on Hadoop self-study lab
Big Data: Get started with SQL on Hadoop self-study lab
Cynthia Saracco
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
Cynthia Saracco
More from Cynthia Saracco
(14)
Using your DB2 SQL Skills with Hadoop and Spark
Using your DB2 SQL Skills with Hadoop and Spark
Big Data: Getting off to a fast start with Big SQL (World of Watson 2016 sess...
Big Data: Getting off to a fast start with Big SQL (World of Watson 2016 sess...
Big Data: SQL query federation for Hadoop and RDBMS data
Big Data: SQL query federation for Hadoop and RDBMS data
Big Data: Querying complex JSON data with BigInsights and Hadoop
Big Data: Querying complex JSON data with BigInsights and Hadoop
Big Data: Using free Bluemix Analytics Exchange Data with Big SQL
Big Data: Using free Bluemix Analytics Exchange Data with Big SQL
Big Data: Big SQL web tooling (Data Server Manager) self-study lab
Big Data: Big SQL web tooling (Data Server Manager) self-study lab
Big Data: Working with Big SQL data from Spark
Big Data: Working with Big SQL data from Spark
Big Data: HBase and Big SQL self-study lab
Big Data: HBase and Big SQL self-study lab
Big Data: Getting started with Big SQL self-study guide
Big Data: Getting started with Big SQL self-study guide
Big Data: Big SQL and HBase
Big Data: Big SQL and HBase
Big Data: SQL on Hadoop from IBM
Big Data: SQL on Hadoop from IBM
Big Data: Explore Hadoop and BigInsights self-study lab
Big Data: Explore Hadoop and BigInsights self-study lab
Big Data: Get started with SQL on Hadoop self-study lab
Big Data: Get started with SQL on Hadoop self-study lab
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
Big Data: Technical Introduction to BigSheets for InfoSphere BigInsights
Recently uploaded
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
shyamraj55
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
Alex Barbosa Coqueiro
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
Kalema Edgar
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
Addepto
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
Mattias Andersson
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Pixlogix Infotech
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
2toLead Limited
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
Fwdays
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
BookNet Canada
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
Rizwan Syed
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
null - The Open Security Community
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
Commit University
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
Dubai Multi Commodity Centre
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
The Digital Insurer
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
comworks
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
Fwdays
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
UiPathCommunity
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
Sergiu Bodiu
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
Fwdays
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
Fwdays
Recently uploaded
(20)
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
"Federated learning: out of reach no matter how close",Oleksandr Lapshyn
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
Big Data: Introducing BigInsights, IBM's Hadoop- and Spark-based analytical platform
1.
© 2016 IBM
Corporation IBM BigInsights: Bringing you big value from Big Data Created by C. M. Saracco, IBM Silicon Valley Lab June 2016
2.
© 2016 IBM
Corporation2 IBM Disclaimer Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.
3.
© 2016 IBM
Corporation3 Agenda The big picture about Big Data IBM’s approach Portfolio overview BigInsights • Open source core platform with Apache Hadoop • IBM technologies for enhanced analytics • How BigInsights fits within a broader IT infrastructure How IBM can help you get off to a quick start
4.
© 2016 IBM
Corporation The Big Picture about Big Data
5.
© 2016 IBM
Corporation5 Business leaders frequently make decisions based on information they don’t trust, or don’t have1in3 83% of CIOs cited “Business intelligence and analytics” as part of their visionary plans to enhance competitiveness Business leaders say they don’t have access to the information they need to do their jobs 1in2 of CEOs need to do a better job capturing and understanding information rapidly in order to make swift business decisions 60% … and organizations need deeper insights Information is at the center of a new wave of opportunity… 2.5 million items per minute 300,000 tweets per minute 200 million emails per minute 220,000 photos per minute 5 TB per flight > 1 PB per day gas turbines 1 ZB = 1 billion TB
6.
© 2016 IBM
Corporation6 Extract insight from a high volume, variety and velocity of data in a timely and cost-effective manner Big Data presents big opportunities Manage and benefit from diverse data types and data structures Analyze streaming data and large volumes of persistent data Scale from terabytes to zettabytes Variety: Velocity: Volume:
7.
© 2016 IBM
Corporation7 What we hear from customers . . . . Lots of potentially valuable data is dormant or discarded due to size/performance issues Large volume of unstructured or semi-structured data is not worth integrating fully (e.g. Tweets, logs, . . .) Not clear what should be analyzed (exploratory, iterative) Information distributed across multiple systems and/or Internet Some data has a short useful lifespan Volumes can be extremely high Query-ready resource for “cold” historic data needed (prevent unwieldy growth of data warehouses) Analysis needed in the context of existing information (not stand alone).
8.
© 2016 IBM
Corporation8 Merging the traditional and Big Data approaches IT Structures the data to answer that question IT Delivers a platform to enable creative discovery Business Explores what questions could be asked Business Users Determine what question to ask Monthly sales reports Profitability analysis Customer surveys Brand sentiment Product strategy Maximum asset utilization Big Data Approach Iterative & Exploratory Traditional Approach Structured & Repeatable
9.
© 2016 IBM
Corporation9 Why invest in analytics? Analytics pay back $13.01 for every dollar spent1 69% created significant positive impact on business outcomes2 60% created significant positive impact on revenues2 53% created significant competitive advantage2 1 “Analytics Pays Back $13.01 for Every Dollar Spent” Nucleus Research, September 2014 2 “Analytics: The speed advantage” IBM Institute for Business Value, 2014
10.
© 2016 IBM
Corporation10 Big Data scenarios span many industries Identify criminals and threats from disparate video, audio, and data feeds Make risk decisions based on real-time transactional data Predict weather patterns to plan optimal wind turbine usage, and optimize capital expenditure on asset placement Detect life-threatening conditions at hospitals in time to intervene Multi-channel customer sentiment and experience a analysis
11.
© 2016 IBM
Corporation11 Landing and Archive Zone Real-time Analytics Zone Enterprise Warehouse and Mart Zone Information Governance, Security and Business Continuity Analytic Appliances Big Data Platform Capabilities Streaming Data Text Data Applications Data Time Series Geo Spatial Relational • Information Ingest • Real Time Analytics • Warehouse & Data Marts • Analytic Appliances Social Network Video & Image All Data Sources Advanced Analytics / New Insights New / Enhanced Applications Automated Process Case Management Analytic Applications Cognitive Learn Dynamically? Prescriptive Best Outcomes? Predictive What Could Happen? Descriptive What Has Happened? Exploration and Discovery What Do You Have? Watson Cloud Services ISV Solutions Alerts IBM Big Data and analytics sample architecture Ingestion and Operational Information
12.
© 2016 IBM
Corporation12 Big Data use expanding rapidly Big data adoption over time, as reported by respondents: 2012 to 2014 2015 22%-27% 25% 0% change 2012 to 2014 2015 24%-26% 10% 250% decrease Educate: Learning about big data capabilities 2012 to 2014 2015 43%-47% 53% 125% increase Explore: Exploring internal use cases and developing a strategy Engage: Implementing infrastructure and running pilot activities 2012 to 2014 2015 5%-6% 13% 210% increase Execute: Using big data and analytics pervasively across the enterprise 2015 IBV study “Analytics: The Upside of Disruption” (ibm.biz/w3_2015analytics)
13.
© 2016 IBM
Corporation13 Big Data technologies pay off 2015 IBV study “Analytics: The Upside of Disruption” (ibm.biz/w3_2015analytics)
14.
© 2016 IBM
Corporation14 Return on investment period for big data and analytics projects as reported by respondents Big Data ROI often < 18 months 2015 IBV study “Analytics: The Upside of Disruption” (ibm.biz/w3_2015analytics)
15.
© 2016 IBM
Corporation15 Big Data in practice: focus areas Survey summaries from Forbes, May 2015
16.
© 2016 IBM
Corporation IBM’s approach
17.
© 2016 IBM
Corporation17 IBM analytics platform strategy for Big Data • Integrate and manage the full variety, velocity and volume of Big Data • Apply advanced analytics • Visualize all available data for ad-hoc analysis • Support workload optimization and scheduling • Provide for security and governance • Integrate with enterprise software Discovery & Exploration Prescriptive Analytics Predictive Analytics Content Analytics Business Intelligence Data Mgmt Hadoop & NoSQL Content Mgmt Data Warehouse Information Integration & Governance IBM ANALYTICS PLATFORM Built on Spark. Hybrid. Trusted. Spark Analytics Operating System Machine LearningOn premises On cloud Data at rest & In-motion. Inside & outside the firewall. Structured & unstructured.
18.
© 2016 IBM
Corporation18 IBM BigInsights for Apache Hadoop and Spark Discovery & Exploration Prescriptive Analytics Predictive Analytics Content Analytics Business Intelligence Data Mgmt Hadoop & NoSQL Content Mgmt Data Warehouse Information Integration & Governance IBM ANALYTICS PLATFORM Built on Spark. Hybrid. Trusted. Spark Analytics Operating System Machine LearningOn premises On cloud Data at rest & In-motion. Inside & outside the firewall. Structured & unstructured. Analytical platform for persistent Big Data – 100% open source core with IBM add-ons for analysts, data scientists, and admins – On premise or cloud Distinguishing characteristics – Built-in analytics . . . . Enhances business knowledge – Enterprise software integration . . . . Complements and extends existing capabilities – Production-ready . . . . Speeds time-to-value IBM advantage – Combination of software, hardware, services and research
19.
© 2016 IBM
Corporation19 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cognos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
20.
© 2016 IBM
Corporation20 BigInsights ISV Partner Ecosystem lHelium SW
21.
© 2016 IBM
Corporation A Closer Look at IBM BigInsights . . . .
22.
© 2016 IBM
Corporation22 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cognos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
23.
© 2016 IBM
Corporation23 IBM Open Platform foundational components Apache Hadoop Distributed file system, popular API (MapReduce) for clustered computing Originally designed for batch processing of massive data volumes, varied data formats Apache Spark General purpose, high-speed data processing engine for clustered computing In-memory processing, popular built-in libraries (e.g., machine learning) No built-in storage. Attaches to other data stores (e.g., Hadoop Distributed File System)
24.
© 2016 IBM
Corporation24 IBM Open Platform: a closer look Timely updates as new open source versions released Install only those components you want / need Compliant with ODPi runtime Ambari 2.2 Flume 1.6.0 Hadoop (includes MapReduce, YARN) 2.7.2 HBase 1.2.0 Hive 1.2.1 Kafka 0.9.0.1 Knox 0.7.0 Oozie 4.2.0 Parquet 2.2 Phoenix 4.6.1 Pig 0.15.0 Ranger 0.5.2 Slider 0.90.2 Solr 5.5 Spark 1.6.1 Sqoop 1.4.6 Titan 1.0.0 ZooKeeper 3.4.6
25.
© 2016 IBM
Corporation25 What is ODPi? • ODPi has an open governance model. Developers form a Technical Steering Committee • All members have an equal vote on ODPi Core decisions. • ODPi has a Board of Directors responsible for the financial, legal and promotional aspects of ODPi. • Non-profit organization accelerating the delivery of Big Data solutions by powering a platform called ODPi Core. • The ODPi Core focuses on a small but critical set of projects • Goal: enables a rapid start and an industry driven definition ODPi Members include: Ampool, Altiscale, ArenaData, AsiaInfo, Capgemini, DataTorrent, EMC, GE, Hortonworks, IBM, Infosys, NEC, Pivotal, PLDT, SAS, Squid Solutions, SyncSort, Telstra, Toshiba, UNIFi, VMware, WANdisco, Xiilab, zData and Zettaset. ODPi & Apache Software Foundation (ASF) ODPi supports the ASF mission ASF provides governance around individual projects without looking at ecosystem and collections of projects ODPi provides a vendor-led consistent packaging model and certification for Big Data components as an ecosystem - Test once ; Run anywhere for big data applications
26.
© 2016 IBM
Corporation26 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cognos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
27.
© 2016 IBM
Corporation27 SQL for Hadoop (Big SQL) SQL-based Application Big SQL Engine Data Storage IBM data server client SQL MPP Run-time DFS 27 Comprehensive, standard SQL – SELECT: joins, unions, aggregates, subqueries . . . – GRANT/REVOKE, INSERT … INTO – UPDATE / DELETE (HBase) – Procedural logic in SQL – Stored procs, user-defined functions – IBM data server JDBC and ODBC drivers Optimization and performance – IBM MPP engine (C++) replaces Java MapReduce layer – Continuous running daemons (no start up latency) – Message passing allow data to flow between nodes without persisting intermediate results – In-memory operations with ability to spill to disk (useful for aggregations, sorts that exceed available RAM) – Cost-based query optimization with 140+ rewrite rules Various storage formats supported – Data persisted in DFS, Hive, HBase – No IBM proprietary format required Integration with RDBMSs via LOAD, query federation BigInsights
28.
© 2016 IBM
Corporation28 Big SQL query federation = virtualized data access Transparent Appears to be one source Programmers don’t need to know how / where data is stored Heterogeneous Accesses data from diverse sources High Function Full query support against all data Capabilities of sources as well Autonomous Non-disruptive to data sources, existing applications, systems. High Performance Optimization of distributed queries SQL tools, applications Data sources Virtualized data
29.
© 2016 IBM
Corporation29 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cognos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
30.
© 2016 IBM
Corporation30 Text analytics Distills structured info from unstructured text Sentiment analysis Consumer behavior Illegal or suspicious activities … Parses text and detects meaning with annotators Understands the context in which the text is analyzed Features pre-built extractors for names, addresses, phone numbers, etc. I had an iphone, but it's dead @JoaoVianaa. (I've no idea where it's) !Want a Galaxy now !!! @rakonturmiami im moving to miami in 3 months. i look foward to the new lifestyle I'm at Mickey's Irish Pub Downtown (206 3rd St, Court Ave, Des Moines) w/ 2 others http://4sq.com/gbsaYR
31.
© 2016 IBM
Corporation31 Extracting information from text Entity Analytics Preventative Maintenance Customer Segmentation Sentiment Affinity … Analyze Text Single column or document • sentence segmentation • tokenization • part-of-speech tagging • language detection Recognize Entity Recognition Machine Data Primitives Sentiment … Describe via extractors Information Extraction (IE) Tagged syntax Classified words / attributes Classified words / attributes Text preparation • extraction operations via lexical analysis via deep linguistic analysis • span operations • join operations • consolidations • … … • verb-centric abstraction • noun-centric abstraction • shallow parsing • …
32.
© 2016 IBM
Corporation32 Web-based tool to define rules to extract data and derive information from unstructured text Graphical interface to describe structure of various textual formats – from log file data to natural language Text analytics tooling
33.
© 2016 IBM
Corporation33 Pre-built text extractors The extractor library contains a rich set of pre-built extractors Finance actions Named Entities Generic Machine Data Sentiment Analysis You can control output properties Output columns and names Row filters Some pre-built extractors can be customized Add / remove dictionary terms
34.
© 2016 IBM
Corporation34 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cognos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
35.
© 2016 IBM
Corporation35 Spreadsheet-style analysis (BigSheets) Web-based analysis and visualization Spreadsheet-like interface Explore, manipulate data without writing code Invoke pre-built functions Generate charts Export results of analysis Create custom plug-ins . . .
36.
© 2016 IBM
Corporation36 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cognos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
37.
© 2016 IBM
Corporation37 What is Big R? R Clients Scalable Statistic s Engine Data Sources Embedded R Execution R Packages R Packages 1 2 3 1. Explore, visualize, transform, and model big data using familiar R syntax and paradigm (no MapReduce code) 2. Scale out R • Partitioning of large data (“divide”) • Parallel cluster execution of pushed down R code (“conquer”) • All of this from within the R environment (Jaql, Map/Reduce are hidden from you • Almost any R package can run in this environment 3. Scalable machine learning • A scalable statistics engine that provides canned algorithms, and an ability to author new ones, all via R “End-to-end integration of R-Project with BigInsights” Pull data (summaries) to R client Or, push R functions right on the data
38.
© 2016 IBM
Corporation38 IBM Open Platform 100% open source platform compliant with ODPi Apache Hadoop ecosystem Apache Spark ecosystem IBM-specific BigInsights features Big SQL (industry standard SQL) Text analytics BigSheets (spreadsheet-style tool) Big R (R support) IBM Streams, Cogmos (limited use licenses) Overview of BigInsights Free Quick Start (non production): • IBM Open Platform • IBM added value features • Community support
39.
© 2016 IBM
Corporation39 Limited use license: IBM Streams Millions of events per second Microsecond Latency Sensor, video, audio, text, Hadoop and relational data sources Just-in-time decisions Powerful analytics Persist to BigInsights, … Platform for real-time Big Data analytics “Data in motion” Gigabytes+ per second or more Terabyte+ per day All kinds of data Insights in microseconds Connectivity to varied data sources
40.
© 2016 IBM
Corporation40 Limited use license: Cognos BI Model, explore, analyze data from many sources Visualize and report on results Connection to BigInsights via Big SQL In-memory dynamic views cache data in Cognos for quick data access Part of IBM BigInsights for Apache Hadoop Demo: https://www.youtube.com/watch?v=yxnoGrK6PSY
41.
© 2016 IBM
Corporation41 Thinking cloud? Think IBM! BETTER ECONOMICS LOWER RISK OF FAILURE FASTER INNOVATION Lower Skill Less Cost+ Buy only what you need. Start small and grow. EQUALS
42.
© 2016 IBM
Corporation42 Build Ready-to-run Hadoop clusters in the cloud IBM Open Platform - 100% open source Hadoop; will align with ODP Based on proven, performant reference architectures Manage Key platform components monitored for availability Hadoop, OS and BigInsights patched and maintained Ambari cluster manager for complete control Support 24x7 cloud operations and support team Access to deep Hadoop expertise Faster time to problem resolution Protect Deployed in world- class, secure SoftLayer data centers Dedicated physical machines Certified SSAE SOC2 Type 1, ISO 27001 IBM BigInsights on cloud http://www.ibm.com/cloud http://www.bluemix.net
43.
© 2016 IBM
Corporation Summary and Fast Start
44.
© 2016 IBM
Corporation44 IBM investing heavily in Big Data and analytics $24B Investment in both organic development and 30+ acquisitions $100M Announced investment in IBM Interactive Experience, creating 10 new labs worldwide 9Analytics Solution Centers 1,000universities Developing curriculum and training for analytics with $1B To bring cognitive services and applications to market
45.
© 2016 IBM
Corporation45 Spark investments: community, core, and consumption Core Accelerating Spark capabilities Community Growing Spark knowledge & expertise Consumption Using Spark within IBM & partner products Spark Technology Center Big Data University SystemML open source contribution Spark stand-alone Hadoop distribution IBM portfolio 30+ research initiatives 3500+ IBM developers and researchers
46.
© 2016 IBM
Corporation46 The bottom line about IBM and Big Data Big Data is a strategic initiative for IBM Significant investments across software, hardware and services. BigInsights Enables firms to exploit growing variety, velocity, and volume of data Delivers diverse range of analytics Leverages and extends open source Provides enterprise-class features and supporting services Complement existing software investments and commercial offerings IBM advantage Full solution spanning software, hardware & services Rapid technology advances through partnerships with IBM Research Global reach
47.
© 2016 IBM
Corporation47 Jump start your efforts with IBM Analytics Stampede Leading the charge for your analytics success IBM’s Expertise - takes the guesswork out and delivers savings in time and cost for your early enablement and success IBM’s Analytics Solution - provides unmatched capabilities for processing and analyzing all types of data Skills & Knowledge Transfer - ensures knowledge transfer and training roadmap for skills enablement in your organization for new analytics requirements Stampede Time to insights Research Product Selection Services Soluiton Success Solution Success Knowledge Transfer Analytics Prototypes BVA / Roadmaps Standard Roadmap IBM Expertise Use Case Selection Skills & Knowledge https://www-01.ibm.com/software/data/services/stampede.html
48.
© 2016 IBM
Corporation48 Want to learn more? Download Quick Start offering Follow tutorials, videos, and more Links all available from HadoopDev – https://developer.ibm.com/hadoop/
49.
© 2016 IBM
Corporation IBM big data • IBM big data • IBM big data IBM big data • IBM big data • IBM big data IBMbigdata•IBMbigdata IBMbigdata•IBMbigdata THINK
Download now