Abstract of the Talk —The potential to extract actionable insights from big data has gained increased attention of researchers in academia as well as several industrial sectors. The field has become interesting and problems look even more exciting to solve ever since organizations have been trying to tame large volumes of complex and fast arriving big data streams through newer computing paradigms. However, extracting meaningful and actionable information from big data is a challenging and daunting task. The ability to generate value from large volumes of data is an art which combined with analytical skills needs to be mastered in order to gain competitive advantage in business. The ability of organizations to leverage the emerging technologies and integrate big data into their enterprise architectures effectively depends on the maturity level of the technology and business teams, capabilities they develop as well as the strategies they adopt. In this paper, through selected use cases, we demonstrate how statistical analyses, machine learning algorithms, optimization and text mining algorithms can be applied to extract meaningful insights from the data available through social media, online commerce, telecommunication industry, smart utility meters and used for variety of business benefits, including improving security. The nature of applied analytical techniques largely depends on the underlying nature of the problem so a one-size-fits-all solution hardly exists. Deriving information from big data is also subject to challenges associated with data security and privacy. These and other challenges are discussed in context of the selected problems to illustrate the potential of big data analytics.

1. ©2015 IBM Corporation
Analytics : Key to go from generating
big data to deriving business value
Piyush Malik
WW Big Data Analytics CoE Leader
IBM Global Business Services, USA
@pmalik1
(Based on a paper published by IEEE co-authored in collaboration with
Dr Deepali Arora, University of Victoria, Canada)

2. ©2015 IBM Corporation|
Talk Outline
1. Why is big data
important?
2. Use of big data to derive
business value
3. How is analytics useful in
deriving value
4. Five patterns of business
intelligence from big data
5. Current challenges
6. Future directions
2009
800,000 petabytes
2020
35 zettabytes
as much Data and Content
44x
80%
*Source: IDC

3. ©2015 IBM Corporation|
Use of big data to derive business value

4. ©2015 IBM Corporation|
What we used to call “Big Data” is becoming the
norm in industry conversations
4
2012 2013 2014 2015
Volume
Velocity
Variety
Veracity
Confluence of
Social, Mobile,
Cloud, Big Data, and
Analytics
Systems of Insight
Data Transforming
Industries
Data will disrupt IT
in the every
industry
Mobile Social
Cloud
Internet of Things
Data

5. ©2015 IBM Corporation|
1 in 2
business leaders do
not have access to
data they need
83%
of CIO’s cited Business
Intelligence (BI) and
analytics as part of their
visionary plan
5.4X
more likely that top
performers use
business analytics
80%
of the world’s
data today is
unstructured
90%
of the world’s data
was created in the
last two years
20%
of available data can
be processed by
traditional systems
Source: GigaOM, IBM Software Group, IBM Institute for Business Value"
Let’s consider the realities associated with data

6. ©2015 IBM Corporation|
How is analytics useful in deriving business value
Big data is analyzed using advanced techniques including machine learning algorithms

7. ©2015 IBM Corporation|
How is analytics useful in deriving business value
7
Supervised Unsupervised
Linear Nonlinear
Single Combined
Easy to Interpret Hard to Interpret
Linear
Regression
Logistic
Regression
Perceptron
Bagging Boosting Random
Forest
Decision Rule
Trees Learning
Naïve k-Nearest
Bayes Neighbours
Multi-Layer SVM
Perceptron
K-Means EM Self-
Organizing
Maps
Common Machine Learning Algorithms
Machine learning algorithms commonly used to analyze big data are:

8. ©2015 IBM Corporation|
Real-time traffic
flow optimization
Fraud and risk
detection
Accurate and timely
threat detection
Predict and act on
intent to purchase
Understand and
act on customer
sentiment
Low-latency network
analysis
Industry Examples of Applied Big Data Analytics

9. ©2015 IBM Corporation|
Five use cases of business intelligence from big data
Case 1: Sentiment analysis in social networks
With an IBM Social Analytics you can decode the psychological genotype
of your customer to achieve unprecedented customer intimacy
9
Psychological profile
 Personality
 Needs
 Values
 Activity profiles
IBM FOAK‘s with …
 Two retailers
 Three hotel chains
 Two airlines
 Two governmental
departments
 Followers analyzed
 200+ million Tweets
 300K+ users analyzed

10. ©2015 IBM Corporation|
Case 1: Sentiment analysis in social networks
Five use cases of business intelligence from big data

11. ©2015 IBM Corporation|
Case 1: Sentiment analysis in social networks
Five use cases of business intelligence from big data
Recently a Telecom company used cloud-based social media analytics to
proactively retain customers before they have decided to leave
Business challenge: Customers continue to shift to mobile and social
channels in the way they converse about brands. Social channels are
often the first way to express grievances and doubts. In real time, negative
sentiments can quickly proliferate and influence existing and prospective
customers. How can we leverage these social channels to connect with
customers in a way that suits their preferences in order to build loyalty
and reduce attrition?
The socially aware solution: The Telco uses customer sentiment derived
from Twitter postings along with other social data and internal company
records to understand customer preferences and predict customers at risk
of attrition. Resource is being shifted from staffing call centres to social
media engagements in order to empower customers and respond to them
via their preferred communication medium.
30% reduction
in customer attrition rate
Increased revenues
 10% increase in Call
Centre Agent revenues
 25% increase in cross sell
and upsell opportunities
Higher Customer
Satisfaction
Rating improved from 1.5 to 2.7
on a five point scale
Powered by
 Real time Twitter feed and other
Social Media data
 IBM SMA 1.3 (DB2, Cognos)
 IBM Big Match
 IBM SPSS Modeler
 IBM Psycholinguistics
 Next Best Action .

12. ©2015 IBM Corporation|
12
xDRs
Billing
CRM
Location
Account
Internet
Network
Millions of events
per second
Dropped Calls
Outgoing International Calls
Call Duration
Extra Call
Contract Expiration
Entered new cell
New Top-Up
5 minutes left on pre-paid
Invoice Issued
Congested Cells
Invoice Paid
Acquired new products
Change contracts
Brand Reputation
Customer Sentiment
Customer is roaming
Customer is at home
3 dropped calls in 10 minutes
Customer is close to a store
Customer enters a shopping area
Invoice paid + ‘liked’ competitor
Smart phone browsing pattern
Customer is watching an OTT
video
Streams of
intelligence
from Social network
Changed Home Location
Broadband Saturation
Where should I
INVEST to
gain/retain more
high-value
customers?
Who is THIS
customer and what
do THEY
want/need?
What should I be
OFFERING specific
customers to
improve individual
ARPU/profitability?
Actionable
insight
MDM EDW/ADW
Big Data Analytics Value proposition Scenario : Combine Network,
Billing, Subscriber, Call Records, etc to gain new, valuable and
actionable insights, prevent churn and improve outcomes
Case 2: Preventing customer churn in telecommunication sector
Five use cases of business intelligence from big data

13. ©2015 IBM Corporation|
Case 2: Preventing customer churn in telecommunication sector
Five use cases of business intelligence from big data
A Malaysian Telco leveraging Big Data in Motion
Data in motion
CDRs
Data
Location
Reload
Provisioning
Others
(future)
8000+
Events
per
second
Microsecon
d Latency
Dropped Calls
Outgoing International
CallsCall Duration
Extra Call
Dongle purchase
Entered new cell
New Top-Up
5 minutes left on pre-paid
RTP Platform
3 Dropped Calls in the last hour
In location of interest
5 Outgoing international Calls in the last day
Data Aggregated at
Single Customer Level
Filtering of high-throughput TPS to
fewer selected events of interest
RSS Feeds
Web sites
GPS Location
13

14. ©2015 IBM Corporation|
Case 3: Enhancing customers’ online shopping experience
a) Enhanced Social Shopping
Five use cases of business intelligence from big data

15. ©2015 IBM Corporation|
b) In-Store Presence Zones
Intelligent location-based technology to gain deep insight
into customer in-store behavior
Enables retailers to integrate the physical and digital experience to facilitate an ongoing
dialogue, create loyalty and deliver an exceptional in-store shopping experience
Presence Zones
Sensors
Case 3: Enhancing customers’ online shopping experience
Five use cases of business intelligence from big data
15

16. ©2015 IBM Corporation|
c) Hyper-personalization and contextual shopping
Enables retailers to microsegment customer base, target based on demographic and
psychographic criteria for 1:1 marketing and lift sales using social commerce,
gamification and big data
Case 3: Enhancing customers’ online shopping experience
Five use cases of business intelligence from big data

17. ©2015 IBM Corporation|
Smart Meter
Analytics
Condition
Based
Maintenance
Smart Meter
Analytics
Smart Meter
Analytics
Distribution
Load
Forecasting &
Scheduling
Improve Generation
Performance
Transform Customer
Operations
Condition
Based
Maintenance
Transform the Utility
Network
Distribution
Load
Forecasting &
Scheduling
Smart Meter Analytics is a Common Element Across a Smarter Energy
Value Chain, Differentiation Comes through Leveraging Big Data
Customer
Insight
Case 4: Generating value from smart utility meters
Five use cases of business intelligence from big data
17

18. ©2015 IBM Corporation|
Case 4: Generating value from smart utility meters
Five use cases of business intelligence from big data
How IBM Assisted an Electric Utility in Leveraging Smart Metering/Big Data
to create Business Value
Load, manage, and analyze information from smart meters, the smart grid and customer
information - use that data to gain customer and operational insights.
What Data?
• Meter readings
• Grid data
• Customer
information
• Meter generated
alerts and
power quality
indicators
• Meter
connection
status
What Capability?
• Load interval meter
readings (Time Series
Data)
• Track & evaluate
energy losses
• Micro customer
segmentation
• Secure Analytics
Warehouse for
operational insight
• Analyze meter failures
and power outages
What Outcome?
• Better understand
and manage outages
• Detect energy theft
• Drive customer
usage portal
• Prepare customer
specific alerts and
communications
• Customer
segmentation and
insights
Smart
Meter Data
Grid Data
Customer
Information
18

19. ©2015 IBM Corporation|
Analytics
and
Reporting
Sentiment Analysis
Call Center Analysis
Offering Management
DistributionGeneration Transmission
EmployeesMaintenance
Suppliers Orders
Marketing GIS
Customers
Smart Meters
Trading
Regulations
Social Media
Sensors
ImprovedAnalytics
Unstructured
Exploration/Discovery
Queryable Archive
InfoSphere
BigInsights
Unstructured
Streaming Structured or Unstructured
Unstructured
ImprovedAnalytics
Structured
Smart Grid Analytics
Distribution Grid Monitoring
Root Cause FailureAnalysis
IBM
InfoSphere
Streams
Real Time Scoring
and Response
Analytics
and
Reporting
ETL
IBM
Data Warehouse
Analytics
and
Reporting
Meter Data Management
Demand Forecasting
Maintenance Scheduling
Case 4: Generating value from smart utility meters
Five use cases of business intelligence from big data
How IBM Assisted an Electric Utility in Leveraging Smart Metering/Big Data
to create Business Value

20. ©2015 IBM Corporation|
Smarter Planet = More Connected, More Vulnerable
Case 5: Improving Security
Five use cases of business intelligence from big data
20

21. ©2015 IBM Corporation|
Threats are evolving
Attacker generic
Malware / Hacking / DDoS
IT Infrastructure
Before . .
Advanced
Persistent
Threat
Critical data /
infrastructure
Attacker
!
Case 5: Improving Security
Five use cases of business intelligence from big data

22. ©2015 IBM Corporation|
IBM QRadar Platform: Taking in data from wide spectrum of feeds and
continually adding context for increased accuracy
Security Intelligence Feeds
Internet ThreatsGeo Location Vulnerabilities
Case 5: Improving Security
Five use cases of business intelligence from big data

23. ©2015 IBM Corporation|
Applying Adaptive Analysis & Classification
Antennae
(Directional,
Omnidirectional)
+ Digitization
Direction
Finding
De-interleaving
Chain Analysis
Classification
Tracking
Stream Computing
As fast, low latency
Signal Processing Functions
Adaptive
History
hadoop technologies
Offline Analysis
Build Models & Patterns
Condition Real Time
Processing
Pulse Data
Analysis
Case 5: Improving Security
Five use cases of business intelligence from big data
Security/Intellig
ence Extension
Lower risk, detect
fraud and monitor
cyber security in
real-time
23

24. ©2015 IBM Corporation|
Big Data Processing
• Long-term, multi-PB storage
• Unstructured and structured
• Distributed Hadoop infrastructure
• Preservation of raw data
• Enterprise Integration
Big Data
Platform
Analytics and Forensics
• Advanced visuals and interaction
• Predictive & decision modeling
• Ad hoc queries
• Interactive visualizations
• Collaborative sharing tools
• Pluggable, intuitive UI
Security Intelligence
Platform
Real-time Processing
• Real-time data correlation
• Anomaly detection
• Event and flow normalization
• Security context & enrichment
• Distributed architecture
Security Operations
• Pre-defined rules and reports
• Offense scoring & prioritization
• Activity and event graphing
• Compliance reporting
• Workflow management
Integrated analytics and exploration in a new architecture
Integrated
IBM
Solution
Case 5: Improving Security
Five use cases of business intelligence from big data
24

25. ©2015 IBM Corporation|
Data ingest
Insights
IBM Security QRadar
• Hadoop-based
• Enterprise-grade
• Any data / volume
• Data mining
• Ad hoc analytics
• Data collection and
enrichment
• Event correlation
• Real-time analytics
• Offense prioritization
Big Data Platform
Custom AnalyticsAdvanced Threat Detection
Traditional data sources
IBM InfoSphere BigInsights
Non-traditional
Security Intelligence Platform
How? By integrating QRadar with Hadoop-based solution
Case 5: Improving Security
Five use cases of business intelligence from big data
25

26. ©2015 IBM Corporation|
Current challenges and future directions
1. Finding the right kind of data to use
2. Managing large datasets
3. Selecting algorithms to extract meaningful information for
different domains
4. Security and Privacy
5. Finding skilled people with good understanding of analytics
• Contextual
• Cognitive
• Public-Private Data Partnerships
• Data Marketplaces
Current challenges
Future Directions
26

27. ©2015 IBM Corporation|
Summary & Conclusion
Questions?
27