Presentation about BigData from a German Webcast: http://business-services.heise.de/it-management/big-data/beitrag/big-data-technologie-einsatzgebiete-datenschutz-160.html?source=IBM_12_2013_IT_Conn

1. Big Data
Wilfried Hoge, Software IT Architect Big Data
hoge@de.ibm.com
@wilfriedhoge

2. How is Big Data transforming the way
organizations analyze information and
generate actionable insights?

3. Paradigm shifts enabled by big data
Leverage more of the data being captured
TRADITIONAL APPROACH
All available
information
Analyze small subsets
of Information
BIG DATA APPROACH
Analyzed
information
All available
information
analyzed
Analyze
all information

4. Paradigm shifts enabled by big data
Reduce effort required to leverage data
TRADITIONAL APPROACH
Small
amount of
carefully
organized
information
Carefully cleanse information
before any analysis
BIG DATA APPROACH
Large
amount of
messy
information
Analyze information as is,
cleanse as needed

5. Paradigm shifts enabled by big data
Data leads the way—and sometimes correlations are
good enough
TRADITIONAL APPROACH
BIG DATA APPROACH
Hypothesis
Question
Data
Exploration
Answer
Data
Insight
Correlation
Start with hypothesis and
test against selected data
Explore all data and
identify correlations

6. Paradigm shifts enabled by big data
Leverage data as it is captured
TRADITIONAL APPROACH
BIG DATA APPROACH
Data
Analysis
Data
Repository
Analysis
Insight
Insight
Analyze data after it’s been processed
and landed in a warehouse or mart
Analyze data in motion as it’s
generated, in real-time

7. How have most companies made
information available for decision
making across the enterprise?

8. Traditional enterprise data and analytics
environments
Typical enterprise data management environment
Data sources
Actionable insight
Predictive analytics
and modeling
Transaction and
application data
Staging area
Enterprise
warehouse
Archive
Data mart
Reporting and
analysis

9. How are leading companies transforming
their data and analytics environment
to provide faster, better insights
at reduced costs?

10. Next generation architecture
Starts from the current data management environment …
Data sources
Actionable insight
Enterprise
Enterprise
warehouse
warehouse
Transaction and
application data
Information
ingestion and
operational
information
Data mart
Data mart
Predictive analytics
and modeling
Analytic
appliances
Analytic
appliances
Reporting, analysis,
content analytics
Information governance
SYSTEMS—SECURITY—STORAGE

11. Next generation architecture
… adds new technologies and capabilities …
Data sources
Real-time analytics
Actionable insight
Machine and
sensor data
Cognitive
Image and video
Enterprise
content
Transaction and
application data
Information
ingestion and
operational
information
+
+
Social data
Third-party data
Exploration,
landing and
archive
Enterprise
Enterprise
warehouse
warehouse
Decision
management
Data mart
Data mart
Predictive analytics
and modeling
Analytic
appliances
Analytic
appliances
Reporting, analysis,
content analytics
Information governance
SYSTEMS—SECURITY—STORAGE
Discovery and
exploration

12. Next generation architecture
… to enable new applications
Data sources
Real-time analytics
Actionable insight
Machine and
sensor data
Cognitive
Image and video
Enterprise
content
Transaction and
application data
Information
ingestion and
operational
information
+
+
Social data
Third-party data
Exploration,
landing and
archive
Enterprise
Enterprise
warehouse
warehouse
Decision
management
Enhanced
applications
Customer
experience
New business
models
Financial
performance
Data mart
Data mart
Predictive analytics
and modeling
Analytic
appliances
Analytic
appliances
Reporting, analysis,
content analytics
Information governance
SYSTEMS—SECURITY—STORAGE
Risk
Operations
and fraud
IT economics
Discovery and
exploration

13. Sample use cases that leverage the
new data management environment
capabilities

14. Dublin City Centre; Robust
and efficient citywide traffic
awareness system, enhance
rapid action on incidents
Need
•
A budget effective solution to improve traffic
awareness system.
•
To bring accuracy in event detection,
inferring traffic condition (road speed) and
prediction of bus arrival.
•
Challenge is to rightly analyze GPS data,
which is typically high data throughput and
difficult to capture
Benefits
•
Monitor 600 buses across 150 routes daily
•
Analyzes 50 bus location updates per
second , using InfoSphere Streams
•
Collects, processes, and visualizes location
data of all public transportation vehicles

15. Architecture for traffic awareness system
Real-time analytics to enhance customer experience
Data sources
Real-time analytics
Actionable insight
Machine and
Machine and
sensor data
sensor data
Cognitive
Image and video
Enterprise
content
Transaction and
application data
Information
ingestion and
operational
information
+
+
Social data
Third-party data
Exploration,
landing and
archive
Enterprise
Enterprise
warehouse
warehouse
Decision
management
Enhanced
applications
Customer
experience
New business
models
Financial
performance
Data mart
Data mart
Predictive analytics
and modeling
Analytic
appliances
Analytic
appliances
Reporting, analysis,
content analytics
Information governance
SYSTEMS—SECURITY—STORAGE
Risk
Operations
and fraud
IT economics
Discovery and
exploration

16. Constant Contact
Transforming Email Marketing
Campaign Effectiveness with
IBM Big Data
Need
• Analyze 35 billion annual emails to guide
customers on best dates & times to send emails
for maximum response
Benefits
• 40 times improvement in analysis performance
• 15-25% performance increase in customer email
campaigns
• Analysis time reduced from hours to seconds

17. Architecture for email marketing
Analyze to maximize response rates
Data sources
Real-time analytics
Actionable insight
Machine and
sensor data
Cognitive
Image and video
Enterprise
content
Transaction and
application data
Information
ingestion and
operational
information
+
+
Social data
Third-party data
Exploration,
landing and
archive
Enterprise
Enterprise
warehouse
warehouse
Decision
management
Enhanced
applications
Customer
experience
New business
models
Financial
performance
Data mart
Data mart
Predictive analytics
and modeling
Analytic
appliances
Analytic
appliances
Reporting, analysis,
Reporting, analysis,
content analytics
content analytics
Information governance
SYSTEMS—SECURITY—STORAGE
Risk
Operations
and fraud
IT economics
IT economics
Discovery and
exploration

18. Battelle, helping reduce
energy costs and enhancing
power grid reliability and
performance
Need
• Assess the viability of one smart grid
technique called transactive control
Benefits
• Engages consumers and responsive assets
throughout the power system to help optimize
the system and better integrate renewable
resources
• Provides the capability to analyze and gain
insight from up to 10 PB of data in minutes
• Increases grid efficiency and reliability through
system self-monitoring and feedback
• Enables a town to avoid a potential power
outage

19. Architecture for smart grid
Analytics to reduce costs and optimize grid
Data sources
Real-time analytics
Actionable insight
Machine and
Machine and
sensor data
sensor data
Cognitive
Image and video
Enterprise
content
Transaction and
application data
Information
ingestion and
operational
information
+
+
Social data
Third-party data
Exploration,
landing and
archive
Enterprise
Enterprise
warehouse
warehouse
Decision
management
Enhanced
applications
Customer
experience
New business
models
Financial
performance
Data mart
Data mart
Predictive analytics
and modeling
Analytic
appliances
Analytic
appliances
Reporting, analysis,
content analytics
Information governance
SYSTEMS—SECURITY—STORAGE
Risk
Operations
and fraud
IT economics
Discovery and
exploration

20. Trust is the most important aspect of a
Big Data solution

21. Trust It
Be proactive about privacy, security and governance
Trust the facts
Ensure privacy
and security
Make risk
aware decisions
Create foundation
of trusted data
Understand usage and
monitor compliance
Model exposure and
understand variability

22. Big Data Privacy and Security
Protect a Wider Variety of Sources
• Protec&on
is
a
pre-­‐requisite
for
the
fundamental
assump&on
of
big
data
–
sharing
data
for
new
insight
• Automa&on
enables
protec&on
without
inhibi&ng
speed
• Data
ac&vity
monitoring
of
NoSQL,
Hadoop,
and
Rela&onal
Systems
• Masking
of
sensi&ve
data
used
in
Hadoop
• Ensures
sensi&ve
data
is
protected,
encrypted
and
secure
RDBMS
InfoSphere
Guardium
InfoSphere
Optim
Hadoop
NoSQL
Data Warehouses
Application Data and Files

23. Data volumes continue to grow fast.
New technologies will be needed in the
future

24. Big Data brings Technical Challenges
IBM invests in future solutions
Huge Volumes of Multimedia Data:
New Storage Requirements
The Challenge of Moore’s Law:
New Storage Methods Needed
IBM’s Approach:
Start at the Atomic Level
how many atoms are needed to store
1-bit of data

25. What are your activities to leverage Big
Data Analytics?

26. Your big data journey
IBM can help wherever you are
Educate
25%
Focused
on
knowledge
gathering
and
market
observa&ons
Explore
47%
Developing
strategy
and
roadmap
based
on
business
needs
and
challenges
Case
studies,
Whitepapers
and
Value
Repors
ibmbigdatahub.com
IBM
Briefings,
Solu&on
Centers
Learning,
explora&on
downloads
&
test
BigDatauniversity.com,
YouTube
Big
Data
Channel
Engage
22%
Pilo&ng
big
data
ini&a&ves
to
validate
value
and
requirements
Execute
6%
Deployed
two
or
more
big
data
ini&a&ves
and
con&nuing
to
apply
advanced
analy&cs
Validate
and
realize
business
value
IBM
Roadmap
Workshop
• Priori&sed
business
use
cases
• Recommend
plaRorm
Solu&on
Design
&
Proof
of
Concept
Enterprise-­‐wide
big
data
ini&a&ves
• Stampede
–
exper&se
and
skills
to
get
value
straight
away
• Enterprise
data
plaRorm
• Validate
business
value
• Demonstrate
capabili&es
Learn
the
technology
&
gain
exper&se

27. THINK