Nowadays, successful Big Data initiatives rely on the ability to act fast and to cope with the variety of data and models, like structured and unstructured data from sensors, social media or databases. In this break-out-session, we will showcase how PRIMEFLEX for Hadoop, a powerful and scalable analytics platform, can help business oriented users and citizen data scientists to collect, transform, analyze and even leverage artificial intelligence for Big Data analysis. Alexander Kaffenberger, Senior Business Developer – Big Data, Category Management EMEIA, Fujitsu

1. 0 Copyright 2017 FUJITSU
PRIMEFLEX for Hadoop
Experience
Big Data
Analytics
Fujitsu World Tour 2017
#FujitsuWorldTour
Alexander Kaffenberger
Fujitsu EMEIA Category Management

2. 1 Copyright 2017 FUJITSU
PRIMEFLEX for Hadoop
InfrastructureSoftwareServices
Analytic Services
Sizing and Integra-
tion
Service
Analytics & Big Data Management
Smart
Entry
Rack
Business Value
 Standard x86 Servers & Components
 Extreme scalability (28TB – ~100 PB)
 Efficient data store
 Self service Big Data Analytics
 Easy data ingestion, modelling and visualization
 100% Business user friendly / 0% programming
 SPOC appliance support
 PoCs and pilots with Fujitsu Data Scientists
1 Copyright 2017 FUJITSU
PRIMEFLEX for Hadoop

3. 2 Copyright 2017 FUJITSU
Automation of Big Data Value Chain
Big Data
Collect
Stream
Structured &
unstructured data
Device data,
Sensor data,
Internet of Things
Cleanse /
Transform
Model / Learn
Analyze
Find
Decide
Navigate
Research &
development,
science
Operation,
automation,
production
Interactive
reporting,
advertising
Rapid modelling for faster insights
Social Media data,
Open data,
Linked data

4. 3 Copyright 2017 FUJITSU
Self-Service Data Science
Big Data
Extract
Collect
Structured &
unstructured data
Devices,
sensors,
Internet of Things
Cleanse
Transform
Analyze
Find
Decide
Act
Research &
development,
science
Operation,
automation,
production
Interactive
reporting,
advertising
Social media, open
data,
linked data
Big Data Import, Analysis, Visualization like Outlook, Excel and PowerPoint
Collect
Stream
Cleanse / Transform
Model / Learn
Analyze
Find
Decide
Navigate
Made for
Citizen
Data
Scientists

5. 4 Copyright 2017 FUJITSU
Big Data Infrastructure Reference Architecture
Consolidated data Distilled essence Applied knowledgeVarious data
Extract, Collect Cleanse, Transform Decide, ActAnalyze, Visualize
Data Sources Analytics Platform Access
Batch processing
platform
Event processing
platform
Fast response
platform
Data bases
Application
server
Web
content
Sensor
data
Apps
Services
Queries
Visualization
Reporting
Notification

6. 5 Copyright 2017 FUJITSU
Integrated Solution PRIMEFLEX for Hadoop
Consolidated data Distilled essence Applied knowledgeVarious data
Extract, Collect Cleanse, Transform Decide, ActAnalyze, Visualize
Data Sources Analytics Platform Access
Batch processing
platform
Event processing
platform
Fast response
platform
Data bases
Application
server
Web
content
Sensor
data
Apps
Services
Queries
Visualization
Reporting
Notification

7. 6 Copyright 2017 FUJITSU
Data Warehouse and Datalake?
PRIMEFLEX
for SAP HANA
PRIMEFLEX
for Hadoop
SAP Vora™
Data lakes
Bridge between
HANA and Hadoop

8. 7 Copyright 2017 FUJITSU
Use cases

9. 8 Copyright 2017 FUJITSU
Use Case: Genome Data Analysis
Challenges
104 Gene-samples without data reduction.
Focusing on cancer genes (~900.000 pos.).
Identification of genes / exones with unsufficient
coverage for reliable recognition of mutations.
Solution Parallel processing of on full raw data
Benefits
Spreadsheets for subject specialist without programming skills
Scaling parallel processing platform for raw data analysis
Discovery of multiple regions in cancer genes with insufficient
coverage (blind spots after data reduction)
Data 104 DNA samples
140.000.000.000 records,
14*1012 bases
45 TB data(18TB compressed)
Time
reduction
2 days to
5 min

10. 9 Copyright 2017 FUJITSU
Use Case: Machine Data Analysis
Challenges
Sensor logs from turning machine.
Using multiple tools on one work piece.
Many files from sensors.
Find anomalies pointing to production failure.
Automation of complex failure detection.
Solution Detect anomalies with AI clustering algorithm
Benefits
Fast wizard driven import of many file formats from many sources
Data
Sensor data from machines
100 files/second/tool application
Rapid data visualization with flip sheet and drag&drop Infographics
Easy cleansing, transformation and analysis by familiar Excel like
Workbooks and visualization and/or export of results
Machine learning: Clustering (K-means) for automatic grouping
Anomalies
& Patterns
detected

11. 10 Copyright 2017 FUJITSU
Import data with Datameer
Import of many files to
HDFS from various shared
or remote sources
(NFS, SSH, FTP, HTTP,…)
Import wizards for
many source formats
(CSV, JSON, XML, …)
Transformation to Excel
like table format
1) Evaluation data set kindly provided by Prof. Dr.-Ing. Joachim Imiela, Geschäftsführer Optvia Unternehmensberatung (http://www.optvia.de)
1)

12. 11 Copyright 2017 FUJITSU
Get a quick overview with Datameer
Use Flip Sheet to view
standard column statistics
Build Drag&Drop Infographics to discover more details
8017 is
the most
used tool

13. 12 Copyright 2017 FUJITSU
Inspect Details and find patterns
Automatically build multiple Workbooks
and Infographics by scripting the
Datameer REST API
Identify
suspicious
graph

14. 13 Copyright 2017 FUJITSU
Use Case: Weather data Analysis
Challenge
Time series of global weather maps not
suitable for local insights.
Fast retrieval of multiple alternative data sets
during planning phases.
Solution Complete transformation of maps into local
time series of meteorological data
Benefits
Time series for locations worldwide
Weather trends for planning e.g. industry, tourism
Simple extension by additional data
Data ECMWF weather maps (35 years)
1.038.240 output files (location dependent)
128 meteorological identifiers
25 TB historical weather data
Time
Location
Wind speed
Wind speed &
direction
Temperature
Pressure
Wind frequency &
direction
Detail selection Month selector
Real time
Analytics
on >25 TB

15. 14 Copyright 2017 FUJITSU
Challenge Difficult interpretation of feed back sources and
opinion trends
Handling „storms“ (e.g.. Paris Attacks >10k/min)
Solution Collection and analysis of consolidated social
media and geodata
Benefits
Automated evaluation of social media data
Monitoring / tracking of trends, Keyword-Searching
Fast identification of feedback sources
Data
Use Case: Sentiment Analysis
Social networks
Sentiment dictionary
Geo-Data



Advanced
Analytics
batch &
real time

16. 15 Copyright 2017 FUJITSU
For excellent results:
Start fast – Find the right way fast
Thomas Alva Edison:
“I have not failed, I just found
10.000 ways that don’t work”.
Pre-configured, pre-tested and
ready to run.
Source:https://de.wikipedia.org/wiki/Thomas_Alva_Edison

17. 16 Copyright 2017 FUJITSU