Big Data trap
francis@qmining.com
@fraka6
Data/Big Data Knowledge Action
People care about Knowledge/actions not data
Agenda
● Big data dilemma
● When are we doing Big Data?
● Maturity/Evolution steps
● The big data trap
● Optimal design = ...
The Big Data Dilemma
Big Data =
Data + IO bounded (disk)
CPU
<100%Data
IO bounded
QA
BI
Maturity
Barriers of entry Levels
Just another barrier of entry
Trap = no KPI
● No KPI -> batch processing -> big data
● KPI -> real time -> no big data complexity
Optimal design = real-time data-mining
● Events -> everything is an event
● + Rule -> create signal from events
● + KPIs -...
Increase chances of success
● Data driven culture
● Data quality culture (Avoid logs)
● Reach Analytics/BI level
● KISS
Recap
● Big Data = Small Data + IO bound
● Big data->Data->Analytics->Mining->Predictive
○ Data Quality = BIGGEST PROBLEM
...
hum...
Questions?
francis@qmining.com
Upcoming SlideShare
Loading in …5
×

Big data trap

469 views
264 views

Published on

Don't fall in the big data trap key principles

Published in: Data & Analytics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
469
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Big data trap

  1. 1. Big Data trap francis@qmining.com @fraka6
  2. 2. Data/Big Data Knowledge Action People care about Knowledge/actions not data
  3. 3. Agenda ● Big data dilemma ● When are we doing Big Data? ● Maturity/Evolution steps ● The big data trap ● Optimal design = real time data-mining ● Increase your chances of success
  4. 4. The Big Data Dilemma
  5. 5. Big Data = Data + IO bounded (disk) CPU <100%Data IO bounded
  6. 6. QA BI Maturity Barriers of entry Levels Just another barrier of entry
  7. 7. Trap = no KPI ● No KPI -> batch processing -> big data ● KPI -> real time -> no big data complexity
  8. 8. Optimal design = real-time data-mining ● Events -> everything is an event ● + Rule -> create signal from events ● + KPIs -> selection of signals (top level) ● + Incident = signal static/dynamic thresholds ● + Root causes analysis ○ Bayesian inference (ratio signal) ○ Signal correlation (std signal) ○ Rule filtering (domain specific)
  9. 9. Increase chances of success ● Data driven culture ● Data quality culture (Avoid logs) ● Reach Analytics/BI level ● KISS
  10. 10. Recap ● Big Data = Small Data + IO bound ● Big data->Data->Analytics->Mining->Predictive ○ Data Quality = BIGGEST PROBLEM ○ Big Data = another barrier of entry ● Big data trap = no KPI ● KISS = real time data mining
  11. 11. hum... Questions? francis@qmining.com

×