Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Big data trap

643 views

Published on

Don't fall in the big data trap key principles

Published in: Data & Analytics
  • Be the first to comment

  • Be the first to like this

Big data trap

  1. 1. Big Data trap francis@qmining.com @fraka6
  2. 2. Data/Big Data Knowledge Action People care about Knowledge/actions not data
  3. 3. Agenda ● Big data dilemma ● When are we doing Big Data? ● Maturity/Evolution steps ● The big data trap ● Optimal design = real time data-mining ● Increase your chances of success
  4. 4. The Big Data Dilemma
  5. 5. Big Data = Data + IO bounded (disk) CPU <100%Data IO bounded
  6. 6. QA BI Maturity Barriers of entry Levels Just another barrier of entry
  7. 7. Trap = no KPI ● No KPI -> batch processing -> big data ● KPI -> real time -> no big data complexity
  8. 8. Optimal design = real-time data-mining ● Events -> everything is an event ● + Rule -> create signal from events ● + KPIs -> selection of signals (top level) ● + Incident = signal static/dynamic thresholds ● + Root causes analysis ○ Bayesian inference (ratio signal) ○ Signal correlation (std signal) ○ Rule filtering (domain specific)
  9. 9. Increase chances of success ● Data driven culture ● Data quality culture (Avoid logs) ● Reach Analytics/BI level ● KISS
  10. 10. Recap ● Big Data = Small Data + IO bound ● Big data->Data->Analytics->Mining->Predictive ○ Data Quality = BIGGEST PROBLEM ○ Big Data = another barrier of entry ● Big data trap = no KPI ● KISS = real time data mining
  11. 11. hum... Questions? francis@qmining.com

×