Building a Big Data Warehouse

1,647 views

Published on

Big Data presentation at XebiCon2013 - http://xebicon.nl/

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,647
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
88
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Building a Big Data Warehouse

  1. 1. GoDataDrivenPROUDLY PART OF THE XEBIA GROUP@mids106jorisbontje@godatadriven.comBuilding a Big DataWarehouseJoris BontjeBig Data Hacker
  2. 2. About MeBig Data HackerData Driven Solution ArchitectHadoopTrainer
  3. 3. About GoDataDriven
  4. 4. Data WarehouseEvolution
  5. 5. http://en.wikipedia.org/wiki/Data_warehouseIn computing, a data warehouse is adatabase used for reporting and dataanalysis.
  6. 6. Database Architecture (1.0)Products)Customers)Orders)Inventory)Sales)DB)
  7. 7. Analytical Database (2.0)Sales&Inventory&Customers&Products&Orders&
  8. 8. Basic DWH ArchitectureTX#DB#Analy+cal#DB#BI#ETL
  9. 9. Data MartsTX#DB#DW#Sales#Mktg#Prch#BI#
  10. 10. Multiple Data-Sourcesother&Files&TX&DB&DW&Sales&Mktg&Prch&BI&
  11. 11. Operational Data StoreDW#ODS#other#Files#TX#DB# Sales#Mktg#Prch#BI#
  12. 12. Hadoop
  13. 13. GoDataDrivenNo HadoopDW#ODS#other#Files#TX#DB# Sales#Mktg#Prch#BI#
  14. 14. GoDataDrivenETL Engineother&Files&TX&DB& Sales&Mktg&Prch&DWBI&
  15. 15. GoDataDrivenTiered Data Warehouseother&Files&TX&DB& Sales&Mktg&Prch&BI&
  16. 16. GoDataDrivenAnalytical Query Engineother&Files&TX&DB&BI&
  17. 17. Tools
  18. 18. Tools
  19. 19. Tools Applied
  20. 20. Tools Applied
  21. 21. Considerations
  22. 22. ConsiderationsBig Data is dirtyAutomate everythingMonitoring and QA become the same thing
  23. 23. My Past TrendsBig Data Forum 2012
  24. 24. My PastTrendsCloud / On-demand
  25. 25. My PastTrendsHadoop Hardware
  26. 26. My PastTrendsBatch → Real-Time
  27. 27. New TrendsXebiCon 2013
  28. 28. TrendsImpalaOpen Source, Real-time Query enginefor Hadoop
  29. 29. TrendsDefacto standard for Hadoop metadata
  30. 30. Simple Database ArchitectureProducts)Customers)Orders)Inventory)Sales)DB)
  31. 31. The future?Products)Customers)Orders)Inventory)Sales)
  32. 32. GoDataDrivenWe’re hiring / Questions? / Thank you!@mids106jorisbontje@godatadriven.comJoris BontjeBig Data Hacker

×