Case Study: Cloud based DWH Solution using Amazon Redshift

1,124 views

Published on

Presentation by Ionut Hedesiu, iQuest at CeBit 2014: Cloud based DWH Solution using Amazon Redshift http://blog.iquestgroup.com/en/cloud-based-dwh-solution-using-amazon-redshift/#.U0KvnVfestU

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,124
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Case Study: Cloud based DWH Solution using Amazon Redshift

  1. 1. Cloud-based DWH Solution Using Amazon Redshift CeBIT | 12 March 2014 Ionut Hedesiu Senior Software Engineer
  2. 2. What is Big Data? Big What does it stand for? Does it really matter?
  3. 3. What if? affordable and intuitive framework complete ETL flow ready in minutes no 3rd party licensing royalties any amount of data no single point of failure
  4. 4. Approach inexpensive, highly performant data warehousing strictly proven open source technologies horizontally and vertically scalable
  5. 5. Solution independent, m etadata-driven modules collection of python modules deployed and tested on enterprise/commodity hardware and Amazon cloud solutions
  6. 6. Implementation • simple virtual Linux boxes • instance auto-spawn • SQL code on the fly • AMQP standard messaging • detailed logging, Splunk • fully configurable
  7. 7. Features enterprise messaging metadata-driven ETL flows multiple work queues detailed logging in multiple destinations secure user access alerts based on user-defined formulas
  8. 8. Benefits  SCALABLE • vertical and horizontal • auto scalability and load balancing CUSTOMISABLE • platform and database agnostic • quick module addition or removal COST-EFFICIENT • minimal cost and development time • very low maintenance cost
  9. 9. Benefits POWERFUL • real-time data analytics • massive parallel processing • intensive data mining and cleansing ROBUST • 99.5% availability • minimal or no maintenance • lightweight framework FLEXIBLE • one central point of control • metadata driven 
  10. 10. Case Study – Global Media Organisation • 500+ source systems • 3 database vendors • local batch processing • no global data overview • no data integration
  11. 11. Implementation Overview • centralised data repository • real time processing • metadata driven • customised to client needs • Python • Rabbit MQ • Amazon Redshift • Tableau
  12. 12. Benefits & Results • tenfold cost reduction • intuitive and easy to use • secure and simple to administer • real time analytics • improved decision-making • minimal to no maintenance • high scalability

×