Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Beyond TCO


Published on

Beyond TCO

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Beyond TCO

  1. 1. 2016-06-29 Beyond TCO Architecting Hadoop for adoption and data applications Reid Levesque – Head, Solution Engineering
  2. 2. Introduction
  3. 3. Topics Technology Use cases Deployment Impact Next steps
  4. 4. Technology – Let’s talk Hadoop
  5. 5. Every company is a technology company… some just don’t know it yet.
  6. 6. Traditional systems under pressure Conventional wisdom • Put the code on an Application Server • Move the data to/from database • Move the data to/from NAS Reality check • This works well for small amounts of data • As data volumes increase this design falls apart
  7. 7. Hadoop to the rescue
  8. 8. How do we get Hadoop into the organization?
  9. 9. How about these use cases?  File archive +Hadoop  Data-intensive grid compute analytics  Database replacement  ETL off-load +Hadoop +Hadoop +Hadoop •Data is online; no need for tape backup •Cheaper than NAS / SAN •Increased performance / scalability •Metadata is easier to get; all the data is in one spot •Improved performance •Lower TCO •Reduced dependence on proprietary software •Reduce RDBMS licensing •Reduced operational cost for analysis •Improved functionality with stored XML •Lower TCO •Additional analytic capability •Better hardware utilization •Lower platform management
  10. 10. Not so much  File archive +Hadoop  Data-intensive grid compute analytics  Database replacement  ETL off-load +Hadoop +Hadoop +Hadoop TCO
  11. 11. Which use case did work?  Current batch was taking 4 hours; which limited the way they did their job  Users wanted interactive response times to design and test their financial models  This was net new functionality that could only be achieved in Hadoop
  12. 12. Now TCO makes more sense  File archive +Hadoop  Data-intensive grid compute analytics  Database replacement  ETL off-load +Hadoop +Hadoop +Hadoop With Hadoop TCO covered, previous use cases are now more compelling.
  13. 13. How do we deploy this?
  14. 14. Which distribution? Pick one:
  15. 15. Time to pick the hardware Is this true?
  16. 16. Commodity hardware + commodity networking = bad architecture
  17. 17. Before there was Hadoop, there were enterprise IT standards To name a few conflicts during the rollout… • Local account UID / names • OS settings • Root access • File locations • Standard mount sizes • Enterprise Active Directory • Monitoring systems Hadoop is NOT flexible on deployment requirements
  18. 18. Who does the work? Single team including: • Dedicated infrastructure team (Compute, Network, Data Center, Operations) • Dedicated Hadoop team (sysadmin/operations, engineering) • Hardware vendor engineers • Hadoop distribution engineers
  19. 19. Into production we go!
  20. 20. What was the impact?
  21. 21. Changing perceptions
  22. 22. Impact across the organization Infrastructure • Networking / Data Center designs • Relationship with storage, cloud, virtualization capabilities • Generating analytic use cases Development • Mega-attractor for talent • Application consolidation • Shifting from IT to business focus Management • Understanding (or accepting) new paradigm • Cross-department architecture alignment • Data-focus rather than application-focus Business • Continuously evolving understanding of capability / possibilities • Next generation IT w/ rapidly evolving ecosystem • Self-service innovation for business users
  23. 23. Lessons Learned Hadoop doesn’t remove hardware maintenance Hadoop development is still development! New paradigm – requires skilled developers A whole new set of error messages to decode There aren’t that many experts
  24. 24. Where do we go next?
  25. 25. Self-service tools
  26. 26. Selling Hadoop internally • This journey has taught me a lot about Hadoop; more than most people at the organization • The biggest tasks are educating the organization and doing simple things as a first step
  27. 27. Thank You