Planning Your Hadoop NoSQL Projects For 2011


Published on

Recorded version available at

Hadoop and NoSQL have made their mark on Yahoo, AOL, Facebook, Digg and Twitter. Are these game-changing technologies starring in your projects for 2011?

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Planning Your Hadoop NoSQL Projects For 2011

  1. 1. Impetus Technologies Inc. Planning your Hadoop/NOSQL © 2014 1 Impetus Technologies Projects for 2011 Recorded version available at
  2. 2. Agenda • Why Hadoop/NoSQL – a primer! • Challenges of moving to big data solutions- Hadoop, MPP and/or NoSQL • Requirement assessment and planning for the right solution • An extension to existing IT strategy or a complete migration? © 2014 2 Impetus Technologies Recorded version available at
  3. 3. The Bottlenecks • Analyze Hadoop/NoSQL fitment • Ease its adoption for business benefits • Minimize risk • Make the transformation seamless and disruption free • Alignment with IT strategy © 2014 3 Impetus Technologies Recorded version available at
  4. 4. The Failure – Proof Way Criteria Costing Implementation Hadoop/ NoSQL Fitment analysis Best Fit solution Create POC © 2014 4 Impetus Technologies Operational Cost Analysis Deployment Cost Analysis Team Training Cost Solution Implementation Production Readiness Deployment Recorded version available at
  5. 5. Hadoop/NOSQL Fitment Analysis Challenges/Factors • Can my business problem be solved using Hadoop/NoSQL? • R&D is costly and resources are limited • What is the best solution among so many options? • Can I utilize ‘Clouds’ for operational costs? Approach • Identify core pain areas, diagnose and choose suitable Big Data solution • Leverage knowledge of experienced Hadoop/NoSQL professionals for compute & data intensive solutions © 2014 5 Impetus Technologies Recorded version available at
  6. 6. Best Fit Solution Identification Challenges/Factors • Hadoop/NoSQL have numerous solutions- which is the right one for me? • Would the solution be cost effective? Approach • Provide early associated costing and deployment strategies • Use technical expertise on Hadoop/NoSQL technologies • Compute intensive and/or Data intensive? • Maximize reuse of existing resources © 2014 6 Impetus Technologies Recorded version available at
  7. 7. Verification with POC Challenges/Factors • How to make sure the solution can be implemented? • How to verify that the solution is cost effective? • Batch processing v/s real time solution? Approach • Provide a working prototype addressing key issues • Does the solution work? • Is it cost effective? • Is it production ready? © 2014 7 Impetus Technologies Recorded version available at
  8. 8. Deployment Cost Analysis Challenges/Factors • Cost involved in development, release and hosting? • Hosting cost on Cloud- Server, RAM, CPU, Storage • Any specific hardware? Approach • Analyzing operational cost of hosting on Cloud • Impetus’ Hadoop deployment toolkit • Leverage knowledge and best practices © 2014 8 Impetus Technologies Recorded version available at
  9. 9. Operational Cost Analysis Challenges/Factors • Recurring hardware/software cost? • Support and maintenance cost? Approach • Operational scalability & efficiency analysis • Operational cost analysis © 2014 9 Impetus Technologies Recorded version available at
  10. 10. Training Cost Analysis Challenges/Factors • Is my IT/Ops team trained to manage development/ deployment? • Hire new resources or train existing team? Approach • Hadoop is map-reduce APIs built atop of Java- can be easily learned by programmer • Will not take long to train the team • NoSQL database design and implementation training © 2014 10 Impetus Technologies Recorded version available at
  11. 11. Solution Implementation Challenges/Factors • Can we use traditional development processes in Hadoop/NoSQL? • How to ensure that proper quality guidelines are being followed? Approach • Development cycle – use proven product development lifecycle management process • Plan for seamless transformation and disruption free operations © 2014 11 Impetus Technologies Recorded version available at
  12. 12. Product Readiness & Deployment Challenges/Factors • How to prepare for production readiness? • How to tune production environment? • How is engineering going to cope with complex implementations? Approach • Proper expertise needed for production deployment • Production readiness checks • Production environment tuning/optimizations • Proper training/handover to engineering team for production deployment © 2014 12 Impetus Technologies Recorded version available at
  13. 13. The Big Picture Business Strategy © 2014 13 Impetus Technologies Infrastructure Development Testing BI and Analytics Data Warehousing Enterprise Architecture Impact on IT Strategy Recorded version available at
  14. 14. What Changes Do I Need To Make Extension to current IT Strategy • Existing code reusability(business logic) • Minor modifications in test strategy • No major changes in release and build management of Hadoop/NoSQL module • Quick setup for development/test/production environments Migration • Complete DW platform • Data storage and archival • BI and Analytics • Application redesign © 2014 14 Impetus Technologies Recorded version available at
  15. 15. The Bottom Line • Hadoop is NOT a new technology but a different programming paradigm • NoSQL is a non-normalized way of data modeling © 2014 15 Impetus Technologies Recorded version available at
  16. 16. Case Study The Challenge • Where to use Hadoop • Low RoI on Hadoop infrastructure • Issues in Hadoop optimization and performance Our Approach • Assessment of the products that actually require Hadoop/ NoSQL • Replaced low latency read-write to NoSQL • Continued the use of ETL and used HDFS © 2014 16 Impetus Technologies The Client • Global leader in internet • services and media Client was able to reuse existing investments Recorded version available at
  17. 17. Case Study The Challenge • Product targeting an enormous user base • Data size ranging in TBs/PBs • Traditional solutions could not address the required data sizes Our Approach • Consultation to determine right Big Data solution • Used our SPDMM methodology to validate NFR • Proposed a suitable NoSQL solution © 2014 17 Impetus Technologies The Client A Fortune-500 enterprise NoSQL solution saved cost for the client Recorded version available at
  18. 18. © 2014 18 Impetus Technologies About Impetus
  19. 19. • Strategic partners for software product engineering and R&D • Thought leaders in cutting-edge technologies • Mature processes and practices that are methodical, yet flexible • Diverse domain expertise © 2014 19 Impetus Technologies Recorded version available at
  20. 20. © 2014 20 Impetus Technologies Q & A
  21. 21. © 2014 21 Impetus Technologies Thank You Write to us at Follow us on Twitter @impetustech Recorded version available at