Choose the Right Tools for Big Data Development


Published on

Leverage Hadoop as your pilot project to gain organizational buy-in and build institutional learning.

Your Challenge
A relational database management system works great under many scenarios, but it has its limitations:
Volume issues typically arise when there is a need to index large databases.
In a multi-source environment, data collisions occur and resolving them can be expensive and time consuming.
Velocity problems arise when large amounts of read/write transactions occur that are expensive to compute.

Our Advice

Critical Insight

Begin your big data implementation with a baseline Hadoop pilot. This pilot will help build your knowledge of big data, how the Hadoop framework satisfies your use cases, and how it operates in your system. Each component in this baseline stack is well understood in the industry and documentation is readily available.

Impact and Result

Provide a step-by-step starting point to begin the rollout of big data development based on your business and technical requirements.
Highlight the challenges, impacts, potential, and mitigations in big data development.
Identify the key metrics, benchmarks, and instrumentation points to measure the success of your big data rollout.

Published in: Data & Analytics
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Choose the Right Tools for Big Data Development

  1. Choose the Right Tools for Big Data Development Leverage Hadoop as your pilot project to gain organization buy-in and build institutional learning. This research is designed for data scientists, app development managers, and developers who: Need to analyze vast quantities and varieties of data quickly. Are looking for strategies to decrease data storage and computing power. Are seeking learning opportunities to apply big data development practices within the organization and other development projects. Data is growing exponentially and the business community feels the need to adopt and service value from big data to stay competitive, but this isn’t easy. Start with a Hadoop pilot to gather institutional learning and validating its fit. Schools of thought for your Hadoop pilot: Custom: Install each Hadoop component by yourself for maximum learning. Stack: Use a pre-fabricated stack with all components integrated for maximum speed. Cloud: Use a cloud vendor with individual components for maximum elasticity. Drivers/Trends/Change: Data is growing exponentially. “CSC estimates that data production will be 44 times greater in 2020 than it was in 2009.” (Computer Sciences Corporation (CSC), 2012) The need to analyze vast quantities and a variety of data quickly is imperative; “50.6% of respondents indicated speed of processing response and 41.3% indicated combining data structure as their initiative big data use case in 2012.” (Enterprise Management Associates (EMA), Operationalizing the Buzz: Big Data 2013, Nov 2013) There is no single product that does all of big data well. It is fragmented. Drastic decrease in cost of storage and computing power. Value Creation: Allows the business to make decisions using vast quantities of data across multiple data sources. See Project Steps Table Below Style Guide - Designers: Use the style guide to get a feel for our brand Rough Sketch This infographic is the creative art of our research. As such, we want to make sure each one is unique and creative. The only real guideline is our fixed width and call to action image. The rest is up to you. 1) Big Data Project Plan – Excel Tool 2) Big Data Business Requirements Template – Word Template 3) 4) 5) 6) 1) Assess your fit and readiness for big data 2) Prepare and roll out your Hadoop pilot 3) Roll out Hadoop in your organization 4) 5) 6) Choose the Right Tools for Big Data Development
  2. Step 1 Assess Your Fit and Readiness for Big Data Your fit for big data will hinge on your current data flow and whether your data stack can accommodate new entities while remaining intact. Step 2 Build Your Project Team Give your team time to understand Hadoop and create a team of data, operations, project management, and product owners to maximize learning potential. Step 3 Roll Out Your Hadoop Pilot Don’t rush into big data; understand what it can and cannot do for the business before implementing a major rollout. Not every project needs to incorporate big data technologies, especially where security is paramount. Step 4 Roll Out Hadoop in Your Organization Your list of metrics will build over time as you begin to conduct performance optimization activities to identify the fit of Hadoop in other areas of your organization.