How to assess Big Dataneeds for yourorganizationMar 2013We have covered the following sections in the paper:• Why should a...
“According to the IBM 2010Global CFO Study, over the nextthree years, organizations thatleverage big data willfinancially ...
How can you assess Big Data needs for your organization?Organization maturityEach organization goes through the 5 levels o...
The next section of the paper illustrates on the three areas of concern for an organization, once it hasdecided to estimat...
After the information is collected, it can be put up in a concise 1-slider format for business to takeactions. The various...
Advantages of Big Data evaluation framework• The evaluation framework is relatively simple and easy to understand, adopt a...
Along with that we also looked at the data needs of these teams 2 years back and what was thestrength of the team at that ...
About the authors:Sayantika Bhaduri (Sayantika_Bhaduri@dell.com) is a Sr. Advisor with the Marketing and Sales analyticste...
Upcoming SlideShare
Loading in …5
×

How to Assess Big Data Needs for Your Organization.

1,827 views
1,761 views

Published on

Entering the complex and huge world of Big Data is important for organizations because of the growing data sizes and the increasing needs of businesses to store, process and interpret meaningful insights of the same. Amidst the current wave of discussions around Big Data, organizations need to ascertain the extent to which they can adopt the new buzzword. To be aware of the emerging trends in the market, and not to be caught unaware when the data explosion hit, organizations can assess their Big Data needs using the framework defined and illustrated in this paper.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,827
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
31
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

How to Assess Big Data Needs for Your Organization.

  1. 1. How to assess Big Dataneeds for yourorganizationMar 2013We have covered the following sections in the paper:• Why should an organization assess its big data needs,• What framework to use to assess big data needs, along-with a case study, and• How to implement the solutions in the organization.By Anirban Sinha & Sayantika BhaduriEntering the complex and huge world of Big Data is important for organizations because ofthe growing data sizes and the increasing needs of businesses to store, process and interpretmeaningful insights of the same. Amidst the current wave of discussions around Big Data,organizations need to ascertain the extent to which they can adopt the new buzzword. To beaware of the emerging trends in the market, and not to be caught unaware when the dataexplosion hit, organizations can assess their Big Data needs using the framework defined andillustrated in this paper.
  2. 2. “According to the IBM 2010Global CFO Study, over the nextthree years, organizations thatleverage big data willfinancially outperform theirpeers by 20 percent or more.McKinsey, in one of its reports,mentioned, US healthcare coulduse big data creatively andeffectively to drive efficiencyand quality, with a potentialvalue add of $300 billion everyyear. A retailer can increase itsoperating margins by 60%,through effective use of bigdata”.Why do you need to assess your organization’s Big Dataneed?The Gen Y era has seen paradigm shift in the way people look at information. They were born at atime when information was manageable and primarily stored through paper medium. Usages ofaudio and video mediums were restricted to only creative fields like music and cinematography.People who had power had information and access to information was restrictive in nature. Thencame a time when nature of communication started changing. Global companies expanded theirfootprints and information started molding into global form. Fast and easy access to information anddata were perceived as game changers and now people who had access to information rose to power.Telecom and IT industries went through fast evolutions and the sudden explosion of information wasfelt everywhere.The burst of information started long back, but was realizedwith the emergence of Google and Yahoo, and laterfollowed by social media sites like Facebook, Twitter, etc.With time, internet became more popular and thepenetration increased multi-folds. Consequently, theamount of information being created everywhere becamehard to store. The need to store and manage data gave birthto a whole new concept of data management and datacenters. As science and technology developed, advancedtechniques for data storage and management evolved.All the above trends led to increased storage of data.Entrepreneurs started mining the data to predict businessgrowth, understand customer behavior and drive revenue.With that started a whole new world of data analytics,where every single business in the world startedunderstanding the benefits of mining data.With the growth of data and analytics around it,technologists developed ways to efficiently store andmanage data. The only way for businesses to keep up withthe ever growing data is to explore beyond their traditionaldatabase management tools.This paper tries to provide insights on how data growth can be handled and how organizations canplan for implementing “Big Data” in the coming years.
  3. 3. How can you assess Big Data needs for your organization?Organization maturityEach organization goes through the 5 levels of maturity in terms of their data environment as shownbelow.While moving from a reactive mindset towards proactive and strategic to the point of being real-time,at some point, an organization might feel that there is a need to think of new solutions to efficientlystore and process large chunks of varied datasets collected at a fast speed, to provide the best valueto the organization. While traditional ways can support the organization in all the stages, adoption ofBig Data tools and technologies might provide an edge to the organization, if need be.According to the conversations we had with a few business leaders across organizations, in most ofthe cases the discussion around the need for change in terms of data management, starts only whenthere is a data intensive project that has come up and/or the current infrastructure/processes doesnot support it. During this phase, identifying alternate method of crunching the data, implementingthe solution and getting the resources ready for the project becomes a challenging task. The paperexplores ways to understand the big data needs for an organization.During the study, we started off with similar issues faced by the 400 odd Dell Global Analytics teamand approached various practices / sub teams to understand the best ways of capturing the latentneed for planning the next steps on Big Data. In the matured state of the organization, while beingproactive and strategic, the teams were still using a mix of traditional and modern analytics. Theanswers they were looking for was the amount of investment on technology and peoplehiring/training to adopt big data and the trade-off in terms of benefits.To answer these we came out with a framework, where by using some simple questions anorganization can capture the future needs of each practice and then take an informed decisionwhether they need to invest on planning for Big Data now/immediate future or buy some time to getprepared for future changes in the organization.Data Growth DataInfrastructureDataManagementApplications &AnalysisData & BusinessStrategyDEVELOPData growthwithout direction& control asbusinesses growREACTIVEData format andstructure emerges– Infrastructuresupport evolvesSUSTAINGovernance &QualityframeworkdevelopsPROACTIVEReportingtranspires arounddata – ITtransformationfor adequatesupportSTRATEGICDecision supportand strategybased oncomplex datamodeling
  4. 4. The next section of the paper illustrates on the three areas of concern for an organization, once it hasdecided to estimate the requirements of Big Data internally.Three tracks of Big Data needsThere are three areas which need to be taken into consideration when an organization decides to movetowards Big Data.Preparing for Big DataThis section illustrates the framework and the background on which it was developed by us and pilotedin Dell Global Analytics.The three-step process of developing the evaluation framework is discussed below:While trying to understand the needs of an analytical organization to imbibe the values of Big data, wecreated a question bank to get answers for. The evaluation framework created based on the questionsformed the basis for interviewing the leaders of various practices and the database / technology SMEsto evaluate the maturity and need of the organization.The one-month long process of interviewing took place at various levels of hierarchy within theorganization. Below diagram depicts the evaluation framework that can be leveraged to ascertain theneeds of an organization in terms of Big Data.4 Vs of Big Data Infrastructure & Applications PeopleQuestions needed toevaluate need for BigDataPreparing properevaluation FrameworkEvaluating leaders andtechnology expertswithin teamsOrganizations need toevaluate whether any of theimmediate, near-term orfuture projects align to the 4Vs – Volume, Velocity,Variety and Value of BigData while addingconsiderable value to thebusinessOne of the needs of BigData is to invest time andmoney on related hardware(servers / storages),platforms (map-reduce) andapplications (analytics &visualization)With the adoption of BigData in an organization,need to upskill (tools andmethodologies) existingresources or hire new skillsets ( Data scientists /Visualization engineers)increases
  5. 5. After the information is collected, it can be put up in a concise 1-slider format for business to takeactions. The various sections in the investigation template highlight how the practice is placedcurrently in terms of Big Data environment, why any future planned project is important and classify fora Big Data title, the current level of infrastructure support from the organization and if we haveresources ready for each of the projects.Opportunity assessment framework (1 Slider)Analytics Offeringand Big DataMapping•Which of the 4Vsdefine the data usagenow & planned-Volume, Velocity,Variety & Value?•Are there projectsplanned in futurewhich would require‘Big Data’infrastructure,platforms orapplications?•Are there projectswhich are currentlylimited due to non-availability ofinfra/platform orapplications?Storage Needs•Is the storage spaceadequate for thefuture?•Do we needadvanced filedistribution platformand parallelprocessing in thefuture?•Is the source of data‘Big Data’ ready?•Are there any ITlimitations on datapull?Analytics ToolNeeds•What are the ‘BigData’ applicationscurrently used andplanned for future?•Are there ‘Big Data’applications whichcan berecommended forfuture analyticalneeds with mostdesirable impact?People Needs•Is there a gap inresource skills to be‘Big Data’ ready?•How many resourcesare needed to be ‘BigData’ ready in thefuture?•Do they need internal/external trainings formaking them ready?•Is there any skill forwhich external hire isnecessary?
  6. 6. Advantages of Big Data evaluation framework• The evaluation framework is relatively simple and easy to understand, adopt and replicate• This framework provides a strong foundation to compare and assess various internal practicesand understand if there are specific teams with higher requirements in terms of Big Data andinvest accordingly• The framework provides the base for understanding the potential of a project ongoing/ plannedand enables the respective teams to plan accordingly to prepare for the future and avail therequired support• The template covers various aspects of understandingo Current and planned future state of the organizationo Storage, tools and applications needed, if anyo Applicability of the 4 Vs of Big dataLimitations• The evaluation framework provides the base study and recommends the best possible area ofadoption of Big Data within an organization. However, it is quite high-level and does not providedetails around the execution or implementation of any of the Big Data values• Many a times, the leaders of the organization might not be privy of the extent of projectsplanned in the future. The success of the framework depends on the depth of the vision theleaders bring to the table in terms of future planned projects and their needs• The framework does not take into consideration the quantitative definition of the 4 Vs of BigData. However, it urges the organizations to qualitatively determine whether the currentinfrastructure / platforms or tools are capable enough to handle the 4 Vs of data• Since, the framework provides the base for discussion and movement towards the next leveldeep dive, it does not provide any insights on the estimated cost of changeApplicationsThe evaluation framework can be used across a range of industries, where deliverables are in the formof projects and data is stored internally for analysis. The framework can be used in a variety oforganizations where analytics form a major part, like, technology, healthcare, web / online, retail etc.Next StepsWe plan to work on refining the framework to include the ballpark cost of change involved in theprocess.How DGA defined this framework?We started the exercise with making a list of all the analytical practices which are working on dataintensive projects or have a scope in future. Then we had a 3 tiered interviewing process for the teammembers.Step 1 – Identify the data experts in the team and understand the present data needs of the team likeand amount of storage space they are using, how much of the space is housed locally and what portionis shared over central storages/servers. We also asked them about the different kinds of data files theystore and how many of them prefer using their workstations to run some of these processes.
  7. 7. Along with that we also looked at the data needs of these teams 2 years back and what was thestrength of the team at that time. Basis these inputs we will know that if they did not get any newprojects and continue of the existing ones what would be their storage needs in a couple of years.This is the baseline and basis the available servers we will be able to assess to total space utilizationby these teams.Step 2 – Talk to the leaders of each practice and ask them about the future projects that they seecoming to their teams in the next 2 years. These predictions can either be basis discussions they hadwith partners during their planning sessions or basis areas DGA wants to work on in future which willadd a lot of value to Dell.Step 3 – Talk to each of the project managers of these teams and evaluate the project requirementsin terms of the 4Vs, Infrastructure support they need and people skills. Considering all these inputs,evaluate the Big Data need for each of these teams. Distill each of the practice needs and prioritizebasis the urgency and budget of the organization.After you assess the Big Data need, how do you implement?Once you have established a need to implement Big Data solutions for your organization the nextquestion would be to understand the different options available. There is no absolute answer for anyorganization today. The options are many –• Cloud solutions/Virtualization,• Opting for a POC for a particular project• Restructuring the IT infrastructure• Outsourcing.All these decisions need to be taken basis the 4 factors mentioned below –1. Budget2. Time available to implement3. People available to work/implement Big Data solution4. Data confidentialityIf the need is immediate and if your organization doesn’t have the time to build infrastructure tosupport it you can think of either outsourcing this or a cloud based solution. Clouds mostly workbetter when we are dealing with the Volume aspect of Big Data. But if this requires analyzing thedata and insight generation then in such a scenario, outsourcing would be the best option.If data security is the most important factor, and the Big Data needs are consistent over a period oftime, the best bet would be to invest in infrastructure and re-engineer the process.If the processing time is the only challenge, we suggest opting for a POC to estimate the time savedbefore investing heavily on new techniques/tools.All of these are plausible solutions and the market is still exploring the different pros/cons of each ofthese options. But what works for an organization will be determined basis how effectively theorganization identifies its future needs and the amount of money it is ready to invest on Big Datasolutions.
  8. 8. About the authors:Sayantika Bhaduri (Sayantika_Bhaduri@dell.com) is a Sr. Advisor with the Marketing and Sales analyticsteam in Dell Global Analytics and based out of Bangalore, India. She holds a Masters in Mathematicsfrom IIT Kanpur, with experience in Marketing Analytics for technology industry.Anirban Sinha (Anirban_sinha@dell.com) is an Advisor with the Contact Center and Services analyticsteam in Dell Global Analytics and based out of Bangalore, India. He holds Bachelors (major) inAerospace Engineering and minor in Mathematics and Computing from IIT Kharagpur and has worked inPrivate Equity, Consulting and Analytics previously.About Dell Global AnalyticsDell Global Analytics seeks to improve Dell’s bottom line through the leveraged use of analyticstouching all aspects of Dell’s business operations. We offer a wide range of analytics services coveringmanagement reporting and dash boarding of key business metrics, forecasting and predictive customerresponse modeling and optimization of key business processes. Value for Dell is unlocked by theapplication of sophisticated data analysis, statistical and mathematical techniques under a Six-Sigmaframework of business process improvement.The range of Dell functions supported includes Supply Chain, Pricing, E-Commerce, Contact CenterOperations, Financial Services, Marketing and Sales.Office:Dell International Services India Pvt. LtdDivyashree Greens,Survey No 12/1, 12/2A, 13/1A, Challaghatta, Varthur Hobli,Bangalore 560071, INDIA

×