The deciding factor: Big data & decision making


Published on

The deciding factor: Big Data and decision-making is a report published by Capgemini and written by the Economist Intelligence Unit. The report examines how far down the road firms in different industries and regions are in utilising big data, and sheds light on the steps some organisations are taking to make data a critical success factor in the decision-making process.

The analysis in the report is based on two major research initiatives:

The Economist Intelligence Unit conducted a survey, completed in February 2012, of 607 executives. Participants hailed from across the globe, with 38% based in Europe, 28% in North America, 25% in Asia-Pacific, and the remainder coming from Latin America and the Middle East and Africa. The sample was senior, 43% of participants being C-level and board executives and the balance—other high-level managers such as vice-presidents, business unit heads and department heads. Respondents worked in a variety of different functions and hailed from over 20 industries. Of the latter, the best represented were financial services, professional services, technology, manufacturing, healthcare and pharmaceuticals, and consumers goods and retail.

To supplement the survey, the Economist Intelligence Unit conducted a series of in-depth interviews with senior business executives as well as independent experts on data and decision-making.

Published in: Business, Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The deciding factor: Big data & decision making

  1. 1. Business Analytics The way we see itThe Deciding Factor:Big Data & Decision MakingWritten by
  2. 2. The Deciding Factor: Big data and decision-makingForewordBig Data represents a fundamental shift in business decision- The survey also highlights special challenges for decision-making. Organisations are accustomed to analysing internal making arising from Big Data; although 85% of respondentsdata – sales, shipments, inventory. Now they are increasingly felt the issue was not so much volume as the need to analyseanalysing external data too, gaining new insights into and act on Big Data in real-time. Familiar challenges relatingcustomers, markets, supply chains and operations: the to data quality, governance and consistency also remainperspective that Capgemini calls the “outside-in view”. We relevant, with 56% of respondents citing organisational silosbelieve it is Big Data and the outside-in view that will generate as their biggest problem in making better use of Big Data.the biggest opportunities for differentiation over the next five For our respondents, data is now the fourth factor ofto ten years. production, as essential as land, labour and capital. It follows that tomorrow’s winners will be the organisations that succeedThe topic of Big Data has been rising rapidly up our in exploiting Big Data, for example by applying advancedclients’ agenda, and Capgemini is already undertaking predictive analytic techniques in real time.extensive work in this area all over the world. That is why wecommissioned this survey from the Economist Intelligence I would like to thank the teams at the Economist IntelligenceUnit: we wanted to find out more about how organisations are Unit and within Capgemini, along with all the surveyusing Big Data today, where and how it is making a difference, respondents and interviewees. I believe this research will doand how it will be used in the future. much to increase understanding the business impact of Big Data and its value to decision-makers.The results show that organisations have already seenclear evidence of the benefits Big Data can deliver. Surveyparticipants estimate that, for processes where Big Data Paul Nannettianalytics has been applied, on average, they have seen a 26%improvement in performance over the past three years, and Global Sales and Portfolio Directorthey expect it will improve by 41% over the next three.2
  3. 3. The Deciding Factor: Big data and decision-makingAbout the Research 43%Capgemini commissioned the The Economist Intelligence UnitEconomist Intelligence Unit to write The conducted a survey, completed inDeciding Factor: Big data and decision- February 2012, of 607 executives.making. Participants hailed from across the globe, with 38% based in Europe, 28%The report is based on the following in North America, 25% in Asia-Pacificresearch activities: and the remainder coming from Latin America and the Middle East and of participants are C-level Africa. The sample was senior, 43% of and board executives participants being C-level and board executives and the balance—other high-level managers such as vice- presidents, business unit heads and department heads. Respondents worked in a variety of different functions and hailed from over 20 industries. Of the latter, the best represented were financial services, professional services, technology, manufacturing, healthcare and pharmaceuticals, and consumers goods and retail. To supplement the survey, the Economist Intelligence Unit conducted a programme of interviews with senior executives of organisations as well as independent experts on data and decision-making. Sincere thanks go to the survey participants and interviewees for sharing their valuable time and insights.3
  4. 4. The Deciding Factor: Big data and decision-makingExecutive summaryWhen it comes to making business At the same time, practitioners query unstructured data, such as textdecisions, it is difficult to exaggerate interviewed for the report—all analytics and sentiment analysis. Athe value of managers’ experience enthusiastic about the potential large number of executives protest thatand intuition, especially when hard for big data to improve decision- unstructured content in big data is toodata is not at hand. Today, however, making—caution that responsibility difficult to interpret.when petabytes of information for certain types of decisions, evenare freely available, it would be operational ones, will always need Although unstructured datafoolhardy to make a decision to rest with a human being.without attempting to draw some causes unease, social mediameaningful inferences from the data. Other findings from the research are growing in importance. include the following:Anecdotal and other evidence is Social media tell companies not only what consumers like but, moreindeed growing that the intensive use The majority of executives importantly, also what they don’tof data in decision-making can leadto better decisions and improved believe their organisations like. They are often used as an earlybusiness performance. One academic to be “data driven”, warning system to alert firms whenstudy cited in this report found that, but doubts persist. customers are turning against them.controlling for other variables, firms Forty-three percent of respondentsthat emphasise decision-making based Fully two-thirds of survey respondents agree that using social media to makeon data and analytics have performed say that the collection and analysis of decisions is increasingly important.5-6% better—as measured by output data underpins their firm’s business For consumer goods and retail,and performance—than firms that strategy and day-to-day decision- manufacturing, and healthcare andrely on intuition and experience for making. The proportion of executives pharmaceuticals firms, social mediadecision-making. Although that study who say their firm is data-driven is provide the second most valuedexamined “the direct connection higher in the energy and natural datasets after business activity data.between data-driven decision-making resources (76%), financial servicesand firm performance”, it did not (73%), and healthcare, pharmaceuticals The job of automating and biotechnology sectors (75%).question the size of the data-sets They may not be as data-savvy as decision-making isused in decision-making. In fact, very their executives think, however: far from over.little has been written about the useof “big data”—which is distinguished majorities also believe that big data management is not viewed strategically Automation has come a long way, but aas much by its large volume as by majority of surveyed executives (62%)the variety of media which generate at their firm, and that they do not have enough of a “big data culture”. believe there are many more typesit—for decision-making. This report is of operational and tactical decisionsan attempt to address that shortfall. that are yet to be automated. This Organisations struggle is particularly true of heavy industryThe research confirms a growing to make effective use where regulation and technology haveappetite among organisations for data of unstructured data held automation back. There is, to beand data-driven decisions, despite theirstruggles with the enormous volumes for decision-making. sure, a limit to the decisions that can be automated. Although technical limitsbeing generated. Just over half of are constantly being overcome, theexecutives surveyed for the report say Notwithstanding the heavy volumes, increasing demand for accountability—that management decisions based one-half of executives say they do especially following the financialpurely on intuition or experience are not have enough structured data to crisis—means that important businessincreasingly regarded as suspect, and support decision-making, compared decisions must ultimately rest with atwo-thirds insist that management with only 28% who say the same about human, not a machine. For less criticaldecisions are increasingly based on unstructured data. In fact, 40% of or risky decisions, however, there is still“hard analytic information”. Nine in respondents complain that they have much scope for decision-automation.ten of the executives polled feel that too much unstructured data. Mostthe decisions they’ve made in the past business people are familiar withthree years would have been better if spreadsheets and relational databases,they’d had all the relevant data to hand. but less familiar with the tools used to4
  5. 5. The Deciding Factor: Big data and decision-makingThis is particularly true of machine-to-machine communication, wherelow-risk decisions, such as whether toreplenish a vending machine or not, willincreasingly be made without humanintervention.Organisational silosand a dearth of dataspecialists are the mainobstacles to putting bigdata to work effectivelyfor decision-making.Data silos are a perennial problem,and one which the business processreengineering revolution of the1990s failed to resolve. Regulationand the emergence of “trusted dataaggregators” may help to break downtoday’s application silos, however.Arguably a longer term challenge isthe lack of skilled analysts. Technologyfirms are working with universities tohelp train tomorrow’s data specialists,but it is unlikely that supply willmeet demand soon. In the nearfuture, there is likely to be a “war fortalent” as firms try and outbid eachother for top-flight data analysts.5
  6. 6. The Deciding Factor: Big data and decision-makingIntroduction26% Moneyball: The Art of Winning an Unfair resources. Although financial services Game, by Michael Lewis, is the story of and healthcare firms have long been an underperforming American baseball big data users—where big data is team—the Oakland Athletics—that defined by its enormous volume turned a losing streak into a winning and the great diversity of media streak by intensively using statistics and which generate it—heavy industry analytics. According to the New York appears to be catching up (see caseis the extent of performance Times, the book turned many business study: GE—the industrial Internet).improvement already people into “empirical evangelists”1.experienced from big data. Nine in ten survey respondents agree An Economist Intelligence Unit survey, that data is now an essential factor of41% supported by Capgemini, of 607 senior production, alongside land, labour and executives conducted for this report capital. They are also optimistic about found that there is indeed a growing the benefits of big data. On average, appetite for fact-based decision- survey participants say that big data making in organisations. The majority has improved their organisations’ of respondents to the survey (54%) say performance in the past three years that management decisions based by 26%, and they are optimistic that purely on intuition or experience are it will improve performance by anis the performance increasingly regarded as suspect (this average of 41% in the next threeimprovement expected view is held even more firmly in the years. While “performance” in thisin the next three years. manufacturing, energy and government instance is not rigorously specified,55% sectors), and 65% assert that more it is a useful gauge of mood. and more, management decisions are based on “hard analytic information”. One may question whether the surveyed firms are as “data-driven” Until recently there was scant research as their executives say. The research to back the Moneyball hypothesis—that also shows that organisations are if organisations relied on analytics for struggling with the enormous volumes decision-making they could outperform of data and often with poor qualitysay that big data their competitors. In 2011, however, data, and many are struggling to freemanagement is not viewed Erik Brynjolfsson, an economist at the data from organisational silos. The Sloan School of Management at the same share of respondents who saystrategically at senior levels Massachusetts Institute of Technology their firms are data-driven also sayof their organisation. (MIT), along with other colleagues there is not enough of a “big data studied 179 large publicly traded culture” in their organisation; almost firms and found that, controlling for as many – 55% – say that big data other variables, such as information management is not viewed strategically technology (IT) investment, labour and at senior levels of their organisation. capital, firms that emphasise decision- making based on data and analytics When it comes to integrating big data performed 5-6% better—as measured with executive decision-making, there by output and performance—than is clearly a long road to travel before those that rely on intuition and the results match the optimism. This experience for decision-making2. report will examine how far down that1 road firms in different industries andmoneyball-data-guys-are-triumphant.html Two-thirds of the executives in the regions are, and will shed light on the survey describe their firm as “data- steps some organisations are taking to2 Brynjolfsson, Erik, Hitt, Lorin M. and Kim, Heekyung driven”. That figure rises to 73% make big data a critical success factorHellen, “Strength in Numbers: How Does Data-Driven for respondents from the financial in the decision-making process.Decision making Affect Firm Performance?” (April 22, services sector, 75% from healthcare,2011). Available at SSRN: pharmaceuticals and biotechnology,or and 76% from energy and natural6
  7. 7. The Deciding Factor: Big data and decision-making On average, respondents believe that big data will improve organisational performance by 41% over the next three yearsSurvey Question: Approximately to what extent do you believe that the use of big data has improved yourorganisation’s overall performance already, and can improve overall performance in the next three years? Now 3 Years45%40%35%30%25%20%15%10%5% Average CEO/President CFO/Treasurer CIO/CTO7
  8. 8. The Deciding Factor: Big data and decision-making Overall, 55% of respondents state that they feel big data management is not viewed strategically at senior levels of their organisationSurvey Question: To what extent do you agree with the following statement:“Big data management is not viewed strategically at senior levels of the organisation.” Strongly Agree Agree Disagree Strongly Disagree Don’t know/Not applicable100%80%60%40%20%0% Total Financial Energy & Consumer Health & Manufacturing Sector Resources Pharmacy Two thirds of executives believe that there is not enough of a “big data culture” in their organisation - this is particularly notable across the manufacturing sectorSurvey Question: To what extent do you agree with the following statement:“There is not enough of a “big data culture” in the organisation, where the use of big data in decision-making isvalued and rewarded.” Strongly Agree Agree Disagree Strongly Disagree Don’t know/Not applicable100%80%60%40%20%0% Total Financial Energy & Consumer Health & Manufacturing Sector Resources Pharmacy8
  9. 9. The Deciding Factor: Big data and decision-makingPutting big datato big use“A lot of people will say data is To keep customers loyal, retailersimportant to their business, but I think have to target customers withit’s incredibly important to healthcare personalised loyalty bonuses,and it’s probably getting more and discounts and promotions. Today, mostmore important,” says Lori Beer large supermarkets micro-segmentexecutive vice president of executive customers in real time and offer highlyenterprise services at WellPoint, an targeted promotions at the point ofAmerican healthcare insurer. Ms Beer sale.compares data in healthcare with“oxygen”—without it, the organisationwould die. Business activity data and point-of-sale data areWellPoint has 34 million members, and considered most valuable across the consumermaking sure their customers get the goods & retail sectorright diagnosis and receive the righttreatment is vital for keeping costsunder control. But getting to the right Survey Question: Which types of big data sets do you see as adding the mostinformation to make the right decision value to your organisation?in healthcare is no mean feat. Thereare terabytes to sift through: millions [select up to three options]of medical research papers, patientrecords, population statistics and Total Consumer goods & retail Top 3formularies, to name a few types ofneeded information. Using that to makean effective decision requires powerful 68.7% 32.0% 27.7% 25.2% 21.9% 18.6% 15.5% 15.5% 10.2% 8.1% 4.3%computing and powerful analytics (seeWellPoint case study). 57.9% 7.9% 42.1% 71.1% 18.4% 21.1% 13.2% 10.5% 5.3% 7.9% 0.0%There is near consensus acrossindustries as to which big data setsare most valuable. Fully 69% of surveyrespondents agree that “businessactivity data” (eg, sales, purchases,costs) adds the greatest value totheir organisation.The only notableexception is consumer goods and retailwhere point-of-sale data is deemed tobe the most important (cited by 71% ofrespondents). Retailers and consumer Business activity data Office documentation (emails, document stores) Social media Point-of-sale Website clickstream data Website clickstream data Geospatial data Telecommunications data (eg phone or data traffic) Telemetry - detailed activity data from plant/equipment Images / graphics Something not on this list (please specify)goods firms are arguably under morepressure than other industries tokeep their prices competitive. Withsmartphone apps such as RedLaser andAmazon’s Price Check, customers canscan a product’s barcode in-store andimmediately find out if the product isavailable elsewhere for less.10
  10. 10. The Deciding Factor: Big data and decision-making42% Office documentation (emails, media to express their anger at the document stores, etc) is the second charge. Verizon Wireless was prompt most valued data set overall, favoured in responding to the outcry, possibly by 32% of respondents. Of the forestalling customer defection to rival other major industries represented mobile operators. in the survey, only healthcare, pharmaceuticals and biotechnology But not all unstructured data is as easyof survey respondents say differ on their second choice. Here to understand as social media. Indeed,that unstructured content is social media are viewed as the second 42% of survey respondents say thattoo difficult to interpret. most valuable data set, possibly unstructured content—which includes because reputation is vitally important audio, video, emails and web pages—is in this sector, and “sentiment analysis” too difficult to interpret. of social media is a quick way to identify A possible reason for this is that today’s shifting views towards drugs and other business intelligence tools are good at healthcare products. aggregating and analysing structured data whilst tools for unstructured data Over 40% of respondents agree that are predominantly targeted at providing using social media data for decision- access to individual documents (eg making has become increasingly search and content management). important, possibly because they It may be a while before the more have made organisations vulnerable advanced unstructured data tools, such to “brand damage”. Social media are as text analytics and sentiment analysis, often used as an early warning system which can aggregate and summarise to alert firms when customers are unstructured content, become mass turning against them. In December market. This may be why 40% of 2011 it took Verizon Wireless just one respondents say they have too much day to make the decision to withdraw unstructured data to support decision- a $2 “convenience charge” for paying making, as opposed to just 7% who feel bills with a smartphone, following a they have too much structured data. social media-led consumer backlash. Customers used Twitter and other social 40% of respondents believe that they have too much unstructured data to support decision-makingSurvey Question: Looking specifically at your department, how would you characterisethe amount of data available to support decision-making? Too much Enough Not enough Don’t knowStructured Unstructured 7.0% 42.1% 49.8% 1.2% 39.6% 30.8% 27.6% 2.0%11
  11. 11. The Deciding Factor: Big data and decision-makingEnough data or too much?Structured or unstructured, mostexecutives feel they don’t have enoughdata to support their decision-making.In fact, 40% of respondents overall Case study: Big data at the bedsidebelieve the decisions they have made For WellPoint, one of America’s In January 2012, WellPoint beganin the past three years would have been largest health insurers, the problem training the supercomputer for the“significantly better” if they’d had all of of ensuring the right treatment plan is first phase of the project. The pilotthe structured and unstructured data provided for its members is becoming system helps WellPoint nurses reviewthey needed to make their decision. increasingly complex. “Getting and authorise treatment requests fromAnd, despite the fact that respondents relevant information at the point- medical providers. It is an iterativefrom the financial services and energy of-care, when decisions are getting process where the nurses followsectors are more likely than average to made, is the holy grail,” says Lori Beer, the existing procedures, examinedescribe their firm as data-driven, they executive vice president of enterprise the response the system provides,are also more likely than the average business services at WellPoint. and then score it based on how well(46% from financial services, and 48% it does. The feedback is used tofrom energy) to feel they could have By some estimates, the body of educate and fine-tune the systemmade better decisions if the needed medical knowledge doubles every so that it will eventually be able todata was to hand. five years. Coupled with an explosion authorise treatments without human in medical research papers is the intervention.At first blush, this may seem rapid conversion of medical records For the second phase, WellPointcontradictory, given the surfeit of data to electronic format. A physician has has partnered with Cedars-Sinaiand the difficulty organisations face in a pile of digital information to sift Samuel Oschin Comprehensivemanaging it, but Bill Ruh, vice president, through yet, according to Ms Beer, Cancer Institute in Los Angeles tosoftware, at GE sees no contradiction. most healthcare providers spend develop a decision-support system“Because the problems we address are very little time with each patient and for oncologists. It is hoped thatgoing to get more and more complex, only see “a slice of the information”. physicians will be able to reviewwe’re going to solve more complex WellPoint wants to provide all the treatment options suggested by theproblems as a result,” he says. “What we relevant information that a healthcare supercomputer at the point of care.find is the more data we have, the more provider needs, in digestible format, Critically, the system won’t just providewe get innovation in those analytics and at the patient’s bedside. an answer; it will show the oncologistwe begin to do things we didn’t think we the documented medical evidencecould do.” “If you look at the statistics, evidence- that supports the probability of why it based medicine is only applied about believes the answer is accurate.For Mr Ruh, the journey to data 50% of the time,” says Ms Beer. “Thefulfilment will be over when he can put issue we often face is that we’re “It is the physician who makes thea sensor on every component GE sells not really using the most relevant ultimate decision,” says Ms Beer. “Thisand monitor the component in real time. evidence-based medicine in diagnosis is not intended to ever replace theIn this way, any aberrant behaviour can and treatment decisions.” A wrong physician.”be immediately identified and either diagnosis and treatment plan can becorrected through a control mechanism deadly for a patient and very costly for There is no end date for the project,(decision automation) or through human WellPoint. and various decision-support andintervention (decision support). “We’re decision-automation tools will bereally trying to get to what we would call WellPoint had been following developed over time. The intent is‘zero unplanned outages’ on everything the advances of IBM’s Watson that the more the WellPoint system iswe sell,” says Mr Ruh. supercomputer for some time and trained, the more accurate diagnoses realised that the natural-language- and treatment plans will become. If processing abilities of the machine this pans out, it will help to drive down would make it ideal for processing the cost of healthcare in the US, where petabytes of unstructured medical wasted health spending in 2009 was information, and drawing meaningful estimated to be between $600 billion conclusions from it in seconds. and $850 billion.12
  12. 12. The Deciding Factor: Big data and decision-makingThe virtues & risksof automation 58%Data can either support a manager in With corporate clients, however, it ismaking a decision (eg, information on much more difficult. “Suppose thatkey performance indicators displayed a ship cannot leave a port due toon a business intelligence “dashboard”) late payment, and suddenly all theor it can automate decision-making bananas go rotten; from a commercial(eg, an automatic stock replenishment perspective, this involves a much higheralgorithm). According to the survey, on risk because the amounts are muchaverage big data is used for decision larger,” says Mr Knorr. “The human on average use big datasupport 58% of the time, and 29% of the element and review by somebody for for decision support. 29%time it is used for decision automation. larger amounts of money won’t goFor Michael Knorr, head of integration away.”and data services at Citi, a financialservices group, deciding whether to However, the job of automatinguse big data for decision support or decision-making at Citi is far fromdecision automation depends on the over. Mr Knorr says the drive for morelevel of risk. automation comes from the increasing expectations of customers and“In the consumer space, where amounts regulators for rapid decision-making. of the time it is used forare small and if you make an error it’s “If you do not have the right level of decision automation.easy to compensate for that error, then automation in place, that means yourautomation might be applicable,” says costs have increased,” says Mr Knorr. “IfMr Knorr. If there is a “false positive”— there is more data and you haven’t keptthat is, a loan is rejected by the system up with automating, then the number ofbased on various set parameters when items you need to review manually willit should have been approved—the have increased, which means you needsituation can easily be remedied with a more resources and people to do call. This strengthens the business case for automation.”14
  13. 13. The Deciding Factor: Big data and decision-making 60% of respondents dispute the proposition that most operational/ tactical decisions that can be automated, have been automatedSurvey Question: To what extent do you agree with the following statement:“Most operational/tactical decisions that can be automated, have been automated.” 54.1% 48.2% 45.2% 34.2% 27.7% 25.1% 13.9% 15.2% 8.2% 8.9% 6.6% 3.6% 3.9% 3.4% 1.7% North America Europe Asia–Pacific Latin America Middle East & Africa 5.1% 5.1% 16.9% Total 5.0% Strongly Agree 29.1% Agree 35.6% 37.3% 48.7% Disagree 13.5% Strongly Disagree 3.8% Don’t know15
  14. 14. The Deciding Factor: Big data and decision-makingAcross all industries and regions, amajority of survey respondents concurthat there is scope for further decisionautomation at their firm. Over 60% of Case study: General Electric andrespondents dispute the propositionthat “most operational/tactical the industrial Internetdecisions that can be automated, have If the first phase of the Internet was electric vehicle charging stations.been automated.” This view is fairly about connecting people, says Billconsistent across industries, although Ruh, vice president of software at “We are putting more and morefewer healthcare and pharmaceuticals General Electric (GE), then the second sensors on all the equipment that wecompanies agree with the statement phase is about connecting machines. sell, so that we can remotely monitor(52%) than manufacturing companies Some people call this “the Internet of and diagnose each device,” says(68%). (Respondents from the education things”, but Mr Ruh prefers the term Mr Ruh. “This represents a hugesector also appear less certain than “the industrial Internet”. Like many productivity gain, because you usedpeers elsewhere that there is much good ideas, the concept preceded to require a physical presence to knowstill to be automated.) There is some the technology. But now, sensors and what was going on. Now we can sell aregional variation, too. No more than big data analytics have reached a level gas turbine and remotely monitor its54% of executives in Asia-Pacific believe of maturity that makes the industrial operating state and help to optimisethe job of automation is incomplete, Internet achievable. Machines are it.”compared with 71% in western Europe. able to talk to each other over vast distances and make decisions without “Trip Optimizer” is a fuel-savingMr Ruh of GE explains why automation is human intervention. system that GE has developed forfar from complete in his industry: “One freight trains. It takes into accountreason is that many of the environments “When you look at business process a wealth of data, including trackwe operate in are highly regulated, so automation, the main productivity conditions, weather, the speed of thewe have to move at a speed that makes gains have been the low hanging train, GPS data and “train physics”,sense within the regulation,” he says. fruit in the consumer, retail and and makes decisions about how“The second is because the sensors entertainment sectors,” says Mr and when the train should brake. Inand the data weren’t really there to Ruh. “But we have not seen many tests, Trip Optimizer reduced fuelautomate anything.” automation and productivity gains use by 4-14%, according to Mr Ruh. in industrial operations.” National With fuel being one of the biggestCertainly decision-automation tools electricity grids, for example, are some overheads for freight train companieshave evolved from simple “if then of the world’s biggest “machines”, (at Canadian Pacific, one user of GE’selse” programmable statements (eg, yet the fundamentals around how system, it makes up nearly one-quarter“if credit rating = AAA, then approve the technology is used and how it of operating costs), a 10% reduction inloan, else reject”) to sophisticated interacts with other systems have not fuel use represents a huge cost saving.artificial intelligence programs that kept pace over the course of a century.learn from successes and failures. But with sensors, control systems and Mr Ruh likens the industrial InternetThe more sophisticated the tools the Internet, a “smart grid” could to Facebook or Twitter for machines.become, the more decisions that can make decisions, such as which energy Whether it is a jet engine or oil rig,be automated. Decision automation, supply to switch to, or which part of a machine is constantly providinghowever, can introduce unnecessary the network to isolate in the case of a status updates on performance. Bigrigidity into business processes. At fluctuation or disturbance. data analytics look for patterns intimes of high instability—such as the performance, and when an anomalycurrent economic climate—companies In November 2011, GE showed its is identified, a decision about theneed to be nimble in order to adapt to commitment to catching up with the best corrective action is automaticallythe changing conditions. Hard-coded business-to- consumer (B2C) sectors taken or a person is alerted so thatdecisions can be costly and time- by opening a new software centre in a decision can be made on the bestconsuming to change. San Ramon, California, with Mr Ruh as course of action. its head. GE is in the process of hiring 400 software engineers (with 100 on “I believe that we’re in the early stages board to date) to complement the of this,” says Mr Ruh, “and we haven’t company’s 5,000 software workers even begun to imagine the algorithms who are focused on developing we’re going to build and how they’re applications for power plants, going to improve the kinds of products aeroplanes, medical systems and and services we offer.”Brynjolfsson, Erik, "Riding the Rising Information Wave–Are you swamped or swimming?", MIT Sloan Experts, /2011/05/18.16
  15. 15. 17
  16. 16. The Deciding Factor: Big data and decision-makingStanding in the wayThe perceived benefits of The road to these riches, however, continue to do so as the overlapharnessing big data for decision- is laced with potholes. The biggest between different regulatory authoritiesmaking mentioned by the survey impediment to effective decision- is rationalised. “Historically, you couldrespondents are many and varied. making using big data, cited by 56% of say the islands of data provided some survey respondents, is “organisational sort of job security,” says Mr Knorr silos”. This appears especially the case of Citi. “If different areas have their for large firms—those with annual own vernacular, then they keep to Perceived benefits of revenue in excess of $10 billion—whose themselves and avoid transparency. harnessing big data executives are more likely to cite silos as That has obviously broken down, mainly for decision-making a problem (72%) than smaller firms with through the regulatory efforts to ensure less than $500 million in revenue (43%). that the financial services industry can “More complete have a consistent, end-to-end data understanding of model that’s easily understood and market conditions The intractable silos can relate the various transactions and evolving and products across the board.” business trends” The business process reengineering (BPR) movement of the 1990s— Silos may also be eroded over time “Better business led by Michael Hammer and by what Kurt Schlegel, a research vice investment decisions” Thomas Davenport—attempted to president at Gartner, an analyst firm, eradicate function silos. By mapping calls “trusted data aggregators”. He “More accurate and processes (eg, “fulfil order”) that points to aggregators which collect precise responses to ran “horizontally” through several data that different firms (often in customer needs” functions (sales, distribution, accounts the same industry) can access and receivable), duplicated tasks and other analyse for their own purposes. But “Consistency of inefficiencies were identified and Mr Schlegel believes that the trusted decision making eradicated, and data was made to flow data aggregator model can also work and greater group more easily across function boundaries. within organisations themselves. And participation in BPR was given a boost by the arrival even where data protection or privacy shared decisions” of enterprise resource planning (ERP) laws prevent a given department software which automated a number from revealing personal information, “Focusing resources of common business processes. an aggregator could anonymise the more efficiently for However, while BPR undoubtedly data and make it available to other optimal returns” improved efficiency and made the inner departments. machinations of functions visible— 56% “Faster growth often for the first time—the “vertical” of my business function silos were soon replaced by (+20% per year)” “horizontal” application silos. Before, data was trapped in functions; now “Competitive it is trapped in ERP, CRM (customer advantage (new data- relationship management) and SCM driven services)” (supply chain management) systems. of survey respondents cited “Common basis—one To some extent, increasing true starting point regulation, especially in the financial “organisational silos” are for evaluation” services, pharmaceuticals and the biggest impediment to telecommunications industries, has effective decision-making “Better risk begun to erode data silos and will using big data. management”18
  17. 17. The Deciding Factor: Big data and decision-making Across all sectors, “organisational silos” are the biggest impediment to using big data for effective decision-makingSurvey Question: What are your organisation’s three biggest impediments to using big data foreffective decision-making?[Select up to three options] 65.8% 63.0% 59.7% 57.1% 58.2%55.7% 54.3% 54.5% 54.3% 52.6% 50.6% 50.0% 48.4% 44.3% 45.5% 43.7% 43.5% 40.0% 36.8% 37.1% 37.0%Too many “silos”—data is not Shortage of skilled people to The time taken to analyse largepooled for the benefit of the entire analyse the data properly. data sets.organisation. 47.8% 48.4% 49.1% 45.7%41.7% 41.8% 39.1% 36.8% 34.9% 34.3% 34.2% 32.9% 27.4% 24.3% 18.4% 20.0% 17.1% 14.5% 13.0% 13.0% 10.9%Unstructured content in big data Big data is not viewed The high cost of storing andis too difficult to interpret. sufficiently strategically manipulating large data sets. by senior management. 41.3% Total Financial Sector 17.1%14.7% 12.7% Energy & Natural Resources 10.9% 7.9% 8.1% 7.9% 8.1% 4.4% 4.3% 4.3% 1.8% 4.3% Consumer goods & retail IT & Technology ManufacturingBig data sets are too complex to Something not on this Healthcare & Pharmacycollect and store. list (please specify).19
  18. 18. The Deciding Factor: Big data and decision-makingFinding the right skillsThe second big impediment to making and mathematics students, has been analytics”—where data sets are loadedbetter decisions with big data is the running for 12 years in the US and is used into memory (RAM), making analysisdearth of talented people to analyse in 18,000 schools; it will be offered to UK much faster—become more refinedit, mentioned by 51% of respondents. schools, for free, from March 2012. SAS and widely deployed, decision-makingFor consumer goods and retail firms it has also developed advanced analytics at the operational and tactical level, atis the single toughest obstacle, cited by courses with a number of universities, least, is likely also to become faster.two-thirds of respondents from those including Centennial College, Canada,sectors. North Carolina State University and Saint Joseph’s University, Philadelphia,“In terms of modelling, there is to provide the next generation of datagoing to be a considerable shortage analysts.[of specialists],” says Professor KSudhir, James L. Frank ‘32 professorof marketing at Yale School ofManagement. “As a nation we generally The time factorfind math and sciences less exciting, The time it takes to analyse largeand I think people have been moving data sets is seen as another majoraway from this to ‘softer’ sciences. impediment to more effective use ofClearly, there is a shortfall, especially big data in decision-making. “I thinkin the analyst domain, and it is going to big data is going to stimulate thecontinue unless we systemically fix it.” need for more CPU [microprocessor]Bill Ruh of GE agrees. “There is going to power, because people are going tobe a war for this kind of talent in the next get very creative and they’re goingfive years,” he says. to invent new algorithms, and we’re going to say ‘My God, everything’sAside from a master’s degree or PhD in slow again’,” says Mr Ruh of GE. “Weeconomics, mathematics, physics, or are going to have to redo our computeother relevant field of science, analysts and storage architectures because theyare also expected to have in-depth will not work where all this is going.”domain knowledge—somethingwhich usually takes years to acquire. Most of the survey respondents haveInterviewees for this report also say not experienced a slowing of decision-that the ideal analyst should have an making due to having to processability to communicate complex ideas large quantities of data. Only 7% sayin a simple manner and should be that it has slowed down decision-customer-focused. Finding people with making significantly, while 35% sayall of these abilities is never going to be it has slowed it but only moderately.easy, and retaining them is going to be (Respondents from transport,even harder as the benefits of big data government, telecommunicationsbecome apparent to more firms. and education suggest a greater deceleration of decision-making thanTechnology companies recognise the other sectors.) The impediment mustproblem and are working with schools be, then, not that decision-makingand universities to develop these much is slowing, but that it is not gettingneeded skills. For example, SAS, a faster. This seems to be borne out bybusiness analytics software firm based the fact that the vast majority (85%)in Cary, North Carolina, developed of executives believe that the issue isCurriculum Pathways, a web-based not the growing volumes of data, buttool for teaching data analytics to high rather being able to analyse and actschool students. The course, aimed on data in real-time. As “in memoryat science, technology, engineering20
  19. 19. The Deciding Factor: Big data and decision-making 85% of respondents say the issue is not about volume but the ability to analyse and act on the data in real timeSurvey Question: To what extent do you agree with the following statement:“The issue for us is now not the growing volumes of data, but rather being able to analyse and acton data in real-time.” Strongly Agree Agree Disagree Strongly Disagree Don’t know/Not applicable100%90%80%70%60%50%40%30%20%10% Total Financial Energy & Consumer IT & Manufacturing Healthcare Sector Resources Technology Total Financial Energy & Consumer Sector Resources 28.7% 21.7% 30.4% 37.8% 56.1% 62.3% 63.0% 48.6% 10.1% 10.1% 4.3% 8.1% 1.3% 1.4% 0.0% 0.0% 3.7% 4.3% 2.2% 5.4% IT & Manufacturing Healthcare Consumer 30.6% 36.4% 26.4% 53.2% 54.5% 62.2% 11.3% 7.3% 6.7% 3.2% 1.8% 2.2% 1.6% 0.0% 4.4%21
  20. 20. The Deciding Factor: Big data and decision-makingConclusionProfessor Alex Pentland, director of the heavy industry, especially in areas suchHuman Dynamics Laboratory at MIT, as energy production and distributionsays big data is turning the process of (“smart grids”) and transportationdecision-making inside out3. Instead of (“smart cars”, etc), excessive automationstarting with a question or hypothesis, of business processes can hamperpeople “data mine” to see what flexibility. Besides, the growing post-patterns they can find. If the patterns financial-crisis regulation calling forreveal a business opportunity or a greater accountability requires humansthreat, then a decision is made about to ultimately make the to act on the information. Prosecutors cannot put an algorithm in the dock.This is certainly true, but improvements The financial crisis has also led to callsin computing power and artificial for greater transparency. As the surveyintelligence systems mean that asking shows, people are increasingly warydirect questions of big data and getting of business decisions based purelyan answer, in real time, is now a reality on intuition and experience. Even if a(see WellPoint case study). Although sizeable minority agree that businessthese systems are still very costly and managers have a better feel fornot widely deployed, this research business decisions than analytics willsuggests that the appetite for real-time ever provide, managers will increasinglydecision-making is huge. And when need to show how they arrived at theirthere is a business demand, it is only decision. And big data will provide aa matter of time before the need if post-decision review—was it a goodfulfilled. decision or not? As one of the survey participants puts it, using big data forMost of the executives polled for this decision-making will lead to “betterreport are also optimistic about the cost decisions; better consensus; betterreductions and efficiencies that can be execution”.had from automating decision-makingusing big data. While there is certainlymuch scope for decision-automation in23
  21. 21. About CapgeminiWith around 120,000 people in 40 countries, Capgemini is one of the world’s foremost providersof consulting, technology and outsourcing services. The Group reported 2011 global revenues ofEUR 9.7 billion.Together with its clients, Capgemini creates and delivers business and technology solutions thatfit their needs and drive the results they want. A deeply multicultural organization, Capgeminihas developed its own way of working, the Collaborative Business Experience™, and draws onRightshore®, its worldwide delivery model.Rightshore® is a trademark belonging to CapgeminiMore information about our services, offices and research is available atwww.capgemini.comWritten by