Big Data - Big Benefit or Big Waste?


Published on

The ability to rapidly analyze Big Data has become a key competitive advantage to manage the
exponential growth of globally available information. Early adopters have already proven benefits both
on the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial to
leverage this potential without spending millions on oversized analytic capabilities. xCon Partners has
coordinated several hundred Big Data projects for a major solution supplier, always supporting the
fast and smooth project execution. We are your ideal partner to tackle the challenges involved in Big
Data and to disclose its full potential by giving strategic guidelines as well as execution support.

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Big Data - Big Benefit or Big Waste?

  1. 1. PERSPECTIVE20121Big Data - Big Benefit or Big Waste?Your Big Data strategy will determine your successBig Data – Blessing or Curse?Google, Amazon and Facebook, three of today’smost powerful companies, have been built aroundBig Data. The information that flows through theirveins is directly linked to their ability to analyze vastamounts of data. Exactly this capability makes themmarket leaders with a position that is hard to attackfor any competitor. No surprise that by now, othercompanies across all industries have likewise startedto gather information in hope to boost their business.T-Mobile USA is only one of many examples forwhich these efforts have already paid off. Byanalyzing huge amounts of data in real time withSAP’s new in-memory database HANA, it is nowpossible to provide targeted offers to more than 21million customers. This results in $10-25 savings foreach won back subscriber and potentially in billionsof additional revenue per year. In the saturatedtelecommunications market this will be a crucialadvantage against the competitors.On the flipside, companies missing this analyticalcompetence will see their market position significantlyweakened with the threat of an attack of a muchbetter prepared competitor always lingering aroundthe corner. Since the amount of digital data doublesevery five years according to recent calculations, thiseffect will further increase. By 2012 we have alreadybeen faced with about 2.6 Zettabytes of global data.Assuming that a human being is able to memorize100 GB of data, it would need four times of the entireglobal population to store this information. With aCAGR of almost 45%, in 2013 already six times ofthe world population would be needed.This trend is boosted by the increasing distribution ofnetworked devices and sensors in our daily life, smartphones, internet and the social media platforms. Thekey challenge is that the amount of data companiesneed to evaluate in acceptable time is nowadaysgrowing faster than the performance of establisheddatabase technologies and analytical tools (compareFigure 1). In addition, available data becomes moreand more semi-structured (XML) and unstructured(documents, e-mails, videos, pictures, etc.) andhence more difficult to analyze.Figure 1: Growing “amnesia” of enterprises1960 1970 1980 1990 2000 2010 2020 20301,000,0001,00011,000,000,000AnalyticalSpeed1)1) # of Data Units that can be evaluated in acceptable time and money2) # of company’s Data Units to be evaluated to stay competitiveSource: xCon Partners analysisEvaluationRequirements2)World WideWebEmailPCtodayEnterprise SocialMediaInternet ofThings# of DataUnitsData amountexceeds analyticalperformanceresulting in amnesiaThe ability to rapidly analyze Big Data has become a key competitive advantage to manage theexponential growth of globally available information. Early adopters have already proven benefits bothon the cost and on the revenue side. However, a well-thought-out Big Data strategy is crucial toleverage this potential without spending millions on oversized analytic capabilities. xCon Partners hascoordinated several hundred Big Data projects for a major solution supplier, always supporting thefast and smooth project execution. We are your ideal partner to tackle the challenges involved in BigData and to disclose its full potential by giving strategic guidelines as well as execution support.
  2. 2. PERSPECTIVE2To avoid resulting amnesia, companies have to eitherdispose part of potentially valuable data untouched orswitch to new innovative analytics techniques.Therefore it is essential for executives to define theoptimal Big Data strategy tailored towards theircompany’s needs.Big Data for Big BenefitsEven though data intense industries such astelecommunication, (multi-)media as well as thebanking and insurance sector face the highestobvious demand for Big Data solutions, basically allindustries and companies can take advantage of thenew and improved analytic possibilities. Onecompelling success story is for example written bythe retail industry which is characterized by a lot ofcustomer interactions. After putting new in-memoryanalytics technology in place, the luxury labelBurberry is now able to access millions of recordsfrom multiple sources (customer data, stockinformation, social media, etc.) nearly in real time.This was made possible due to a speed increasefactor of 14,000 – requests now take one secondinstead of previously nearly five hours. The real-timeanalytics system is used to profile customers,allowing making tailored customer offerings on thefly. Hence, it allowed Burberry to implement a veryefficient customer-oriented sales strategy and to buildup a consistent global brand.Certain cross-industry effects related to Big Data andnew analytics technologies are affecting all types ofcompanies: Fact based decisions instead of relying onassumptions: For business critical decisions, allhistorical facts and current events can beconsidered instead of relying on estimates andassumptions. Interactive scenario simulation: Calculate theimpact of alternative scenarios in almost real timeto assess risks and fine tune the chosen model.This is especially interesting for priceoptimizations which can now be based on allavailable historical data. Real time customer interaction support: Alwayshaving an up-to-date customer behavior profileavailable will create a very personal customerexperience and enable you to react in an optimalway. Micro personalization: Tap directly into yourcustomer base by defining very precise customersegments to run individual micro campaigns andto measure their success. New business models: Real-time analyticalcapabilities can also be used to access emergingmarkets. New sensors like smart phones or smartmeters will for example enable new businessmodels around location based services or homeautomation.Even in the field of crime reduction, Big Data iscontinuously gaining importance: Fraud Miningapplications are used by international financeinstitutions with high transaction rates for systematicidentification and prevention of fraud. JP Morgan isfor example successfully deploying this technology innumerous projects for identifying potential fraudamong traders.Another example in this field is the joint venture ofPaymint AG and Fraunhofer IAIS that successfullydeveloped and implemented the application MINTIfy.This solution protects millions of European credit cardaccounts from fraud by analyzing thousands ofattributes in the transaction history and identifyingconspicuous patterns.Different branch, similar problem: TomTom BusinessSolutions, a commercial vehicle fleet specialist, isAre you faced with Big Data?Benefiting from Big Data requires analyzing hugeamounts of data from different sources at veryhigh speed. An exact quantitative definition of BigData is difficult since computing power isconstantly growing. For example the amount ofdata used by the NASA in 1969 for the moonlanding operation can nowadays easily behandled by a modern smart phone. Therefore werecommend this pragmatic quantification: BigData cannot be captured, stored, managed andanalyzed by the commonly used software andhardware within a tolerable period of time. In thissense the magnitude is specific for your businessmodel and your Big Data capabilities. A goodindication on the criticality for your business canbe derived by assessing the following three V’s: Volume: What amount are you faced with? Variety: How diverse is your data? Velocity: How fast do you need to analyze it?
  3. 3. PERSPECTIVE3facing over 1.5 billion real-time notifications from175,000 vehicles and more than 1 billion requests permonth. Fujitsu accepted the challenge andimplemented an Oracle based solution which is nowable to handle 200,000 input-output-operations persecond with response times of under one millisecond.This grants fleet managers and transport plannersaccess to real time information to optimize the routingof their vehicles and to reach a higher fleet utilization.In-Memory, Scale Out or NoSQL?Once the need to handle Big Data is evident, thequestion of the best IT solution(s) arises. To makeuse of the company’s Big Data, many differenttechnologies (e.g. in-memory computing) andmethodologies (e.g. visual analytics) need to beevaluated to define a comprehensive Big Datastrategy. Depending on the business problem, thecurrently most promising technologies to significantlyspeed up and broaden data evaluations are in-memory computing, a horizontal scale-out approach,NoSQL databases, or a combination of those. Thebest fitting technology always depends on thecompany’s specific business context.The “trick” of in-memory technology is to avoid slowhard disk access by constantly keeping all relevantdata in RAM. As the average access speed betweenRAM and disk differs roughly by a factor of 100 to1,000, data operations can be performed significantlyfaster. Obviously, the speed increase depends on thespecific business context and the chosenimplementation. No wonder that currently almost allof the leading global enterprises evaluate thistechnology. In addition, in-memory solutions are oftencombined with other powerful concepts to furtherboost their performance. Some database vendors forexample use their know-how of specific applicationsto optimize the underlying data structure: by usingcolumnar architectures, only columns that contain thenecessary data to determine the answer are beingprocessed which leads to significantly fasterresponse times. Additionally, advanced datacompression capabilities are applied to reduce thesize of information that needs to be stored andanalyzed. In-memory technology “standalone”currently works best with relational databases andstructured data, providing instant results.Speed improvements with even higher factors than1,000 are possible: Automotive ResourcesInternational (ARI) is now able to analyze millions ofdata points collected from approx. 923,000 vehicles3,600 times faster than with traditional technologies.After a three week implementation of the SAP HANAdata mart solution, the company now benefits from5% cost reduction in total overhead expense andfrom increased contact center performance.Currently, the in-memory market is still dominated bydata warehouse projects aiming to provide optimalmanagement decision support (see Figure 2). Butalmost as many companies gather their ownexperience by experimenting with customdevelopments. We expect that some of the mostradical game changing innovations will arise out ofthese initiatives.Figure 2: Market split of current in-memory activitiesComplementary or as an alternative to in-memorycomputing, Hadoop is one of the most prominentframeworks to implement scale-out scenarios.Developed by the Apache group, it allows for thedistributed processing of large data sets acrossclusters of computers using simple programmingmodels. It is designed to scale out from singleservers to thousands of machines, each offering localcomputation and storage. At the core of Hadoop is animplementation of the “MapReduce” algorithm: first,the “Map” function divides the original query into sub-segments and calculates their results on any numberof distributed nodes. Second, the “Reduce” functioncentrally aggregates these intermediate results andreturns the answer. A whole set of additional Hadoopcomponents supports the seamless integration intothe enterprise environment: Flume and Sqoop helpwith data population, Mahout encapsulates datamining capabilities, Hive and Pig assist with querygeneration. This framework is already used by mostof the leading online companies like Google,Amazon, and Facebook for searching and analyzingOn-DemandSolutionsDataWarehouseCustomDevelopmentApplicationsStandardizedConfigurationsSide-by-SideAccelerationSource: xCon Partners analysis
  4. 4. PERSPECTIVE4their data. Scale-out technology works especially wellwith unstructured and semi-structured data.NoSQL (Not Only SQL) databases are a goodalternative if Big Data performance and scalability ismost important and 100% consistency is notrequired. Compared to the sophisticated relationaldatabases, they are better suited to handle largevolumes of multi-structured data. The biggestdisadvantage is that the majority of NoSQLimplementations no longer required transactions tobe ACID (atomic, consistent, isolated, durable). Thisis one major difference to the in-memory databasesand disqualifies this technology for online-transaction-processing that does not compromise ondata accuracy. Besides the most prominent NoSQLimplementations BigTable (Google) and Dynamo(Amazon), it is to mention that with HBase, a versionexists which is especially meant to be deployed ontop of HDFS, the Hadoop Distributed File System. Byallowing low-latency lookups in Hadoop, it combinesthese two Big Data technologies.In certain cases, also Hadoop and in-memorytechnologies need to be combined to achieve thedesired computing power. One example is a newcancer research solution realized by Charité. Theuniversity medical center has proven that it ispossible to reduce the time required to analyze atumor by a factor of 1,000, reducing it from hoursdown to a few seconds. This offers the possibility toadjust cancer treatments before the patient leavesthe hospital.Increasingly, new players position themselvessuccessfully and offer their own Big Data solutions:the online retailer OTTO was seeking for a possibilityto improve its warehouse management and its salesforecast system. The predictive-analytics softwareNeuroBayes by Blue Yonder was applied here,handling over one billion forecasts in a year. The self-learning system is able to process over 135 GB or300 million of daily new data sets, boosting theforecast efficiency up by 40%.At the same time, established vendors like SAP orOracle develop a whole set of new in-memory basedapplications that will set the industry standard in themid future. Some early adopters and co-innovatorshave already migrated.Additionally to the described major Big Datatechnologies, there are various new and innovativeanalytic solutions being developed, tailoredspecifically towards Big Data. Those analyticsolutions are usually combined and sit on top of oneor several of the described Big Data technologies.Due to the high number of different approaches,describing them would go beyond the purpose of thisPerspective and needs to be evaluated individuallyfor the specific business purpose.Big Data Market OverviewThe Big Data market size strongly depends on themarket definition. Sticking to the Big Data definitionprovided above, we size the Big Data market(Software, Hardware and Services related to andused by In-Memory, Scale-Out, and NoSQLtechnologies) for 2012 at about 10 billion Euros.Looking in the past and the years to come, we predicta CAGR of 40% for the upcoming five years whichbrings us to a market size above 50 billion Euros in2017. Major driver for this significant market growth isthe explosion of data due to increasing use of socialmedia, mobile devices and the internet of things.There are more than 100 players active in the BigData market with solutions tailored specificallytowards Big Data. Figure 3 provides xCon Partners’view on the most important players in the Big Datamarket. Naturally, this matrix is frequently subject tochange due to the fast moving and dynamic market.Figure 3: Big Data Market OverviewBig Data Capability2)Challengers LeadersHigh Potentials InnovatorsSAPOracleMicrosoftGoogleAmazonIBM NetezzaHP VerticaTeradata AsterEMC GreenplumCurrentMarketPosition1)Cloudera1010DataSAS10genMapRHortonworksVMware1) Besides Big Data market share considers also market share inDBMS, Data Warehouse, BI, Enterprise Process Management2) Considers completeness (In-memory, Hadoop, NoSQL, etc.),maturity and vision of Big Data solutionSource: xCon Partners analysisFacebookKognitioParAccelMarkLogic
  5. 5. PERSPECTIVE5To determine the “Current Market Position”, besidesthe still very volatile Big Data revenue, also marketshares in related and established markets likeDatabase Management Systems (DMS), DataWarehouses (DW), Business Intelligence (BI), andEnterprise Process Management (EPM) have beenconsidered. The “Big Data Capability” is not onlydetermined by the completeness of the offeredsolution, but also by its maturity and the vision of thevendor.The seven biggest vendors with complete offeringsfor In-Memory or Scale-Out Solutions in the Big Datamarket are currently IBM Netezza, Oracle, Microsoft,SAP, HP Vertica, Teradata Aster, and EMCGreenplum.The market position of those seven vendors is on theone hand threatened by the innovators which aremature internet companies that have developed theirown and very sophisticated Big Data solutions likeGoogle, Amazon or Facebook. On the other hand,there are many emerging players entering the marketwith new and innovate Big Data technologies andsolutions.Companies need to select the most suitable solutionsand best positioned vendors carefully to not run intothe trouble of discontinued products.Rely on xCon Partners’ ExperienceSince 2011, xCon Partners has been involved in andcoordinated several hundred Big Data Projects,always supporting the fast and smooth projectexecution. Therefore, we are your ideal partner fortackling the involved challenges and for disclosingthe full potential. Besides strategic recommendations,we also support our clients during the execution toensure successful implementations. From havinganalyzed hundreds of slipped projects we can giveindications on common implementation risks and helpto set realistic project targets and roadmaps.From our experience, the Big Data topic is bestaddressed with the following 5-step approach (seealso Figure 4):1) Conduct a Big Data readiness check toderive a clear picture of available information,data sources and own analytic capabilities2) Calculate the business potential forenhancements to the current business modeland for new business opportunities includinga cost-benefit check3) Understand which technology and solution isbest suited to implement the Big DatastrategyFigure 4: xCon Partners Big Data ServicesxCon Partners Big Data ServicesBigDataStrategyImplementand Track Under-standTechnologyandMarketBig DataReadinessCheck CalculateBusinessPotentialPlanImplementation Define roadmap and project plan System & integrator selection Technology assessment Overview on Big Data market(vendors & service providers) Transparency on currentmarket adoption of differentsolutions Support proof of conceptactivities Cost-Benefit analysis Analyze available informationand additional data sources Evaluate current capabilities forBig Data analytics Benchmarking Analyze potentialenhancements to currentbusiness and identify newbusiness opportunities Define additionalinformation requirementsand analytics capabilities Estimate effort andbusiness potential Project management Expert insights based on400+ tracked activeBig Data projects Avoid project delays Measure success12345
  6. 6. PERSPECTIVE64) Define a realistic implementation plan andselect the right partners5) Implement and measure the successIt is important that Big Data and the technology totackle the involved challenges must not be an end initself. As a starting point, the development of aholistic and company adapted Big Data strategy iscrucial for success and should allow executives tomaintain a clear vision of what they want to achieve.Based on this, the appropriate data, the proper levelof detail and the best technologies have to beselected. If this is done correctly, companies have thechance to gain a significant competitive advantage.Even increasing the revenues by up to 30% may notbe out of reach. This is the target that the onlinegame company Bigpoint is aiming for. It will bereached by analyzing more than 5,000 game eventsper second in real time to offer their players anindividualized game environment and to enablepersonalized micro sales. This will further strengthenBigpoints already strong position in the market.Securing and strengthening the market position byleveraging Big Data is also possible for you: withxCon Partners’ proven holistic Big Data approachand our comprehensive project history, we are thepartner of your choice to develop a well-thought-outBig Data strategy for you.About the AuthorsPercy 176 21 30 44 50Leonid 173 2482 043About xCon PartnersxCon Partners is a strategy and managementconsulting firm focusing on Business-IT-Alignment –We link business and IT!We offer a combination of in-depth experience ininternational management and strategy consultingand special know-how in the CIO and CTO area ofinformation and technology management.Our clients benefit from a cooperative consultingapproach, always considering the individual andunique situation of the client. Our extensive partnernetwork gives us on-demand access to furtherindustry specific and functional know-how wheneverneeded.With office locations across Germany (Bremen,Wiesbaden and Munich), we are proud to serve ourDAX 30 and mid-sized customers with close distanceand perfect reachability. For more information, pleasevisit © by xCon Partners GmbH 2013.All rights reserved.