Published on

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. SAP Solutions for AnalyticsBig Data Analytics GuideBetter technology, more insightfor the next generation of business applications
  2. 2. Big Data Analytics Guide 2012 Big Data Analytics Guide 2012Big Data Analytics Guide 1
  3. 3. Big Data Analytics Guide: 2012 Published by SAP © 2012 SAP AG. All rights reserved. SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects Explorer, StreamWork, SAP HANA, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. Sybase is an SAP company. Crossgate, m@gic EDDY, B2B 360°, and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. Crossgate is an SAP company. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies (“SAP Group”) for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. Library of Congress Cataloging-in-Publication Data SAP Big Data Analytics Guide 2012: How to prosper amid big data, market volatility and changing regulations / Edited by Don Marzetta. p. cm. ISBN 978-0-9851539-6-0 1. Big data. 2. Analytics. 3. Databases2 Big Data Analytics Guide
  4. 4. Welcome to Big Data Analytics Guide 2012 Big Data Means Big Business By Steve Lucas, Executive Vice President and General Manager, Database and Technology, SAP With more and more people spending much of their existence in the digital world—whether it’s for work, play, learning, or to socialize—the amount of data being generated is truly astounding. Just think about the number of SMS messages and emails sent, phone calls placed, and Facebook updates made every minute, and it boggles the mind how much data is traversing networks around the world. At SAP we believe this confluence of events is a golden opportunity for enterprises , to rethink how they do business, and our goal is to help them do their jobs better than ever. As co-chair of the Big Data commission sponsored by TechAmerica, I’m hearing firsthand how government and private enterprise is evaluating how to embrace a Big Data world. But this requires organizations to take a new approach to data. Innovations in in-memory computing are turning the whole idea of data management on its head by allowing enterprises to get rid of the complexity that’s been encroaching on their systems. That’s why we’ve built the real-time data platform with SAP HANA at its core, giving enterprises the foundation they need to embrace Big Data. But we recognize a platform is not enough. Together, we need to dream new ways to do business by leveraging Big Data insights. And at SAP we’re serious about making this a reality by investing in new businesses through the $155 million venture start-up fund dubbed the ‘SAP HANA Real-Time Fund,’ through the HANA Start-up program, through innovation labs around the world, and most importantly by co-innovating with our customers. To further our path toward better, smarter ways of working with data, we’ve put together a series of articles in the Big Data Analytics Guide. Within these pages you’ll find real solutions, real ways of operating, real results and perhaps most importantly real technology that can be used in your business today. The guide outlines the opportunity and business case for Big Data in the first chapter, and subsequent chapters look at SAP technology innovations, real-world examples, and insights from analytic leaders who are on the forefront of the Big Data market. In the last chapter, you’ll find a set of interesting market research statistics that highlight how C-level executives are using Big Data now and their plans for using it to their advantage in the future. So join in the conversation and co-innovate with us to re-invent business. nBig Data Analytics Guide 3
  5. 5. Table of Contents 03 Welcome to Big Data Analytics Guide 2012 30 Analytics Advantage Big Data Means Big Business 30: Data Variety Is the Spice of Analytics By Steve Lucas, Executive Vice President and By Amr Awadallah, CTO, Cloudera General Manager, Database and Technology, SAP 33: Text Analytics for Speed Reading—Do You Mean 06 Big Data Opportunity What You Say? 06: Measuring the Value and Potential Yield By Seth Grimes, Strategy Consultant and Industry of Big Data Projects Analyst, Alta Plana By Dan Lahl, Director of Analytics, SAP 36: Image Recognition, Pattern Identification, and 08: The Numbers are In: Early Stage ROI the New Memory Game and Proof of Concept By Joydeep Das, Director, Data Warehousing and By David Jonker, Product Marketing Director, SAP Analytics Product Management, SAP 10: Analytics in the Cloud: Traversing a 38: Technology Alone is Not the Answer Legal Minefield By Byron Banks, Vice President of Business Analytics By Dr. Brian Bandey, Principal, Patronus Marketing, SAP 13: Big Data Analytics Earns High Scores in the Field 40 Analytics Innovations 40: What’s All the Hadoop-la About? By Wayne Eckerson, Principal, BI Leader Consulting 016: Big Data Is Only a Small Part of the Opportunity By Mike Upchurch, Chief Operating Officer, Fuzzy Logix 43: Fast Flowing Decisions Through Streams of Data By Irfan Khan, Senior Vice President and 18 Business Analytics Roadmap Chief Technology Officer, SAP Database and Technology 18: Business Value through Operational BI By Claudia Imhoff, President of Intelligent Solutions, Inc. and Founder of the Boulder BI Brain Trust 45: Age of Influence: Making the Most of Social Networks By Bruno Delahaye, Senior Vice President Worldwide 21: Real-time Data Platform for a Real-time World Business Development, KXEN By Amit Sinha, Head, Database and Technology Innovation, SAP 48: Embracing a Standard for Predictive Analytics By Michael Zeller, Ph.D., CEO, Zementis 23: How HANA Changes the Database Market By Ken Tsai, Vice President HANA Solution Marketing, SAP 51: How Modern Analytics “R” Done By Jeff Erhardt, Chief Operations Officer, Revolution Analytics 25: DBTA: Data Marts Can’t Dance to Data’s New Groove By John Schitka, Senior Product Marketing Manager, 54: Navigating a 4G World Sybase IQ By Greg Dunn, Vice President, Sybase 365 28: In-Database Analytics: Reducing Travel Time 57: Increasing the IQ of Everyone with Analytics By Courtney Claussen, Sybase IQ Product By Jürgen Hirsch, CEO, Qyte GmbH Manager, SAP 60 Market Data 60: The Big Deal with Big Data By IDC 68 Company Index Big Data Analytics Guide 5
  6. 6. Big Data OpportunityMeasuring the Value and Potential Yieldof Big Data ProjectsWhere should companies look for the return on investmentthat determines whether, or how much, Big Data projectspay off?By Dan Lahl, Director of Analytics, SAP intelligence tools, and training for the people who will use it to make decisions. Where to Look for ROIData analytics has traditionally been Companies can look for ROI in these three ways: do whatexpensive and inefficient, but new analytical they’re already doing better, do more of what they’re already doing, and do things they’ve never thought about before.platforms optimized for Big Data are Big Data provides new solutions for current problems,heralding a brave new world. Hadoop, an converts incomprehensible data to actionable businessopen-source Apache product, and Not Only recommendations, and makes previously impossibleSQL (NoSQL) databases don’t require the business models possible.significant upfront license costs of traditional The first, and often most compelling, benefits of upgradingsystems, and that’s making setting up an to a Big Data platform are in speed and cost savings platform—and seeing a return on The new systems allow organizations to do what they arethe investment (ROI)—more accessible than already doing faster, better, and cheaper. What used to take hours or days suddenly only takes minutes. As data volume,ever before. velocity, and variety have grown, legacy data warehouse systems have been bogging down—unable to handle biggerCosts are coming down, but this is no free lunch. Crunching data, more users, and increasingly complex queries. ResultsBig Data analytics still requires hardware, database take longer. Users get frustrated and stop using the dataadministrators, developers to build the models, business because it takes too long. Many data warehouses are in this predicament today, having hit a performance and scalability6 Big Data Analytics Guide
  7. 7. wall. Big Data technologies break through these roadblocks, opportunities to collect data and put it to work. Used thisdelivering faster performance and higher availability. way, analytics often deliver a huge ROI, helping a company identify problems early on (or before they happen) andThe second benefit is the ability to do more. Not only will take steps to fix (or prevent) them. Predictive analyticsa Big Data system not bog down at current usage rates,but it can also handle more users, more data, and morecomplex queries. For example, instead of storing a year’sworth of loan default records, a lender can store 30 years’ Wherever data becomes part of aworth and perform more detailed analyses, with more business process, such as in customeraccurate results. support or sales, companies canAdditionally, a Big Data system can handle a mixed workload. measure customer satisfaction, salesInstead of processing a few queries from a handful of powerusers, it can handle a bunch of short technical queries by figures, and other established metrics.a large front-line service team, such as a customerlookup that identifies cross-sell and up-sell opportunitiesin real time. can recognize customer care or sales opportunities at the right moment, offering discounts or related productsThe third piece of Big Data ROI is that it opens up new that increase satisfaction and boost sales.opportunities. Once the data has been liberated andemployees can get at it, they find all kinds of new ways Success for the Long Termto use Big Data. For example, if a utility company had Once an enterprise decides that its analytics project haspreviously outsourced the analytics from its self-service legs, there are some well-established tactics to help assureWeb pages and analyzed that data separately from its its long-term success. Foremost is communication. Keepother customer service channels, Big Data technology everyone in the loop from strategy to deployment—andcan bring all the analysis together, in-house. That opens beyond. Key individuals should never be blindsided bythe possibility to track customer behavior in multiple hiccups or new developments in the process.channels at the same time. This kind of 360-degreevisibility provides a more accurate picture of the customer New technology built to handle today’s veritable delugeexperience and customer satisfaction, providing new and of data is bringing down the cost of analytics, deliveringdeeper insight into any business. better performance, and helping companies put data to work in new ways. Measuring the ROI of a Big Data projectShow Me The Metrics can be accomplished by establishing company metricsWhen establishing metrics for any new analytics project, that highlight analytics usage, encouraging data teamsfocus in two areas: employee usage of the resulting and employees to define actionable reports, andintelligence and key performance indicators for the incorporating analytics into more decision makingprocesses where analytics will be used. throughout the enterprise. nFirst, create metrics and track data for the project itself.Think of employees as the data team’s customers, and Dan Lahl has been in high tech forestablish measures that show the total amount of almost 30 years. In addition to bringinginformation consumed in the organization: who’s using to market SAP Sybase SQL Server,the data, how much are they using it, and what are they SAP Sybase ASE, and SAP Sybase IQ,using it for? The higher the usage rates, the better. Lahl has evaluated multiple emerging technology areas leading to EII,Second, wherever data becomes part of a business ETL, and GRID technologyprocess, such as in customer support or sales, companies purchases for the company.can measure customer satisfaction, sales figures, and otherestablished metrics. Comparing numbers before and afterthe Big Data implementation should clearly show whatthe organization has gained as a result—as well as otherBig Data Analytics Guide 7
  8. 8. ENTERPRISES, BUSINESSES, AND BUSINESSES AND GOVERNMENTS ARE SEEING SIZABLE RETURNS ON THEIRINVESTMENT IN BIG DATA ANALYTICS PROJECTS.The Numbers are In:Early Stage ROI and Proof of ConceptBy David Jonker, Product Marketing Director, SAP AOK Hessen, a public health insurance organization in Germany, uses pattern analysis to detect fraudulent invoicing, and it has reclaimed US $3.2 million of unjustified charges.As new technology helps organizations put Lawson HMV Entertainment, a leading retailer in Japan oftoday’s massive data sets to use, real-world DVDs, Blu-ray discs, books, and games, integrates in-store and Website data into one massive marketing database thatexamples and research are proving that the company uses to fuel its targeted email campaigns toanalytics helps cut costs, increase revenue, customers. The result has been 3 to 15 times higher purchaseeliminate waste, and otherwise boost the rates and double digit revenue growth.bottom line.Real Returns Table 1. Big Data Analytics ROIRecent reports confirm that organizations around the worldare finding an edge by incorporating advanced analytics into Who Whatbusiness processes, leading to informed decisions around American Airlines $1 million annual fraudcustomer satisfaction, new service success rates, and detectioncompetitive analysis (see Table 1.) American Airlines has State of Sao Paulo, Brazil $100 million untaxed earningssaved roughly $1 million annually in fraud detection, helpingthe company identify forms of fraud it never knew existed and Cell© $20 million saved on one projecteliminating the loopholes that criminals were exploiting. AOK Hessen $3.2 million fraud detection HMV Japan 3 to 15 times higherThe Sao Paulo State Treasury Department has so far identi- purchase ratesfied $100 million in untaxed earnings, which allowed taxinspectors to adopt a proactive approach to tax evaders,investigating telltale behavior patterns in its data and taking Research Backs Up the Numberscorrective measures early on, rather than punitive ones after After studying 179 large public companies that use whatthe fact. they call “data-driven decision making,” authors from MIT and the Wharton School concluded that organizations using analytics in their decision-making processes derived 5 to 6% better “output and productivity” than if they had not used analytics.Organizations around the world arefinding an edge by incorporating Researchers at the University of Texas studied how analytics affected the finances, customer activities, and operations ofadvanced analytics into 150 Fortune 1000 firms. According to the research, thebusiness processes. product development area alone justifies deploying analytics8 Big Data Analytics Guide
  9. 9. According to a University of Texas study, product development alone justifies deploying analytics for a typical Fortune 1000 enterprise.Table 2. Legacy vs. Next Generation Real-Time Analytics in Telecommunications Legacy Analytics Infrastructure Next-Generation Real-Time Analytics Infrastructure Storage cost High Low Analytics Offline Real time Data loading speed Low High Data loading time Long Average 50% faster Administration time Long Average 60% faster Complex query response time Hours/days Minutes Data compression technique Not mature Average 40 to 50% more data compression Support cost High Lowfor a typical Fortune 1000 enterprise. The study states, The sources of these significant ROI metrics vary by company“Revenue due to a company’s ability to innovate new products and industry. To examine an example in depth, consider howand services increases with data accessibility and specialty using analytics for real-time applications has impacted theproducts and services, which, in turn, is positively affected telecommunications market (see Table 2). You can see how aby data intelligence.” modern analytics environment leaves legacy decision-support systems in the proverbial dust. nHow positive is that impact? According to the analysis, a$16.8 billion company might see an extra $64 million top-linedollars over five years if analytics are put into the hands of David Jonker is focused on market strategy“more authorized employees” who “can make use of informa- for the SAP Data Management andtion…to better spot trends, demand patterns, improve recom- Analytics product lines, includingmendations for decision making and profile match.” All of those SAP Sybase IQ, ASE, Replication Server,benefits could contribute to new-product revenue. Such firms and SQL Anywhere. Jonker’s careercould also add $14 million in new customer sales annually. includes more than 10 years in software engineering and product managementThe University of Texas report also discovered that the roles before leading the SAP product marketing teamscomprehensive use of analytics inside a company improved for data management and analytics.results in the operational areas of asset utilization, forecastingand planning, and on-time delivery of products or services. Forexample, wider use of analytics can lead to an 18.5% improve-ment in planning and forecasting for a typical firm studied.Big Data Analytics Guide 9
  10. 10. TO AVOID LEGAL LIABILITY, ORGANIZATIONS THAT WANT TO REAP THE BENEFITS OF CLOUD-BASED BIG DATAANALYTICS MUST CAREFULLY VET PARTNER TECHNOLOGY.Analytics in the Cloud:Traversing a Legal MinefieldBy Dr. Brian Bandey, Doctor of Law obligations that go to nondisclosure; but may also restrict the uses to which the data can be put and define what level of security is to be employed.When a corporation mines the Big Data within Other data might be owned by the corporation, but identifiesits IT infrastructure a number of laws will living individuals (whether directly or indirectly). Data Protec- tion Law (as it’s generally known) is concerned with theautomatically be in play. However, if that access, use, movement, and the technological safeguards tocorporation wants to analyze the same Big prevent disclosure of Personal Identifying Information (PII).Data in the cloud—a new tier of legalobligations and restrictions arise. Some of A corporation will also own secrets about itself which, if disclosed, might cause irreparable damage. Officers owethem quite foreign to a management stakeholders a legally binding ‘duty of care’ to take all reason-previously accustomed to dealing with its own able precautions to ensure the security of such within its own infrastructure. Due to restrictions on processing, both from Data ProtectionA corporation holding Big Data will possess different types of and Confidentiality Laws, care will need to be taken whendata which the Law will automatically classify and attach law- building the data warehouse to be analyzed. Certain classes ofbased obligations. data may need to be excluded.Some of that data may not be owned by the corporation. It All of these different types of law intersect over the area ofmay be a third party’s data which it holds pursuant to a Confi- Big Data storage, security, and processing. They produce adentiality Agreement. Such agreements may not only produce matrix of law-based obligations which, in many areas, cannot be delegated or avoided—only met. SecurityOften law-based security obligations But what happens when we translate that matrix into the cloud?cannot be delegated to the cloudservices provider. Legal responsibility The first matter is that of security. Breaches occasioning the loss of data can cause an abundance of law-based difficulties:may remain with the data controller. from breach of contract, fines under Data Protection Law, uncapped damages due to the release of third-party secrets and so on. But why is this the “first matter”? The corporation cedes actual security to its cloud services provider. Instead of the corporation implementing its own10 Big Data Analytics Guide
  11. 11. The cloud computing architecture must be able to identify what data is in which jurisdiction and, if necessary, keep it directly; that role is handed to the cloud services These are not matters of academic technical interest–but goprovider. A great deal is said about service level agreements to the ability of the corporation to discharge what are often,on this subject—their utility and importance. Frankly, I don’t non-delegable, unavoidable legal duties.see it that way. What remedies are available to the corporationunder a SLA other than contractual remedies? Usually none! Personal Identifying Information Moving on from security; there is the matter that is generallyIn my opinion, it is highly likely that money damages will not referred to as the trans-border movement of PII. Many coun-put the corporation back in the position it would have been— tries either restrict or prohibit the exporting of PII. To do sobut for the security/contractual breach. can even be a corporate crime—certainly exposing the wrongful exporter to the likelihood of a hefty fine, adverseNo. What is needed is the choice of a correct cloud security publicity, and reputational loss.architecture of sufficient robustness. One may ask why is thata legal topic? Surely it is strictly an IT matter? I take the view Thus the problem for our conceptual corporation is thethat one must look to the propensity of the cloud technology nature of cloud computing itself. By that I mean that theitself to cause the corporation legal exposure. advantages of scalability, flexibility, and economies of scale that are accessed through the technological advantage ofThe Duty of Care owed by officers to their stakeholders, the distributing data across a number of servers which may notcorporation’s duty to those persons whose PII it holds, and all be in the same country. Thus PII may be automaticallythe contractual obligations it owes with respect to third-party exported illegally.confidential information—all compel the corporation to exer-cise expertise, care, and prudence in the selection of a techno- There are two avenues open to the corporation to obviate thislogically secure cloud computing environment. This means ‘unlawfulness.’ The first is to choose a Big Data warehousingthat they must look beyond the cloud services provider per se, and analytics architecture which, with certainty, can confineand discharge the Duty of Care through due diligence on the data storage and processing to servers residing in nominatedtechnology underpinning it. How capable is the architecture of legal jurisdictions. The cloud computing architecture must besecuring the data? Is the architecture built to be secure and able to identify what data is in which jurisdiction and, if neces-resistant to the correct range of security threats? How robust sary, keep it there. The second is to transform the PII so that itand secure is it against measurable benchmarks? no longer constitutes, in law, PII. Data which is not PII cannot be subject to data protection law.Secondly, there are significant technological differencesbetween a cloud computing environment and a corporation’s For some time now, medical researchers have shared patient‘owned’ infrastructure. I am referring especially to the integrity information internationally through a process of eitherof multi-tenancy architectures. A leaky multi-tenancy system anonymize or pseudonymization. Anonymization is a processmust cause a significant probability that the corporation will whereby the identifier sub-data is removed, prior to export,be in breach of its obligations to many prospective litigants. thus enabling any type of processing, anywhere. The dataThus real attention will need to be given to the architecturethat isolates one ‘data set’ from another and keeps it isolated.Big Data Analytics Guide 11
  12. 12. needs to be of a configuration that can still be effectively damages in these scenarios will never, in my opinion, be suffi-processed in the absence of identifier sub-data. Where the cient compensation for the owners of Big Data.presence of a form of identifier sub-data is required forprocessing (or analysis) pseudonymization is used. Rather, the requirements of the law need to be soundly and accurately matched and, indeed, mapped onto the cloudThe aim of these two forms of de-identification is to obscure computing technology at hand. Only then can the minefield ofthe identifier sub-data items within the patient records suffi- Big Data analytics in the cloud be successfully traversed—ciently that the risk of potential identification of the subject of without an explosion. na patient record is minimized to acceptable and permissiblelegal levels. Although the risk of identification cannot be fullyremoved, it can often be minimized so as to fall below the Dr. Brian Bandey is acknowledged as one ofdefining threshold. the leading experts on Computer Law and the international application of IntellectualAnalytics Property Law to Computer and InternetThere is no reason, in law, why Big Data analytics cannot be Programming Technologies. His experienceperformed lawfully in the cloud. However, in order to do so, in the global computer law environmentsignificant attention needs to be directed to the actual soft- spans more than three decades. He is theware and hardware programming architectures to be author of a definitive legal practitioners textbook and hisemployed—and match those to the matrix of laws which commentaries on contemporary IT legal issues areoperate over the storage, use, processing, and movement of regularly published throughout the world. Dr. Bandey isdata. It may seem strange that I am advocating an almost now well-advanced upon the unique route of studyingtechnology-centric solution to what is clearly (and perhaps for a Second Doctorate of Law advancing the currentsolely) a law-based problem. But as I said before—money state of the art in the Intellectual Property in Internet and Cloud Technologies with St. Peter’s College at the University of Oxford in England. In order for Big Data analytics in the cloud to be lawful, the requirements of the law need to be accurately mapped onto the cloud computing technology at hand.12 Big Data Analytics Guide
  13. 13. INTERNET METRICS, TELECOMMUNICATIONS, AND FINANCIAL SERVICES PROVIDERS ARE USING BIG DATAANALYTICS TO BOOST PROFITS AND ADD CUSTOMERS.Big Data AnalyticsEarns High Scores in the FieldWhile industries vary greatly in what they need ready for analysis, modern Big Data analytic softwarefrom their data, and even companies within performs faster and readily scales to handle as many users and as much data as needed.the same industry are unalike, virtually everyorganization in every market has these two By seeking out Big Data inside and outside your organizationrelated problems: what to do with all the and using it to push intelligence deep into the enterprise,information pouring into data centers every organizations can be more responsive, more competitive, and more profitable. At the heart of these endeavors are columnar-second, and how to serve the growing number based databases and in-database analytics.of users who want to analyze that data. Companies from many industries are already taking advan-Massive data sets, even those including variable data types tage of technological advances in storing and analyzinglike unstructured data, can be analyzed. Not only are they Big Data to gain business insight and provide better service to customers. These real-world examples prove the value of Big Data analytics.In healthcare, the move to electronic comScore Stops Counting Visitors andmedical records and the data analysis of Starts Counting Profitspatient information are being spurred by comScore, a cloud-based provider of analytics services and solutions for the eCommerce marketplace, realized when itestimated annual savings to providers in began operations that the focus of Internet marketing wasthe tens of billions of dollars. shifting from visitor counts to profitability. comScore’s Customer Knowledge Platform provides a 360-degree view ofBig Data Analytics Guide 13
  14. 14. customer behavior and preferences as they visit sites Suntel’s traditional relational database was unacceptablythroughout the Internet. The service monitors surfing and sluggish. “We reached a point,” explains Tariq Marikar, directorbuying behavior at every site visited by consumers who have of information technology and solutions delivery, “at which weopted in to having their Internet behavior analyzed. were seeing a 20% overload on our production database, which was unacceptable to us.”With millions of Web users signing up to be monitored, thedata collected was enormous. comScore applies its analytics “Additionally, we wanted to be able to run reports and queriesto more than 40 terabytes of compressed data, while adding against years of historical data rather than just a few months.close to 150 gigabytes every week. We knew we needed to create a separate repository—a data warehouse—specifically designated and designed forDespite this volume of data, query-response time is excep- reporting and analytics in order to solve this problem.”tional. “We are able to mine the data and produce results forour customers much more quickly. That helps them market Suntel achieved its goal by adopting a column-store datamore effectively and generate more business,” says Ric Elert, warehouse designed for advanced analytics. “It’s very impor-vice president of engineering at comScore. tant to our business to be able to view large volumes of histor- ical data,” says Marikar. As with comScore, the “compressionThe company achieves 40% compression ratios using capability has meant that the data residing on our productioncolumn-store technology. Had they used a traditional database only requires about one-third the space.”approach, comScore says its storage costs would have beenmuch higher. The new platform can scale as Suntel increases the numbers of users and explores other strategies for tapping into this“The compression is extremely important to us because we valuable data. “We’re exploring ways to exploit this data trovehave fire hoses of data,” says Scott Smith, vice president data to develop ways to customize the customer experience acrosswarehousing. “We have a tremendous amount of data. More different sized customers and to implement programs todata than most people will ever see.” cross-sell and upsell our services,” says Markar.Suntel Introduces Airtel VodafoneCustomized Service Offerings to Sri Lanka Makes Better Decisions with Business IntelligenceAs Sri Lanka’s fastest growing telecommunications company, In Spain, Airtel Vodafone created a data warehouse to help itSuntel has 500,000 customers. The company puts the latest accurately analyze and predict customer activity. Thetechnology, innovative thinking, and an unprecedented service company developed a data warehouse with informationcommitment into customizing telecommunications solutions generated from multiple departments and organized the datafor its subscribers. according to the company’s business map. The data ware- house allows Airtel Vodafone to convert data into valuable business intelligence. Query demands on the company’s data warehouse are“We’re exploring ways to exploit this data intense. More than 1,000 employees use the data warehouse trove to develop ways to customize the for multidimensional analysis. The information has specifically designed structures for the data concerning customers, customer experience across different infrastructures, and company processes. This structure sized customers and to implement allows users to extract the data to create modeling and simulation processes. programs to cross-sell and upsell our services.” Data-mining techniques extract information on customer behavior patterns. Airtel Vodafone’s customer-facingTarik Markar personnel are able to input the information they collect on aDirector of Information Technology and Solutions Delivery daily basis, so that it is integrated with data already stored inSuntel14 Big Data Analytics Guide
  15. 15. the warehouse. This data is subsequently combined and technologies necessary to achieve these feats make itconverted into information structures for inquiries. possible for only the largest players in the industry to continu- ously throw faster hardware and network gear at the problem.The data warehouse environment comprises marketing data- For everyone else, says Larry Tabb, CEO of the Tabb Group,bases, call systems, customer service, GSM network statis- a financial services technology consultant, you need to betical data, invoicing systems, collections and retrievals, and all significantly smarter. To compete, Tabb says, “you need tologistics information. The marketing team uses the same raise the analytical barriers.”information as those in finance, although they look at it fromdifferent angles and use it for different analyses. In the face of these Big Data challenges, some in the analytics industry caution that companies might be smarter to take aHaving this structured data allows Airtel Vodafone to provide “more is less” approach to analyzing data sets. Sometimesboth detailed and summarized information of the company’sactivities directly from the data warehouse. These advantagesare helping Airtel Vodafone make informed business decisionsbased on customer activity. To speed decision making, firms are applying analytics to business processesVertical Industries Reap the Rewards of Data for financial transactions executed byOther industries are waking up and taking advantage of computers. Humans once were solelyBig Data analytics. In healthcare, the move to electronicmedical records and the data analysis of patient information responsible for the decisions, but noware being spurred by estimated annual savings to providers in only computers can work as fast as thethe tens of billions of dollars. In the manufacturing sector,outsourcing a supply chain may save money, according to data is moving.McKinsey & Company, but it has made it even more criticalfor executives, business analysts, and forecasters to acquire,access, retain, and analyze as much information as possible you might hear arguments that applying analytics to smallerabout everything from availability of raw materials to a data sets is, in effect, “good enough.” More often than not, thispartner’s inventory levels. argument is made by those that can’t analyze large data sets.In these and other industries, users are lining up outside the As Google Chief Economist Hal Varian observes, analyzing aCIO’s door, asking to analyze the incoming flood of data. And small, random slice of data can indeed yield valid results. Butwhen they get access, users want query-response times to get a truly random data set, that sliver of information needscomparable to what they experience using search engines to come from a massive amount of information. Without asuch as Google and Bing. large enough pool of data to draw from, the validity of your analytics processes can be called into question. In otherIn some markets, response times that satisfy humans aren’t words, Big Data generates the best valid data. nfast enough. These enterprises demand machine-to-machinespeeds for analytics.According to the publication Wall Street & Technology,financial services companies are under increasing pressureto accelerate decision making from “microseconds tomilliseconds to nanoseconds.” To speed decision making,firms are applying analytics to business processes forfinancial transactions executed by computers. Humansonce were solely responsible for the decisions, but now onlycomputers can work as fast as the data is moving. The priceyBig Data Analytics Guide 15
  16. 16. CONSIDERING BIG DATA ALONE IS INSUFFICIENT; ANALYTICS MUST ALSO BECOME PERVASIVE ACROSS THEENTERPRISE IN ORDER TO TRULY LEVERAGE THE OPPORTUNITY.Big Data Is Only a Small Part of the OpportunityBy Mike Upchurch, Chief Operating Officer, Fuzzy Logix their time was high. Analysis also required moving data from a database to an analytics server, processing it and pushing it, back. Just moving the data was 80% of the work—akin to trucking our pile of sand 10 miles to sift it.The opportunity Big Data represents goesbeyond the data and the related new Today, new, powerful data warehouse systems using in-database analytics can quickly ingest and process Big Datatechnologies that capture and store it. The real wherever it resides. What’s more, business users can nowbenefit is that organizations can derive better sift through data using familiar reporting tools, gaining easybusiness intelligence from far more sources access to powerful on-demand analytics and allowing datathan ever before, and make it available to scientists to focus on building models instead of running reports. Best of all, these new solutions generally cost arounddecision makers at every level. The keys to 20% less to build than traditional platforms and performsuccess are designing your Big Data analytics more than ten times support business goals and enablingdecision makers to take action. Start With “Why” Data analysis is more accessible than ever, and it can solve many problems—but not all of them. The key to identifyingIt’s easy to collect large amounts of data. Knowing what to do which problems to tackle is to start with “why.” Why are wewith it all—and making changes based on what you learn—is analyzing Big Data? First, assess your strategic goals. Thesethe challenge. We can liken the task to searching for diamonds could be growing market share, controlling cost and risk, orin a giant pile of sand. Storing the sand is easy, but sifting understanding customer behavior. Then, determine if usingthrough it requires a special set of tools, as well as a sufficient analytics will deliver value.understanding of what you’re looking for, why, and what you’regoing to do when you find it. There are two important questions to answer: Can the company use data models to derive insight, and can it actHistorically, data analysis has been a story of complexity, on the results? Working through this process will helplimited capacity, elaborate tools, cryptic results, and poor determine where your organization can realize value fromdistribution. Special equipment was required; only a small Big Data analytics.number of people knew how to use it; and the demand on Changing Company Culture Companies need a focused plan, great execution, the right technical platform, and the ability to operationalize the resultsToday, new, powerful data warehouse of analysis. Without accompanying cultural change, however, those things will only deliver a fraction of the potential value ofsystems using in-database analytics Big Data analysis.can quickly ingest and process Big Data Let’s go back to the diamond mine one more time. They havewherever it resides. new sifting equipment that tells the miners where the highest- value diamonds are, but the miners aren’t authorized to react to the information. The best equipment can’t make up for broken culture. Employees should be able to run analytics and see actionable answers on demand: a forecast of how close the sales team16 Big Data Analytics Guide
  17. 17. It’s crucial to create a culture that rewards decisions and encourages analytics innovation, which may require modifying incentive and bonus to meeting this month’s numbers, a customer’s credit history and the actions of customers with similar histories,score, or a report of which advertising keywords to buy an analytics engine can recommend actions that will reducetoday. Armed with information, employees must also be churn, or suggest products or services that will be thecomfortable and confident taking action before the value of customer’s next likely purchase. One call center leveragedthe insight diminishes. Big Data analytics and saw a 10% reduction in churn, an 8% increase in per call revenue, and a 12% improvementAs a company incorporates the use of analytics, employees in cross-sale revenue.will have ideas about how to improve on the original models.Building a culture that encourages constant testing and Health care organizations are using Big Data analyticslearning—as well as providing access to a flexible platform when evaluating care quality and efficiency. Using traditionalthat can accommodate new ideas—will greatly improve the methods, analyzing more than 700 million lines of claims datavalue companies can reap from Big Data. can take six weeks and a dedicated team of analysts, and only produce reports twice a year. With Big Data solutions, riskIt’s crucial to create a culture that rewards decisions and management teams can now run the models in 22 minutesencourages analytics innovation, which may require modifying and take immediate action to improve quality of care,incentive and bonus structures. Not allowing employees to act reducing the window during which risk can go unnoticedis the most common point of failure for analytics projects— from six months to less than a week.don’t make that mistake. It’s rarely mentioned in discussionsof Big Data, but it can make or break an analytics initiative. Big Data analytics are ushering in a new era of predictive insight that is changing how companies operate and engageMaximizing Results with their customers, suppliers, and employees. To takeMany companies are succeeding at their search for value advantage of the opportunity, companies must start within Big Data. They have the systems and infrastructure the “whys,” align analytics projects with business needs, andto capture and analyze Big Data; they have operational quantify the value that can be created. To realize the value,processes in place; and their employees have permission employees must have access to powerful, innovative, andto act on the results. For these companies, the payoff can proven technology, participate in the process, understandbe dramatic. the results, and be empowered to act. Get all of this right, and your diamonds will shine bright, creating competitiveFor example, equity traders may need to buy or sell assets advantage and financial gain. nduring the trading day to balance their portfolios, but oneday’s Opera feed can contain data for 500,000 to 1 milliontrades. If portfolio risk can only be calculated overnight, then Mike Upchurch is responsible for customerinstitutions are exposed to an unquantifiable amount of risk acquisition, partnerships, global operations,during each trading day. With Big Data analytics, traders can and corporate culture at Fuzzy Logix.get real-time pricing and calculate risk throughout the day. Previously he worked at Bank of America,The result is that they can rebalance their portfolios at the leveraging trading instruments to createexpense of less agile traders. Millions of dollars can be won consumer products, mining consumerand lost by having better information than the institution on data to identify trading opportunities, andthe other side of the trade. Other examples of capitalizing on building and implementing a strategy that grew telephoneBig Data include modeling loan default risk on demand, and mortgage lending from $9 billion to $22 billion in fourstress-testing entire portfolios in a fraction of the time years. He has also held a number of strategy andrequired by traditional solutions. operational roles at global technology companies.Call centers use analytics to better serve customers, reducechurn, and cross-sell new products. By analyzing a customer’sBig Data Analytics Guide 17
  18. 18. Business Analytics RoadmapBusiness Value through Operational BIAlign operational and real-time business analytics andanalytics technology with true business requirementsand capabilities to ensure greater success in reachingbusiness and IT goals.President of Intelligent Solutions, Inc. and This Checklist Report helps you determine how to align theFounder of the Boulder BI Brain Trust implementation of operational and real-time BI and analytics technology with true business requirements and capabilities to ensure greater success in reaching business and IT goals.Excerpted from TDWI Checklist Report, “Delivering HigherBusiness Value with Operational Business Intelligence and 1. RECOGNIZE THAT NOT ALL ANALYTICS MUST COMEReal-Time Information” FROM THE DATA wAREHOUSE ENvIRONMENT. The data warehouse (DW) is a key supplier of data analytics,Operational BI (OBI) is a popular topic in most but it’s not the sole supplier of analytics. Other forms ofbusiness intelligence (BI) shops these days, analytics are needed for a fully functioning OBI environment. Because many analytics used in OBI require low-latency orand rightfully so. OBI enables more informed real-time data, organizations try to speed up the overallbusiness decisions by directly supporting processes of the DW—trickle-feeding the data, automatingspecific business process and activities. analyses, and so on—in an effort to make it the sole supplier of analytics. Although this approach works for some low-OBI has had a dramatic impact on traditional BI environments latency analytics, at some point the DW team must turn toand on a new audience of BI users. These users now have other analytical techniques to complete the OBI picture.immediate access to the insights they need when makingdecisions about customers, products, and even campaigns One of these techniques is event analytics. Event data iswhile these business activities are happening. created by business activities (generated by banking transac- tions [ATM], retail operations [POS, RFID], market trades, and Web interactions) or by system events (generated by sensors, security devices, or system hardware or software). Event analytics applications often perform their analyses even before the transactional data is stored in an operational18 Big Data Analytics Guide
  19. 19. system. For example, many fraud-detection applications Another trade-off is the soundness and flexibility of the archi-analyze transactions for fraudulent characteristics first and tectural infrastructure in terms of allowing for delivery ofthen store them in transactional systems for further information in different latency time frames (more on thisprocessing. Obviously, the DW contributes to the overall OBI later). Building an OBI solution that is inflexible or fragile justenvironment by generating the fraud models used by the to meet an arbitrary time frame may spell disaster. If theevent analytics software. action time requirement changes (and it almost certainly will) from two hours to one hour, you don’t want to have to rebuildAnother technique is to make BI analytics (or its results) the entire architecture.available as callable services within an operational workflow.Embedded BI services can be external to the workflow (as a To avoid this situation, the BI implementers must understandpart of a service-oriented architecture) or included within the how the business community interacts with OBI, from eventworkflow itself. These services come in two flavors. The firstcalls a stored analysis or model, uses it dynamically duringthe workflow, and receives the results before invoking the nextactivity—for example, calling a stored analysis to dynamically Embedded BI services can bedetermine a loan applicant’s credit worthiness. The second external to the workflow (as a part oftype retrieves the static results from an earlier analysis; forexample, a customer service representative (CSR) retrieves a service-oriented architecture) ora customer’s lifetime value score or segment ID stored in a included within the workflow itself.DW. Both types are employed by a business process orperson to support real-time or near-real-time businessdecisions and actions. occurrence to action taken. Interactions must include theThe combination of traditional data analytics, embedded BI impact of the growing usage of tablets and mobile, and event analytics forms the foundation of OBI. All OBI must reach its audience with the appropriate informationthree must come together at appropriate points in the work- formatted for the myriad mobile devices available today.flow to provide a mature and effective operational decision-making environment. 3. DETERMINE THE PROPER INFRASTRUCTURE FOR BUSINESS-CRITICAL OPERATIONAL BI.2. MATCH REAL-TIME CAPABILITIES FOR INCREASING BI Although traditional BI processing is often critical to businessAGILITY TO ACTUAL BUSINESS NEEDS. operations, a temporary failure of the BI system will not typi-There is a lag between the time an event happens and the cally affect short-term business operations. Also, given thattime a company responds to it. This lag is caused by several the BI system is separated from operational processing, itfactors, such as preparing the data for analysis, running the means that BI processing has little effect on operationalanalysis, and determining the best course of action based on performance except during the capturing of operational data.the results—for example, taking action when a campaign sellsa product that is about to run out of stock. Clearly, the ability The situation with OBI is different from traditional BIto reduce the time to action here (stopping the campaign or because it is closely tied to the daily operations of thechanging the featured product) can have significant impact on business. A failure in an OBI system could severely impacta company’s revenues and reputation. This is BI agility. It business operations. This risk is especially relevant for OBIrequires that the action time match the business need. applications that support close to real-time decision making, such as fraud detection.However, there is a trade-off. Is it timely enough for the busi-ness or is it actually too fast? Even if the business requires There are several approaches to supporting OBI, includingreduced latency, can the business users correctly process the embedding BI in operational processes, accessing live opera-inputs that quickly? Can the operating procedures handle the tional data, and capturing operational data events and trickle-time frame appropriately to ensure a correct reaction? There feeding them to a DW. All of these approaches have the abilityare many moving parts in an OBI environment, and any that to affect the performance of operational systems.are out of sync or incomplete can cause an erroneous deci-sion to be made. In this situation, the cost of creating such a It is very important, therefore, that the infrastructure of thelow-latency BI environment may be more than the actual BI system, its underlying DW environment, and related opera-benefit the company receives. tional systems be capable of providing the performance,Big Data Analytics Guide 19
  20. 20. scalability, availability, and reliability to meet OBI service 5. UNDERSTAND THAT OPERATIONAL BI IS MORE THANlevels. The cost of providing such an infrastructure increases SIMPLY CAPTURING MORE TIMELY these service levels approach real time, and these costs It is often assumed (incorrectly) that OBI simply involvesmust be balanced against the business benefits achieved capturing more timely data. Certainly data consolidationand the ability of the organization to exploit a more agile (ETL), data replication, and data federation (enterprise infor-decision-making environment. mation integration [EII]) technologies have advanced to the point that we can capture data and make it available in a far4. UNDERSTAND THAT OPERATIONAL BI IS NOT JUST A more timely fashion than ever before. For example, using log-TECHNOLOGY SOLUTION. based changed data capture (CDC) has distinct advantagesIt’s critical that BI implementers be able to tie BI applications for speeding up data integration and processing for a operational applications and, even more importantly, with Without doubt, real-time or low-latency data is an importantoperational processes. Yes, technology is important, but feature of OBI processing. In addition, there are other factorsperhaps just as important are the standard operating proce- that need to be considered when improving BI agility anddures (SOPs) that must be followed by business personnel. supporting faster decision making.Many BI implementers do not realize that their OBI solutionimpacts how people perform their jobs. Without under- Once operational data has been captured, it needs to bestanding how SOPs will be affected, the OBI team can cause analyzed and the results delivered to the BI consumer, whichsevere problems with operations or, worse, find their solutions may be a business user or another application. The time itbeing ignored or circumvented. takes to analyze the data increases the time (the action time) it takes for a business user or an application to make a deci-As a first step, the BI team should study, understand, and sion. It is important, therefore, that the actual queries used indocument the full business workflow using the new BI applica- the analysis are optimized for good performance. It is also important that the underlying query processing engine is opti- mized for efficient analytical processing. In some instances,As a first step, the BI team should the analytical results may be precalculated to reduce action times (customer lifetime value scores, for example).study, understand, and document thefull business workflow using the new The efficient delivery of results to the BI consumer is also important for OBI success. The delivery medium used (dash-BI application board, portal, mobile device, action message) must be selected to match the action time requirements of the busi- ness. The availability of automated decision-making featurestion. OBI applications can cause big changes to processes and such as alerts, recommendations, and decision workflows canprocedures. When they do, the team must determine how the help business users make faster decisions. In near-real-timeSOPs must change. For instance, will they need to be rewritten decision-making situations (fraud detection, for example),or enhanced to include the new OBI application? What impact fully automated decision-making features may be employed. nwill this have on the workforce? Who will create and maintainthe new SOP? This contribution was extracted from “Delivering Higher Business Value with Operational Business Intelligence andThe team must also determine which personnel will be Real-Time Information.” To read the entire document, go to:affected by the new procedures and what training they will The team must study how these personnel make delivering-higher-business-value-with-operational-bi-and-decisions, how they access and use information, and how real-time-information.aspxthey monitor the impact of their decisions on the company.Training must be ongoing and flexible to accommodate theinevitable turnover in operational personnel. Some of the Claudia Imhoff, Ph.D. is an analyst andworkforce may immediately grasp this new paradigm; speaker on business intelligence and theothers may not. infrastructure to support these initiatives. She is the president of Intelligent Solutions, Inc., a data warehousing and and founder of the Boulder BI Brain Trust. She has co-authored five books on these topics and writes articles and research papers for technical and business magazines.20 Big Data Analytics Guide
  21. 21. A NEw APPROACH IS NECESSARY IN TODAY’S ALwAYS-ON wORLD. SAP IS DELIvERING A PORTFOLIO FOR THEREAL-TIME BUSINESS.Real-time Data Platform for a Real-time WorldBy Amit Sinha, Head of Database and Technology While business is happening faster, many IT departments areInnovation, SAP still using traditional data management tools designed in the 1980s when the pace of life and business was slower, and the amount of data was much smaller.Professor Richard Wiseman, author of The challenge is that today enterprises are looking to analyzeQuirkology, compared the ‘pace of life’ in 31 terabytes or petabytes of data in the moment, instead of days or weeks in the past. Yet, the underlying infrastructure hascountries by studying how fast people walk. remained status quo, with enterprises being forced to spendThe study definitely fits the title of his book! time ‘shoehorning’ old technology into their data centers toMore interesting is that the overall pace of life address new problems. And many of them have reached theincreased by 10% over a 10-year period, and breaking’s only getting faster. Smartphones, wireless Instead, a new approach is required that can not only mine thenetworks, and an ‘always on’ lifestyle is further information and make sense of it, but do it in real time. Toaccelerating the pace of people’s lives and of empower organizations to remain competitive in today’sbusiness and generating vastly more data at constantly evolving market, SAP has committed to helping them unleash the value of Big Data through a new approach tothe same time. data management. It all starts with a foundation based on the SAP HANA database, a state-of-the-art in-memory platform, whichInstead, a new approach is required that allows enterprises to cut out the complexity that’s crept into IT environments. SAP HANA’s extreme performance andcan not only mine the information andmake sense of it, but do it in real time.Big Data Analytics Guide 21
  22. 22. innovation for the next generation of applications is redefining These solutions together are just the beginning of SAP’s goalthe database market by helping customers access and deliver of providing customers with a single, logical, real-timeinformation at speeds up to 100,000 times faster than platform for all transaction and workloads. By leveragingpreviously available. the industry-leading SAP Sybase data management products, customers will be able to transact, move, store, process, andSurrounding HANA, the centerpiece of SAP’s real-time data analyze data in real time while reducing overall costs.platform, are several components that bring the best ofdatabase innovation forward. Sybase IQ, the # 1 column The old way of doing things is no longer acceptable.database on the market, offers enterprises the best overall The new world of data needs a new data platform, andtotal cost of ownership by reducing administration by SAP is committed to helping enterprise IT departments75% and reducing data storage volumes by more than 70% evolve from complex, slow-moving entities into a morethrough advanced data compression algorithms. simplified architecture that enables Big Data, cloud services, as well as analytic, transactional, and mobile applicationsSAP Sybase Adaptive Server Enterprise (ASE) is the #1 trans- while preserving investment in existing applications in aactional database and in use by most Wall Street firms. non-disruptive way. nSAP Sybase ASE delivers top performance for enterprises,reduces risk due to security breaches or system failures andincreases efficiency by simplifying administration and effi- Amit Sinha leads marketing for SAP’sciently using hardware and storage. technology platform, data management, and real-time applications. Prior to this role,Another piece of the real-time database puzzle is SAP Sybase he led the market introduction of SAP HANA.SQL Anywhere, the #1 mobile and embedded database that Previously, as Vice President of Businesssupports advanced synchronization, out-of-the-box perfor- Network Transformation, Amit was respon-mance with little to no DBA support and the ability to enable sible for driving the phenomenon of collab-applications in remote locations. oration across business boundaries through innovations in SAP’s portfolio. He has worked with customers on newLastly, enterprise information management (EIM) cloud-based collaborative applications that empowersolutions from SAP enable enterprises to rapidly ingest, people and communities to collaborate, leverage infor-cleanse, model, and govern data in order to improve the mation for collaborative decision making, and ultimatelyeffectiveness of Big Data across operational, analytical, enhance the company’s business model. Amit is a grad-and governance initiatives. uate of the Indian Institute of Technology (IIT) Bombay and the Haas School of Business at the University of California, Berkeley. By leveraging the industry-leading SAP Sybase data management products, customers will be able to transact, move, store, process, and analyze data in real time while reducing overall costs.22 Big Data Analytics Guide
  23. 23. ADvANCEMENTS TO IN-MEMORY DATABASES, LOwER MEMORY COSTS, AND THE COMBINATION OFTRANSACTIONS AND ANALYTICS MOvE HANA INTO A CLEAR LEADERSHIP POSITION.How HANA Changes the Database MarketBy Ken Tsai, Vice President of HANA Solution Decision-makers “swimming in information” need a databaseMarketing, SAP designed to navigate data’s deep waters. While some databases can be a life raft, helping an organization stay afloat, the SAP HANA in-memory database gives a company fleet command over oceans of Big Data. This advanced, high-In the 2011 spy thriller Page Eight, the director performance database is dramatically changing the market.general of Britain’s domestic intelligence The cost of memory is one sign of market change. Whenagency, MI5, played by veteran actor Michael traditional databases were first designed memory wasGambon, utters a lament expressed by many extremely expensive. The big database vendors traded thea corporate CEO. “This building is swimming speed of memory for more cost efficient storage on information,” he complains. “We have But that has changed and dramatically. In 1990 a terabyte of memory cost more than $100 million. Today a terabyte ofinformation coming out of ears.” What’s memory costs under $5,000. In three years it’s estimated thatdifficult, he adds, is to determine whether price will fall to one quarter of that and by 2018 users will paysomething is important or not. one-thirtieth. Given that CPU memory is at least 50,000 times faster than accessing data on a mechanical disk drive, with memory being that cheap the reasons not to use in-memory databases have vanished.Given that CPU memory is at least Combining analytics and transactions is another change50,000 times faster than accessing data upending the market. HANA provides the power needed in both analytics and transactions to streamline businesseson a mechanical disk drive, with memory activities. In fact, SAP HANA portends the end of thebeing that cheap the reasons not to usein-memory databases have vanished.Big Data Analytics Guide 23
  24. 24. separation of online application processing (OLAP) and SAP HANA scales linearly along with the growth in the volumeonline-transaction processing (OLTP) database functions in and velocity of a company’s information sources. SAP HANA’slarge organizations, providing instead a single, massive data columnar architecture is data agnostic, ideally suited for thestore for both transactional and analytical database activity variety of Big Data pouring into organization’s today. There’swith performance levels previously unimaginable by decision no practical limit to the capacity of SAP HANA.makers. Such a combination of business functions will berevelatory for corporate leaders. Hasso Plattner, in his 2009 Most important, SAP HANA is fast. Not just whiteboard-theorypaper “A Common Database Approach for OLTP and OLAP fast, but real-world business fast. Take Liechtenstein-basedUsing an In-Memory Column Database,” concludes that when Hiliti Corp., a global provider of value-added products to thethe merging of the two processes occurs “that the impact on construction and building maintenance industry. Its applica-management of companies will be huge, probably like the tion of the SAP HANA database merged transactional andimpact of Internet search engines on all of us.” analytic functions to improve the sales and support process by many orders of magnitude; in one case, improving theChange for the Better response time for analyzing 53 million customer data recordsSAP HANA is a 100% in-memory database software appliance to two to three seconds from what once took two or threedesigned to run on Intel processors and optimized for specific hours. In Japan, Mitsui & Co. Ltd.’s retail operations experi-advances in chip design such as multi-core processors, enced a stunning 400,000 times performance improvementshared memory, and multi-socket topology. According to Intel, in its inventory management application with SAP HANA over“SAP HANA enables real-time decision making by bringing all the prior database’s performance. And Germany’s T-Mobilethe data in your enterprise within the reach of decision implemented SAP HANA to analyze huge data volumes inmakers in seconds, not weeks or months, in an easy-to-use seconds—up to 1 billion rows and a 300 trillion record set in asformat so your company can run smarter and perform better.” little as 16 seconds, dynamically modifying its marketing andThe company concludes that SAP HANA delivers “an unprece- promotions vehicles to deliver more effective results.dented robustness in real-time business analysis.” Leading through InnovationCisco, for example, has applied HANA to its seasonality anal- The arrival of SAP HANA has already changed the marketysis of customer purchase sentiment to a mere five seconds landscape. Competitors are following SAP’s lead and areno matter filters it applies to the report. While Lockheed announcing in-memory databases in an attempt to stay in theMartin has improved its labor utilization report by 1,131x in performance game. However, because SAP began its develop-responsiveness. And fashion and fragrance leader PUIG, with ment years ago, it has a long head start and will be able stay iniconic brands such as Prada and Paco Rabanne, are now able the lead for the foreseeable future as it continues to predict sales trends in real-time for new products andmarkets with 400x boost in report execution. However, the biggest opportunity SAP HANA creates will be for business. It will unleash powerful and innovative applica- tions that exploit the wealth of knowledge within a company’s trove of Big Data. It will improve the capabilities and respon- siveness of operations, finance, marketing, engineering, andSAP HANA’s columnar architecture is virtually all areas of business. No longer will CEOs feel likedata agnostic, ideally suited for the they are swimming in information. Rather, they will be sailing across it, fully in control and charting new opportunities forvariety of Big Data pouring into increased growth and profitability. norganization’s today. There’s no practicallimit to the capacity of SAP HANA. Ken Tsai is the head of SAP HANA product marketing team at SAP and is responsible for driving marketing, communication, and adoption of SAP HANA in-memory data platform worldwide. Tsai has 17 years of experiences with application development, middleware, database, and enterprise applications. He has been with SAP for the past 7 years and is a graduate of University of California, Berkeley.24 Big Data Analytics Guide
  25. 25. TO PROvIDE BUSINESS INTELLIGENCE FOR EvERYONE IN AN ENTERPRISE, DATA DELIvERY AND ANALYSISMUST BECOME MORE NIMBLE THAN DATA MARTS CAN BE.DBTA: Data MartsCan’t Dance to Data’s New GrooveBy John Schitka, Senior Product Marketing Manager, and still widely rely upon—are no longer sufficient for today’sSybase IQ needs. You can put data marts near the top of that list. Data marts were a reaction to the extreme performance limi- tations of traditional enterprise data warehouses. The dataLimitations in scalability and business demand warehouse itself, which came of age in the 1990s, representedfor analytics are causing IT departments a tremendously enticing vision—offering to virtually every department across the enterprise an opportunity to see itsto rethink the traditional data warehouse/ performance metrics and find out what’s working and mart strategy in favor of a powerful,centralized business analytics information grid. That is, data warehouses would have answered all of those questions, if only users could get to the data. Most organiza-Few things in the world are changing as dramatically as data. tions quickly discovered that data warehouses—with theirData has tapped out a powerful rhythm to keep time with centralized, brittle architecture—performed abysmally undertechnology’s bleeding edge, leaving many technologies strug- unpredictable workload demands. Even the load of just a fewgling to keep up. It should come as no surprise, then, that users could degrade performance precipitously. It quicklymany of the data strategies that IT departments developed— became clear that if they wanted to scale the data warehouse, organizations would need to replicate and distribute the data locally. Thus, data marts were deployed. The Power of PredictionBusiness leaders are looking for ways to While data marts were never a perfect solution, they adequatelygain deeper insights from data, to enable addressed businesses’ urgent need to let stakeholders from across the organization explore the data and uncover themore business users to search for these insights they hold. But while data mart deployments havedeep insights, and to directly embed these largely continued unabated for the past decade, business hasinsights into core business processes.Big Data Analytics Guide 25