In-Memory Analytics: Strategies for Real-Time CRM


Published on

We are on the brink of true “in-memory analytics,” a technology that will allow operational data to be held in a single database that can handle all the day-to-day customer transactions and updates as well as analytical requests—in virtually real time. To drive the shift to this new tech­nology, CIOs must make sure the business understands its advantages and devise a governance strategy to manage its rollout and monitor its use.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

In-Memory Analytics: Strategies for Real-Time CRM

  1. 1. Perspective Olaf Acker Dr. Florian Gröne Adrian Blockus Dr. Carsten Bange 3 columns width 2 columns widthIn-Memory AnalyticsStrategies forReal-Time CRM
  2. 2. Contact InformationAtlanta Canberra Frankfurt New YorkRalph Alewine David Batrouney Olaf Acker Jeffrey TuckerPartner Principal Partner Partner+1-404-519-0184 +61-2-6279-1235 +49-69-97167-453 jeffrey.tucker@booz.comBeirut Chicago London São PauloRamez Shehadi Eduardo Alvarez Greg Baxter Jorge LionelPartner Partner Partner Principal+961-1-336433 +1-312-578-4774 +44-20-7393-3795 jorge.lionel@booz.comBerlin Delhi Milan ShanghaiFlorian Gröne Suvojoy Sengupta Enrico Strada Andrew CaineySenior Associate Partner Partner Partner+49-30-88705-844 +91-124-499-8700 +39-02-72-50-93-00 andrew.cainey@booz.comAdrian Blockus Düsseldorf Munich SydneyAssociate Dietmar Ahlemann Bernhard Rieder Peter Burns+49-30-88705-896 Partner Partner +49-211-3890-287 +49-89-54525-670 +61-2-9321-1974 Booz & Company
  3. 3. EXECUTIVE For years, the process of devising customer data queries and creating business intelligence reports has been a lengthy one.SUMMARY That’s because the information needed must be pulled from operational systems and then structured in separate analyti- cal data warehouse systems that can accept the queries. Now, however, we are on the brink of true “in-memory analytics,” a technology that will allow operational data to be held in a single database that can handle all the day-to-day customer transactions and updates as well as analytical requests—in virtually real time. The advantages of in-memory analyt- and software, companies can cut the ics are many: Performance gains will total cost of ownership of their cus- allow business users to retrieve better tomer data efforts significantly. queries and create more complex models, allowing them to experiment To drive the shift to this new tech- more fully with the data in creating nology, CIOs must make sure the sales and marketing campaigns, and business understands its advantages to retrieve current customer informa- in terms of better customer intel- tion, even while on the road, through ligence and lower overall cost. To do mobile applications. The result- so, they must make a strong business ing boost in customer insights will case for the transformation—always give those who move first to these a challenge with business intel- systems a real competitive advantage. ligence systems—including ease of Companies whose operations depend use, better analytical reports, and on frequent data updates will be able better decision making. And they to run more efficiently. And by merg- must devise a governance strategy to ing operational and analytical sys- manage the technology’s rollout and tems, with their attendant hardware monitor its use.Booz & Company 1
  4. 4. ANALYTICS Consider the plight of an insurance sales rep sitting down with a customer other products that might suit the customer’s current needs. As a result,ON THE FLY to discuss changes to his life insurance the rep has a much better chance of policy. The only information he has up-selling the customer on his current about the customer may be months policy and cross-selling him on other old, so pricing a new policy given the products. customer’s changing circumstances will have to wait until the rep can add Until recently, large customer relation- the new information to the system ship management (CRM) systems back in the office. And because he depended on two separate databases: can’t analyze the new data immedi- The operational database maintained ately, there’s no opportunity to offer the day-to-day, high-volume trans- the customer specific details on new actional data, while the analytical products he might be interested in. database took the data needed to per- form specific customer analyses and Now, however, a new technology stored it separately. As a result, it was called “in-memory analytics” lets that impossible to run real-time queries sales rep enter new data into his com- against the most up-to-date customer pany’s database from a tablet com- data. Thanks to major advances in puter sitting on his lap as the meeting the speed, cost, and sophistication of is taking place. In real time, he can storage and memory technology and analyze the customer’s new situation in the power of processors, however, and generate current pricing informa- the promise of real-time analytics— tion, as well as information about through which business users can “In-memory analytics” lets a sales rep enter new data into his company’s database from a tablet computer on his lap as he is meeting with a customer.2 Booz & Company
  5. 5. access the full set of operational data formance is dramatically improved. growth through more powerful up-when creating their reports—is finally This increase in performance allows selling and cross-selling.becoming a reality (see Exhibit 1). end-users to run more complex que- ries and gives them better modeling • Lower costs: Total cost of owner-By giving business users access to truly capabilities, adding up to greater ship is significantly lower comparedlive customer data, in-memory tech- business value (see “Inside the to traditional data warehouses, innology will transform how companies Technology,” page 4). part because all the data is nowanalyze and use that data. As such, it stored in one place. And while in-offers three significant benefits over • Customer value creation: In- memory technology allows for thetraditional data warehouses: memory analytics gives business analysis of very large data sets, it is users instant self-service access to much simpler to set up and main-• Performance improvements: the information they need, pro- tain. Rapid departmental deploy- Because users can now interact viding an entirely new level of ments can free up IT resources with and query data in memory, customer insight that has the very previously devoted to responding to response time and calculation per- real potential to maximize revenue requests for reports.Exhibit 1An Integrated CRM Architecture Can Speed Up Analytics Requests TODAY THE FUTURE Analytical CRM Operational CRM Real-Time CRM Transactional data, analytics User Interface Mining requests Campaigns Campaigns Sales Service Segmentation Sales Real time Real time Decision Models Customer Care In-Memory Analytics Batch scores, modelsSource: Booz & Company analysisBooz & Company 3
  6. 6. Inside the Technology Until very recently, the effort to create, store, and analyze critical transactional data related to all kinds of business activities was a cumbersome and expensive process. Operational data—the high- volume, transaction-heavy data generated through a variety of business processes, including sales, order management, and customer care operations such as call centers—was maintained in huge data warehouses to ensure reliable performance and data integrity. And because the sources of the operational data typically varied significantly, maintaining it all in a single database with the homogeneous data model that could serve as a “single point of truth” proved very beneficial. Meanwhile, the analytical data used to gather customer and performance insights, to segment customers, and to model and predict future behavior through customer usage and payment history, for example, was typically drawn periodically from the operational database and maintained separately. As valuable as that analytical data proved in boosting customer profitability and allowing more efficient up-selling and cross- selling, the architecture had some very real downsides. Because it had to be duplicated periodically, the data in the analytical data warehouse was frequently at least a day or two—and sometimes as much as a month—out of date. A specially designed data mart had to be built for practically every new analysis request, which meant long deployment cycles, low project success rates, and ever-growing data volumes at ever- higher cost. And the process introduced an additional layer of complex analytical software into the enterprise architecture, requiring additional training. Typical business users could only generate pre-defined standard analytical reports; anything more complex needed to be set up by a handful of power users. Now, however, this long-time separation between operational and analytical databases is finally coming to an end. With the emergence of multicore processors, increasing clock speeds, and 64-bit technology, combined with the rapid decrease in the price of memory, data can be managed entirely in main memory. While the idea of managing data in memory is not new, efforts to do so were hampered until recently by the fact that the old 32-bit architecture could address only 4 gigabytes of memory, and processors were not fast enough to give in-memory databases any real performance advantage. But with new4 Booz & Company
  7. 7. ways of organizing, buffering, and accessing the data, the performance improvements are significant (see Exhibit A). The capacity capabilities of these systems are now equaling those of large disk-based databases. For example, a pilot implementation of a 40-terabyte in-memory database was recently completed, and theoretically, databases as large as 16 exabytes (16,384 petabytes) could be managed with in-memory technology, based on today’s architecture. Throughput is seven times higher, and the response time is virtually Guidelines: instantaneous. 11.0 million aölkdfölka Exhibit A 32.8% In-Memory Technology Offers Vastly Superior Response Time and Throughput 30.1% RESPONSE TIME THROUGHPUT TABLE HEADI (IN MICROSECONDS) (IN THOUSANDS OF TRANSACTIONS PER SECOND) A4 format: - width for 3 colu 5x 7x - width for 2 colu 700 140 Letter format: - width for 3 colu 19x 600 120 - width for 2 colu 500 100 Lines: 0,5 pt On-Disk Lines for legend: Database 400 80 In-Memory 300 60 Database Note: Please always de otherwise InDesi 200 40 file. These colors can 100 20 Approved Colo 0 0 Update Select Source: Booz & Company analysis 3 columns width 2 columns widthBooz & Company 5
  8. 8. DRIVING Several factors—involving improved analytical speed and performance mobility is becoming the norm. In every industry, customers nowFACTORS and better analytical results—are expect instant responses to their driving the push to in-memory requests and questions; in this technology at the enterprise level. environment, in-memory technology allows companies to create an Business Demand entirely new level of customer The delays that typically arise experience, and it gives users instant out of the periodic extraction, access to the data they need to transformation, and loading (ETL) provide online self-service, real-time of data from the operational to the customer segmentation and dynamic analytical systems may be generally pricing. acceptable in doing trend analysis and forecasting. But traditional data Time-sensitive industries like warehouses simply cannot keep pace airlines and transport logistics with today’s business requirements will now have access to real-time for fast and accurate analytical information in running their data, especially in situations where operations, and the resulting Traditional data warehouses simply cannot keep pace with today’s business requirements for fast and accurate analytical data.6 Booz & Company
  9. 9. increase in efficiency will become interactive data visualization response times, as users nowa significant competitive advantage as the end-user interface, which expect instantaneous results.(see Exhibit 2). allows many more people in Since in-memory analytics allows the organization to make use data to be accessed directly fromPerformance of Analytics of these systems. However, the memory, query results come backMost analytical applications have new interfaces, which offer users much more quickly than theymoved beyond the spreadsheets interactive dashboards and the would from a traditional disk-and tables offered by traditional ability to perform much more based data warehouse. The time itreporting tools and now use intuitive tasks, demand very fast takes to update the database is alsoExhibit 2Selected Benefits of Real-Time Analytics Across Different Industries Finance ILLUSTRATIVE EXAMPLES Automated trading, online banking, huge volumes Defense Travel Just-in-time Exploding people tracking “look-to-book” ratios Real-Time Business - Optimization - Operations Telecoms - CRM Online Gaming - BI Value-add services, User authentication billing, subscriber & authorization, database consolidation, credit check, fraud management Retail best bets Multichannel selling, cross-selling/ up-selling, personalizationSource: Booz & Company analysisBooz & Company 7
  10. 10. significantly reduced, and the system and customer data put the onus volumes and the proliferationcan handle many more queries (see on organizations to maintain this of applications dependentExhibit 3). data and keep it available for years. on databases, companies are Much of this data still resides on struggling to manage the manyGrowing Data Volumes legacy systems, which are costly to business intelligence (BI) effortsThe sheer amount of transaction operate and maintain. In-memory being developed throughout theirdata being digitally captured and analytics allows such data to be organizations. In many cases, forstored is increasing exponentially, accessed rapidly on an ad hoc basis, instance, users simply want accessas are unstructured forms of data without having to build additional to their specific transactionalsuch as e-mail, video, and graphics. complex data marts and load data systems for reporting and analysis,According to one estimate, 0.8 into them. Instead, these systems without the need to deploy a fullzettabyte of data was produced allow users to connect to legacy data data mart. In-memory analyticsin 2009—if a gigabyte were the stores, populate an ad hoc database, removes the need to build complexsize of a sesame seed, a zettabyte conduct the analysis, and then performance layers such aswould equal the diameter of the discard the in-memory database multidimensional cubes within thesun—and that is expected to rise to once the analysis is complete. data warehouse; instead, users can Guidelines:35 zettabytes by 2020. At the same run their analytical applications 11.0 million =time, tighter regulations involving Speed of Deployment directly against an in-memorythe tracking of financial transactions Given the rapid growth of data performance layer. aölkdfölka = 32.8% = 30.1% =Exhibit 3Performance Comparison of Different Database Types TABLE HEADINGS RESPONSE TIME A4 format: - width for 3 columns - width for 2 columns Microseconds In-Memory Databases Letter format: - width for 3 columns - width for 2 columns Milliseconds Lines: 0,5 pt Lines for legend: 0,5 Disk-Based Databases Disk-Based, Memory-Cached Databases Note: Seconds Please always delete otherwise InDesign w 100 1,000 10,000 100,000 file. These colors can’t be Throughput (transactions per second) Approved Colors, TSource: Booz & Company analysis8 Booz & Company
  11. 11. GREATER In addition to the real gains in performance and speed offered by from the enterprise architecture, reducing complexity and the infra-INTELLIGENCE in-memory analytics, these new structure the traditional systems systems can significantly improve the required. Furthermore, the source quality of the business and customer data has to be created or populated intelligence they generate. And they only once and then is immediately can transform how that intelligence is available for any kind of analysis. delivered, and to whom. The benefits Consequently, organizations can include the following: operate at a higher level of perfor- mance, deliver more reports per • Improved decision making: The hour, and free up capacity on the ease of use of in-memory technol- source systems for other opera- ogy allows anyone in the organi- tional purposes. zation, from business analysts to managers, to build their own que- • Self-service business intelligence: ries and dashboards with very little In-memory analytics allows any technical expertise. Control over user to easily carve out subsets critical data shifts away from those of the enterprise business intelli- who manage it to the stakeholders gence environment for convenient who own and use it, allowing them departmental usage. Work groups to make better business decisions. can operate autonomously without affecting the central data warehouse • Richer insight: The significantly workload. In addition, in-memory greater processing speed and calcu- technology enables a much greater lation performance of in-memory degree of ad hoc analysis within the technology lets end-users develop organization and allows users to richer, more complex models, source data rapidly, build analytical enabling better customer segmenta- applications, and conduct specific tion and more powerful campaign investigations. Once the analysis planning in the CRM space, for is no longer required, it can be example. The result is significantly disposed of easily. Quick response greater business value for the times and strong visual interfaces system as a whole. also enable mobile BI applications, which can be used by salespeople to • Increased efficiency: Converting to gain a complete view of customers, in-memory technology as a plat- based on real-time customer sales form for analysis allows a whole data, while on the road. technological layer to be removed In-memory technology enables a much greater degree of ad hoc analysis within the organization.Booz & Company 9
  12. 12. STEPS FOR The virtues of in-memory analytics are many, but as with any new technol- to relying entirely on the IT depart- ment to perform its “data magic inTHE CIO ogy, it is the responsibility of the CIO the basement,” a process that can to make these virtues clear both to top take days. Moreover, users frequently management and to business users. avoid using traditional BI tools Most important, in-memory analyt- because of their inherent complexity ics must be seen as part of a broader and difficulty. With in-memory analyt- BI strategy that takes into account ics, as we have seen, response time can its overall business value and the be virtually instantaneous, and users underlying technology architecture, have the ability to design their own while remaining aware of the chal- queries. It is critical to make clear to lenges inherent in every major new the business that a significant portion technology. of the value of in-memory technol- ogy lies in its ability to open up these The top priority is to educate the busi- bottlenecks and offer users greater ness as to the value and advantages access to fresh data and increased of in-memory analytics, as well as to query flexibility. the costs and risks involved. In many organizations, business users of ana- As part of the education process, CIOs lytical tools have grown accustomed should identify and point out particu- A significant portion of the value of in-memory technology lies in its ability to open up bottlenecks and offer users greater access to fresh data.10 Booz & Company
  13. 13. larly valuable business opportunities old warehouses as they implement ture is critical. Don’t try to convert thethat in-memory technology offers. and test the new system, an additional entire architecture to in-memory tech-These might include the ability to go up-front cost that must be taken into nology all at once. Instead, develop abeyond traditional BI reports to create account. It will also be necessary to thorough investment road map thatpowerful applications such as what-if conduct a thorough cleansing of all includes both a plan for incorporatinganalyses, interactive filtering, and pat- customer data to avoid contaminating in-memory technology in the stan-tern discovery, all in an easy, intuitive the new system with bad information. dard architecture when possible—infashion. These capabilities should be order to prevent business units fromactively promoted in order to foster Once such systems are installed, most adopting it as part of a “rogue” ITa high-performance decision-making companies will struggle to calcu- effort—and a strategy for switchingculture. But to accomplish this, many late the tangible cost-of-ownership over to the in-memory technology onof the organizational processes with benefits, such as overall infrastructure a department-by-department basis.which both business users and the IT savings or lower administrative labordepartment are familiar will need to costs. In order to build a better busi- Building a governance strategy thatbe rethought—an effort that, like any ness case, CIOs must gain an in-depth can effectively manage the potentialmajor change, must be planned and understanding of the different types explosion in the number of analyti-executed carefully. of BI applications and user segments cal applications is essential. Such a involved, as well as the extent of the strategy should include an inven-Calculating the business case for any administrative maintenance effort tory of analytical applications thatBI effort, traditional or otherwise, required. In-memory analytics can clearly defines owners and use casesis difficult. The new technology will then be better integrated into the over- and that can serve as the basis for arequire a significant up-front invest- all BI tool strategy and positioned to wider rollout of in-memory analyt-ment in new storage hardware and either replace or complement current ics throughout the organization. Ansoftware, and in the training needed BI solutions. established “BI competence center”for both the business and IT staffs with the authority to drive standard-to make the best use of it. Moreover, The proper role of in-memory analyt- ization and exercise governance willcompanies will need to maintain their ics in a company’s overall BI architec- be invaluable.Booz & Company 11
  14. 14. HIGHLIGHTS • End-users consistently rank slow query performance as among the top three concerns that affect their perception of the value of business intelligence systems. • In-memory analytics offers a vast improvement in process speed, query quality, and customer insight over traditional operational/ analytical customer data warehouse systems. • The shift to in-memory Conclusion By combining operational and analytical databases into a single technology will be driven by instantly available warehouse, demand from the business in-memory analytics will give for real-time customer and business users access to a whole operational information. new realm of crucial customer information, transform how they • CIOs must develop a use that information, and thus give strong business case for them a real competitive advantage implementing in-memory in the race to gain better customer analytics, including the new insights more quickly. As with any business opportunities it will business intelligence effort, however, enable and its advantage the new technology’s virtues must in terms of total cost of be sold to business users, and its use ownership. must be monitored and managed carefully to ensure that all users are getting the most out of it.12 Booz & Company
  15. 15. Resources“The BI Survey 9,” Business Application Research Center,“Not Your Typical Marketing Campaign: The Next Wave ofTechnology-Driven Marketing,” Booz & Company, 2009.“Loyalty by Numbers: An Integrated Approach for TelecomCompanies,” Booz & Company, 2010. the AuthorsOlaf Acker is a partner in Adrian Blockus is an associateBooz & Company’s Frankfurt with Booz & Company basedand Dubai offices. He focuses in Berlin. He specializes in IT,on business technology technology, and strategy trans-strategy and transformation formation programs for telecomprograms for global companies and technology companies. Hein the telecommunications, is an expert in business intel-media, and high technology ligence and data Dr. Carsten Bange is theDr. Florian Gröne is a founder and managing directorsenior associate with of the Business ApplicationBooz & Company in Berlin. He Research Center (BARC), theworks with communications, leading market analyst for busi-media, and technology industry ness intelligence and data man-players on defining their go-to- agement in central Europe. Hemarket strategies and operating consults regularly on businessmodels and on transforming intelligence and data manage-customer-facing processes ment strategy, architecture, andand enabling technologies. He technology selection.leads the firm’s CRM Center ofExcellence in Europe.Booz & Company 13
  16. 16. The most recent Worldwide Officeslist of our officesand affiliates, with Asia Bangkok Helsinki Middle East Florham Parkaddresses and Beijing Brisbane Istanbul Abu Dhabi Houstontelephone numbers, Delhi Canberra London Beirut Los Angelescan be found on Hong Kong Jakarta Madrid Cairo Mexico Cityour website, Mumbai Kuala Lumpur Milan Doha New York Seoul Melbourne Moscow Dubai Parsippany Shanghai Sydney Munich Riyadh San Francisco Taipei Oslo Tokyo Europe Paris North America South America Amsterdam Rome Atlanta Buenos Aires Australia, Berlin Stockholm Chicago Rio de Janeiro New Zealand & Copenhagen Stuttgart Cleveland Santiago Southeast Asia Dublin Vienna Dallas São Paulo Adelaide Düsseldorf Warsaw DC Auckland Frankfurt Zurich DetroitBooz & Company is a leading global managementconsulting firm, helping the world’s top businesses,governments, and organizations. Our founder,Edwin Booz, defined the profession when he estab-lished the first management consulting firm in 1914.Today, with more than 3,300 people in 61 officesaround the world, we bring foresight and knowledge,deep functional expertise, and a practical approachto building capabilities and delivering real impact.We work closely with our clients to create and deliveressential advantage. The independent White Spacereport ranked Booz & Company #1 among consultingfirms for “the best thought leadership” in 2010.For our management magazine strategy+business, to learn more aboutBooz & Company.©2010 Booz & Company Inc.