Predictive analytics: the next big thing in BI?


Published on

Published in: Technology, Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Predictive analytics: the next big thing in BI?

  1. 1. E-BookPredictive analytics: the next bigthing in BI?Predictive analytics goes beyond traditional business intelligence,enabling users to churn through large volumes of both historical andreal-time data in an effort to build predictive models. In this eBook,learn about predictive analytics technology and why it’s gettingincreased attention from prospective users. Read about real-worldpredictive analytics projects and get expert advice on organizing apredictive analytics program and on developing and utilizing predictivemodels in business operations. Get examples of predictive analytics inaction as well as insight on the potential benefits and challenges ofusing predictive analytics software. Sponsored By:
  2. 2. E-Book Predictive analytics: the next big thing in BI? E-Book Predictive analytics: the next big thing in BI? Table of Contents Predictive analytics early adopters focus on individual customer analysis Data mining, predictive analytics: trends, benefits and challenges To be effective, predictive analytics software must be tied to action Using predictive analytics tools and setting up an analytics program Resources from IBMSponsored By: Page 2 of 17
  3. 3. E-Book Predictive analytics: the next big thing in BI?Predictive analytics early adopters focus onindividual customer analysisBy Jeff Kelly, News EditorTarget isn’t just a name for the Minneapolis-based retail chain. The company applies thatterm to business operations on a daily basis, using predictive analytics technology to targetits marketing programs to individual “guests,” as Target calls its customers.“We are able to derive guest expectations through mining our data,” said Andrew Pole, headof media and database marketing at Target.Target isn’t the only company that uses predictive analytics to zero in on customer behaviorand expectations on a micro level. In fact, rather than identifying and predicting largermarket or economic trends, most early adopters of the technology are using predictiveanalytics software to tailor marketing campaigns and identify up-sell opportunities down tothe individual customer level, according to speakers at the 2010 Predictive Analytics Worldconference in Alexandria, Va. The goal, they said, is to better understand what specificcustomers are likely to spend their money on.Take Paychex Inc. The Rochester, N.Y.-based company’s core business is processingpayrolls for its corporate clients. Paychex also offers 401(k) services, a business it is eagerto expand. Until recently, however, Paychex sales reps were cold calling payroll clients tosee if they might be interested in adding the 401(k) services, according to Jason Fox, aninformation system and portfolio manager in Paychex’s enterprise risk managementdivision.The cold calling proved to be an inefficient way to sign up new 401(k) customers: Nearlyhalf of Paychex’s clients use the company’s payroll services but not its 401(k) offerings.“That’s a lot of revenue to leave on the table,” Fox said.Sponsored By: Page 3 of 17
  4. 4. E-Book Predictive analytics: the next big thing in BI?Predictive analytics tools point the way to prospective customersThe company decided to invest in predictive analytics technology to help identify which of itspayroll-only clients were the most likely to be interested in the 401(k) business. Theanalytics routines take into account whether a client uses a competitor’s 401(k) services ornone at all, as well as its credit rating and payment history at Paychex.With the most likely 401(k) clients identified, Paychex can then allocate its availablemarketing budget to the various prospects based on their perceived value and likelihood ofsigning on, Fox said.At Monster Worldwide Inc., Jean Paul Isson and his team are using predictive analyticstechnology to help differentiate the New York-based company from other online careerssites.In addition to its flagship job posting services, Monster offers services such as resumemining and careers website hosting to corporate clients. Predictive analytics helps Monsteridentify which services to market to which clients, said Isson, who is vice president of thecompany’s global business intelligence and predictive analytics division.Isson added that the predictive analytics software has become a crucial tool for thecompany as it takes on new, and free, job listing sites in the careers services market. “It’sthe only way we can optimize ourselves,” he said.Using predictive analytics to keep the cash registers ringingAt Target, predictive analytics technology helps the retailer maximize the amount ofrevenue it gets from each customer, whether people shop online or in stores, while alsoenabling the company to allocate its marketing resources more efficiently, Pole said.With data mined from online transactions, loyalty card use and demographics databases, forexample, Target creates a profile of each customer and determines the amount of moneythat he or she is likely to spend with the company in a given year.Sponsored By: Page 4 of 17
  5. 5. E-Book Predictive analytics: the next big thing in BI?So if, with the help of the predictive analytics software, Target determines that customer Xcan afford to spend $5,000 annually, the company tailors its marketing efforts accordingly.And when the customer reaches the $5,000 mark, Target can stop spending moneymarketing to him if it decides that any additional efforts aren’t likely to induce him to makemore purchases, Pole said.Target also uses predictive analytics to determine how much of a marketing investment isrequired to get a particular customer to buy a certain product. For some customers, a $1coupon might be enough to get them to buy dishwasher soap, while others might need onlyhalf of that to induce a sale. With that kind of information in hand, Pole said, Target’smarketing department can decide which customers are worth marketing to in givensituations.Sponsored By: Page 5 of 17
  6. 6. E-Book Predictive analytics: the next big thing in BI?Data mining, predictive analytics: trends, benefitsand challengesBy Craig Stedman, Site EditorPredictive analytics software is getting increasing amounts of attention from technologyusers, vendors and analysts. The advanced analytics technology is designed to enableorganizations to mine data and build predictive models that can help them analyze futurebusiness scenarios, such as customer buying behavior or the financial risks of proposedcorporate investments.Until now, data mining, predictive analytics and advanced business modeling technology hasbeen used almost exclusively by highly skilled – and highly paid – statisticians,mathematicians and quantitative analysts. But that’s changing as business intelligence (BI)and analytics vendors offer more user-friendly predictive analytics tools – or is it? In thisinterview, conducted via email, Forrester Research Inc. analyst James Kobielus assesses thecurrent state of predictive analytics software and provides an overview of predictiveanalytics trends and the potential benefits and challenges of using the technology.There’s a lot of talk about predictive analytics being the next big battleground inthe business intelligence market. Do you agree? And if so, why is that? Yes, I agree.The core BI market has become quite crowded with vendors providing solutions that do agreat job of supporting rich analysis of historical data. It would be a gross oversimplificationto claim that the traditional BI market has become commoditized. However, vendors all overthe BI arena are looking to new types of advanced analytics applications as a way ofavoiding the “me too” syndrome of look-alike offerings that blur into each other and fail todifferentiate in a way that can justify a premium price.Predictive analytics is a natural evolution path for BI offerings, and it’s something that manyusers want but have often needed to obtain separate from their current BI tools. Predictiveanalytics can play a pivotal role in day-to-day business operations. If they’re available toinformation workers – not just to Ph.D. statisticians and professional data miners –predictive modeling tools can help business people continually tweak their plans based onSponsored By: Page 6 of 17
  7. 7. E-Book Predictive analytics: the next big thing in BI?what-if analyses and forecasts that leverage both deep historical data and fresh streams ofcurrent-event data.From a general standpoint, is predictive analytics software ready for broader use?Or are there limitations that need to be addressed first? Yes and no. Yes, Forrester isseeing an impressive new generation of user-friendly predictive analytics tools that aregeared to the needs of the mass market of information workers and other nontraditionalusers.But no, traditional predictive analytics tools are still very much the province of a specializedcadre of statistically and mathematically savvy modelers with an academic background inmultivariate statistical analysis and data mining – although most of the establishedpredictive modeling vendors have made great progress in rolling out more user-friendlyvisual tooling. Still, I had to reflect the current state of the industry when I published myForrester Wave report on predictive analytics and data mining tools in early 2010. I didn’tput a huge emphasis on features geared to business analysts, subject matter experts andother “nontechnical” information workers. The core problem with today’s offerings is thatmany of them remain power tools with a steep learning curve and a commensurately highprice.What’s happening with predictive analytics software? Can you give us an overviewof the key technology trends that you’re tracking? The key trend is the move towarduser-friendly, self-service, BI-integrated predictive analytics tools that encourage morepervasive adoption. Another trend is the move toward integrating more predictive analyticsfunctionality into the enterprise data warehouse, through in-database analytics. That’s anapproach under which data preparation, statistical analysis, model scoring and otheradvanced analytics functions can be parallelized and thereby accelerated across one or moredata warehouse nodes. In-database analytics also enables flexible deployment of a widerange of resource-intensive functions – such as data mining and predictive modeling – to acluster, grid or cloud of high-performance analytic databases.We’re also seeing the growing adoption of open frameworks for building predictive analyticsmodels for data mining, text mining and other applications. The principal ones areMapReduce and Hadoop, which have been adopted by a wide range of vendors of analyticsSponsored By: Page 7 of 17
  8. 8. E-Book Predictive analytics: the next big thing in BI?tools and data warehouse platforms. In the coming year, we’ll also see the beginning of anindustry push toward an open development framework for inline predictive models that canbe deployed to complex event processing (CEP) environments for real-time data streamingapplications. Still another trend is the embedding of predictive analytics features incustomer relationship management (CRM) applications to drive real-time “next best offer”recommendations in call centers and multichannel customer service environments.Why should prospective users be interested in predictive analytics? What are thepotential benefits or competitive advantages that companies can get from it?Business is all about placing bets and knowing if the odds are in your favor. Businesssuccess depends on your company being able to predict future scenarios well enough toprepare plans and deploy resources so that you can seize opportunities, neutralize threatsand mitigate risks. Clearly, predictive analytics can play a pivotal role in day-to-daybusiness operations. It can help you focus strategy and continually tweak plans based onactual performance and likely scenarios. And, as I noted in a recent Forrester blog post, thetechnology can sit at the core of your service-oriented architecture strategy as you embedpredictive logic deeply into data warehouses, business process management platforms, CEPstreams and operational applications.The grand promise of predictive analytics – still largely unrealized in most companies – isthat it will become ubiquitous, guiding all decisions, transactions and applications. For thetechnology to rise to that challenge, organizations must move toward a comprehensiveadvanced analytics strategy that integrates data mining, content analytics and in-databaseanalytics. We’ve sketched out a vision of “service-oriented analytics,” under which youbreak down silos among data mining and content analytics initiatives and leverage thesepooled resources across all business processes.You may agree that this is the right vision but have doubts about whether there is apractical, incremental roadmap for taking your company in that direction. In fact there is,and it starts with reassessing the core of most companies’ predictive analytics capability:your data mining tools. As you plan your predictive analytics initiatives, you should avoidthe traditional approach of focusing on tactical, bottom-up, project-specific requirements.You should also try not to shoehorn your requirements into the limited feature set ofwhatever modeling tool you currently happen to use.Sponsored By: Page 8 of 17
  9. 9. E-Book Predictive analytics: the next big thing in BI?On the flip side, what kind of challenges or issues should people consider and beprepared for when they’re weighing a possible deployment of predictive analyticssoftware? The learning curve, complexity and cost of predictive analytics tools are theprincipal challenges. Also, if you’re committed to deploying sophisticated predictiveanalytics, you’ll need to hire specialized, expensive talent to handle data preparation andcleansing, build and score predictive models, and integrate the models and their results intoyour BI, CRM and other application environments. And if you decide to integrate yourpredictive analytics initiatives with your data warehouse through in-database analytics,you’ll need to bring the groups who handle those functions together and get them speakinga common language.Sponsored By: Page 9 of 17
  10. 10. E-Book Predictive analytics: the next big thing in BI?To be effective, predictive analytics software mustbe tied to actionBy Jeff Kelly, News EditorLike many advanced and emerging technologies, predictive analytics software has a certaindegree of coolness associated with it. When corporate and business executives see that thetechnology can accurately predict which customers are likely to buy what products, they getexcited.But what good is a prediction if companies don’t do anything with the insight? Not much,according to Dr. Eric Siegel, president of consulting firm Prediction Impact Inc. andchairman of the 2010 Predictive Analytics World conference.As predictive analytics starts to gain more traction and deployments increase, thetechnology must be used to tie insight to action to be truly effective, Siegel said. Companiesmust devise business rules that trigger specific actions when predictions are made, headded.Insurance companies, an early adopter of predictive analytics technology, are a goodexample of this, Siegel noted. Insurers use predictive analytics software to determine theriskiness of taking on a particular customer, he said. The potential risk is then tied directlyto the price of the insurance policy being offered to that customer.At a retail organization, connecting predictive analytics to action could mean triggeringmarketing campaigns based on a customer’s likeliness to purchase a certain item or service,Siegel said. At financial services companies, the technology could be used to identifypotential fraud and then prompt an audit.Whatever the industry, predictive analytics software used in isolation doesn’t do anybodymuch good. But that’s not all that companies considering predictive analytics projects needto keep in mind, according to other speakers at the conference, which was held inAlexandria, Va.Sponsored By: Page 10 of 17
  11. 11. E-Book Predictive analytics: the next big thing in BI?Predictive analytics demands significant data prep work, user buy-inThere is significant prep work that must go into a successful predictive analytics initiative,said Paul Coleman, director of marketing statistics at retail giant Macy’s Inc. He estimatesthat getting data prepped before even applying predictive analytics technology to it is about80% of the job.“Building [predictive data] models is at least as complex as your business,” Coleman toldattendees. And, he cautioned, “the models are only as good as the data” that goes intothem.Jean Paul Isson agreed. Isson, vice president of global business intelligence and predictiveanalytics at Monster Worldwide Inc., said data governance and data quality are key tosuccessful predictive analytics projects.At Monster, for example, company executives first had to decide on the definition of“customer,” Isson said. Initially, they came up with seven possible definitions. Not until theyagreed on a single one could the provider of online job listings and career managementservices move forward with predictive analytics, he added.Isson also said that internal change management is important when deploying predictiveanalytics technology. He noted that most predictive analytics initiatives fail not because offaulty predictive data models but from a lack of executive buy-in and poor end-usertraining.Marketing executives who are hitting their numbers will likely be reluctant to adopt a newtechnology such as predictive analytics, Isson said. As a result, he advised, it’s important toshow them how the technology can improve their success rates and then train them on howbest to use the associated tools.Jason Fox, an information system and portfolio manager in Paychex Inc.’s enterprise riskmanagement division, told conference attendees that finding and enlisting subject matterexperts from business operations was crucial to the company’s predictive analyticsinitiatives.Sponsored By: Page 11 of 17
  12. 12. E-Book Predictive analytics: the next big thing in BI?“We identified subject matter experts to ensure that business conditions were met,” Foxsaid. He also sought out champions of the technology in Paychex’s sales department –people who could tout the benefits of the predictive analytics software to their colleaguesand help boost end-user adoption.Technical obstacles to predictive analytics successThere are also technical factors to consider, Coleman said. Data contained in flat files, forexample, is relatively simple to model for predictive analytics but then difficult to change,he warned. Data in relational databases, on the other hand, is more flexible to work withbut can be limited by data volume constraints, according to Coleman.Companies should consider the type of data that they plan to exploit and how it’s storedbefore starting a predictive analytics initiative, he recommended. Those factors might alsoplay a role in determining the type of workers that a company hires to oversee itsdeployment and use of predictive analytics software.In the end, however, all of the required efforts are worth it because of the business insightsthat can be gained through the use of predictive analytics tools, the conference speakersagreed.“Inside this data, there’s a customer in there someplace,” Cole said.Sponsored By: Page 12 of 17
  13. 13. E-Book Predictive analytics: the next big thing in BI?Using predictive analytics tools and setting up ananalytics programBy Rick Sherman, SearchBusiness ContributorBusiness intelligence (BI) software has become widely used – even to the point of beingpervasive in many organizations. But for the most part, predictive analytics tools are stillused by only the most sophisticated data-driven enterprises.In addition, whereas IT groups typically develop BI dashboards and reports for businessusers, predictive analytics models usually are created by a handful of highly skilled endusers. It can be an eye-opening experience for IT workers to realize that the people whobuild predictive models are more data-savvy and technically oriented than they are. In fact,predictive model builders often view the IT staff merely as data gatherers whose purpose isto feed their data-hungry models.The industries that pioneered the use of predictive analytics software are insurance,financial services and retail. Companies in those industries share the need to understandwho their customers and prospects are, how to up-sell and cross-sell products and services,and how to predict customer behavior (including bad behavior through processes such asfraud detection.) Predictive analytics tools can help in all of those areas. Other industriesthat have benefited from the technology include telecommunications, travel, healthcare andpharmaceuticals.Across industries, there are common approaches that can be taken in building the requiredpredictive models, selecting technology and staffing up for successful predictive analyticsprojects.Building predictive models is a combination of science and art. It’s an iterative process inwhich a model is created from an initial hypothesis and then refined until it produces avaluable business outcome – or discarded in favor of another model with more potential.Developing and then using predictive models involves the following tasks:Sponsored By: Page 13 of 17
  14. 14. E-Book Predictive analytics: the next big thing in BI? 1. Scope and define the predictive analytics project. What business processes will be analyzed as part of the initiative, and what are the desired business outcomes? 2. Explore and profile your data. Because predictive analytics is a data-intensive application, considerable effort is required to determine the data that’s needed for the project, where it’s stored and whether it’s readily accessible, and its current state. 3. Gather, cleanse and integrate the data. Once the necessary data is located and evaluated, work often needs to be done to turn it into a clean, consistent and comprehensive set of information that is ready to be analyzed. That process may be minimized if an enterprise data warehouse is leveraged as the primary data source. But external and unstructured data is often used to augment warehoused information, which can add to the data integration and cleansing work. 4. Build the predictive models. The model builders take over here, testing models and their underlying hypotheses through steps such as including and ruling out different variables and factors; back-testing the models against historical data; and determining the potential business value of the analytical results produced by the models. 5. Incorporate analytics into business processes. Predictive analytics tools and models are of no business value unless they’re incorporated into business processes so that they can be used to help manage (and hopefully grow) business operations. 6. Monitor the models and measure their business results. Predictive models need to adapt to changing business conditions and data. And the results they’re producing need to be tracked so that you know which models are providing the most value to your organization. 7. Manage the models. Prune the models with little business value, improve the ones that may not yet be delivering on their expected outcome but still have potential, and tune the ones that are producing valuable results to further improve them.With a typical BI project, business users define their report requirements to the IT or BIgroup, which then identifies the required data, creates the reports and hands them off tothe users. Similarly, in predictive analytics deployments, a joint business-IT team mustscope and define the project, after which IT assesses, cleanses and integrates the requireddata. At this point, though, predictive analytics projects deviate from conventional BISponsored By: Page 14 of 17
  15. 15. E-Book Predictive analytics: the next big thing in BI?projects because it is the users – for example, statisticians, mathematicians andquantitative analysts – who take over the process of building the predictive models.The IT or BI group re-enters the picture after the models have been developed and startbeing used by business and data analysts. For example, IT or BI teams might incorporatethe predictive analytics results into dashboards or reports for more pervasive BI use withintheir organizations. They might also take over the physical management of predictivemodels and their associated technology infrastructure.To run predictive models, companies require statistical analysis, data mining or datavisualization tools. Typically, predictive analytics software and other types of advanced dataanalytics tools are used by experienced analytics practitioners who are well versed instatistical techniques such as multivariate linear regression and survival analysis.Most BI vendors sell integrated product suites that include query tools, dashboards andreporting software. But if they offer predictive analytics software, it tends to be sold as aseparate and distinct product. While that’s starting to change, the predictive analytics toolsnow being used primarily come from vendors that specialize in statistical analysis, datamining or other advanced analytics.Predictive analytics tools turn the BI software selection process onits headCompared with a typical BI software evaluation, where the IT or BI group drives thesoftware selection process while soliciting input and feedback from business users, anevaluation of predictive analytics tools is turned upside down – or at least it should be.Ideally, the statisticians and other users who build the predictive models take the lead inevaluating the predictive analysis tools that are being considered, with IT providing input onthe software’s potential impact on the organization’s technology infrastructure. In this case,the users are likely to be the only ones who understand the statistical or data miningtechniques they need and whether the various tools can support those requirements.Sponsored By: Page 15 of 17
  16. 16. E-Book Predictive analytics: the next big thing in BI?Predictive model builders and users must have a strong knowledge of data, statistics, anorganization’s business operations and the industry in which it competes. Companies, evenvery large ones, often have only a small number of people with such skills. As a result,predictive modelers and analysts are likely to be viewed as the star players on a dataanalytics team.The typical organizational structure places predictive analytics experts in individual businessunits or departments. The analysts work with business executives to determine the businessrequirements for specific predictive models and then go to the IT or BI group to get accessto the required data. In this kind of structure, IT and BI workers are enablers: Their primarytasks are to gather, cleanse and integrate the data that the predictive analytics gurus needto run their models.In conclusion, the critical success factors for successful deployments of predictive analyticstools include having the right expertise (i.e., predictive modelers with a statisticalpedigree); delivering a comprehensive and consistent set of data for predictive analyticsuses; and properly incorporating the predictive models into business processes so that theycan be used help to improve business results.About the author: Rick Sherman is the founder of Athena IT Solutions, a Stow, Mass.-based firm that provides data warehouse and business intelligence consulting, training andvendor services. In addition to having more than 20 years of experience in the IT business,Sherman is a published author of more than 50 articles, a frequent industry speaker, anInformation Management Innovative Solution Awards judge and an expert contributor toboth and He blogs at The DataDoghouse and can be reached at By: Page 16 of 17
  17. 17. E-Book Predictive analytics: the next big thing in BI?Resources from IBMEvaluate: IBM Cognos 8 Business IntelligenceAbout IBMAt IBM, we strive to lead in the creation, development and manufacture of the industrysmost advanced information technologies, including computer systems, software, networkingsystems, storage devices and microelectronics. We translate these advanced technologiesinto value for our customers through our professional solutions and services businessesworldwide. By: Page 17 of 17