The new normal in business intelligence


Published on

The new normal in business intelligence is about the transformational changes that take place in the digital world and definitely change the nature of BI. Business models in the global marketplace are reshaped through the application of information technology. The Internet is the societal operating system of the 21st century and its underlying infrastructure - the clud computing model - represents a disruptive change. A networked infrastructure, big data from disparate sources and social media among other trends as the self-service model and collaboration are changing the way BI systems are deployed and used.

Published in: Business, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The new normal in business intelligence

  1. 1. The New Normalin Business Intelligence Studie en Advies Johan Blomme Data Consulting Services
  2. 2. The new normal in business intelligence is about the transformationalchanges that take place in the digital world and definitely change the natureof business intelligence. Business models in the global marketplace arereshaped through the application of information technology.The Internet is the societal operating system of the 21st century and itsunderlying infrastructure – the cloud computing model – represents a« disruptive » change. A networked infrastructure, big data from disparatesources and social media among other trends as the self-service model andcollaboration are changing the way BI systems are deployed and used. 2
  3. 3. The New Normal in BIIntroduction 3
  4. 4. • In today’s marketplace, change is a constant.• Products are increasingly commoditised, development cycles have shortened and expectations of consumers are rising. To achieve a sustainable competitive position, companies must react in an agile way to changing market conditions.• The current business environment evolves from a transition towards globalization and a restructuration of the economic order. The pace of technological changes that allow instant connectivity and the current era of ubiquitous computing that resulted from it, represent « the new normal in business intelligence». 4
  5. 5. • As an industry, business intelligence has to adapt to environmental changes.• The evolution of the Internet as a new societal operating system, reshapes the future of business intelligence.• The Internet evolves as a platform for the use of interoperable resources (storage, computing, applications and services) and drives the development of information intensive services in the 21st century. Increasingly, the cloud becomes the vehicle for the Internet of Services.• The business ecosystem generates a huge amount of data in terms of volume, variety and velocity, and requires businesses to take on a data-driven approach to differentiate. It’s about gaining actionable insights faster than the competition by reducing the data-to-decision gap.• This highlights the integration of structured and unstructured data (esp. social media content) to derive actionable insights from « big data » and the leverage of predictive analytics for agile decision-making. 5
  6. 6. • The exponential growth of data and the increased reliance on insights derived from data for decision-making, causes a shift in the focus of business intelligence. BI is more than an IT- function and is about people and business decisions.• Therefore, the emphasis of next-generation BI should be on designing solutions that focus on answering business questions of the end user. In the field of BI the finished product is not a dashboard displaying metrics but actionable intelligence answering the business question at hand. Users want seamless access to information to support decision-making in their day-to- day activities.• The future direction of BI will thereby be shaped by the new age of computing. In both their personal and professional lives, Web-savvy users have adopted the principles of interactive computing and have come to demand customizable BI-tools with high responsiveness. Business intelligence, and the insights it delivers, evolves towards an enterprise service that follows the lines of a self-service model with business users producing their own reports in an interactive way and performing analytics on demand. 6
  7. 7. • Furthermore, Web 2.0 and social networks function as catalysts for highly intuitive user interfaces and the collaborative features of computing allow users to share insights, which transforms BI from a solitary to a collaborative activity.• Companies are exploring the connection between analytical activity and knowledge sharing. Combined with collaborative technologies that « crowdsource » intelligence from various partners of the extended enterprise, this approach provides the context for better and faster decision-making. 7
  8. 8. The factors that constitute the new normal in BI can be summarised as follows : The Future Internet Predictive Analytics Big Data The Social Media Analytics New Normal Cloud Computing in BI Collaborative BI Embedded BI User Empowerment / Self-Service BI 8
  9. 9. The New Normal in BI1. The Future Internet 9
  10. 10. • The main objective of enterprise computing is to be adaptive to change.• The new generation of enterprise computing must enable pervasive BI deployments : – spreading BI to more users and more devices : • consumerization of IT : enterprise computing aligns with consumer-class technologies ; • BI-tools are more and more organized around the user’s experience to interactively discover hidden relationships, trends and patterns and to create new information and relate it with external data sources ; – using multiple data sources : the use of structured as well as semi- and unstructured data sources (e.g. social media content) extends the playing field of BI. 10
  11. 11. • The new generation of enterprise computing needs to be developed within the perspective of the future Internet : – the Internet as data source : • BI applications no longer limit their analysis to data inside the company and increasingly source their data from the Internet to provide richer insights into the dynamics of today’s business ; – the Internet as software platform : • BI applications are moving from company-internal systems to service-based platforms on the Internet. 11
  12. 12. Web-based technologies enable BI-applications are deliveredthe implementation of user-configurable as a service on the Web or BI applications connecting to a wide hosted in the cloud arrangement of data INTERNET-ENABLED NE ING XT IT-INFRASTRUCTURE UT -G MP E CO NE RA ISE TIO R PR TE N EN 12
  13. 13. • The Internet of the future gives rise to a new Business business model that allows enterprises to form business networks : Networks – in the knowledge economy economic activity is based on highly networked interactions ; The Future – the amount of digital collaboration is increasing Internet among people, things and their interactionsInt rvic (through the Internet of People and the Internet Se ern es of Things, networking is expanding not only in ta et Da person-to-person interactions, but also in of person-to-machine and machine-to-machine g Bi interactions). 13
  14. 14. Globalization T he ng ndi Con sum xpa r e E as m fo es of IT rization eb e g e W osyst chan Th Ec x s sE ine Bus Device-Indepen Information AcceDemographic Shifts Drivers of Workforce NETWORKED INFRASTRUCTURE dent ss Hyp e Soc r Adop tive ora ial N tion lab logies Tec etworki of Col hno hno logy ng Tec Bandwidth Cloud Computing & Connectivity 14
  15. 15. • Business networks take on a data-driven approach to differentiate and apply fact- Business based decision-making enabled by advanced Networks analytics: – economic interactions are based on the principle of scarcity and in the knowledge economy the concept of scarcity applies to information ; The Future – information in itself does not create competitive Internet advantage (access to lots of information hasInt rvic already become ubiquitous) ; competitive Se ern es advantage is defined as access to information, ta the decisions based on that information and the et Da actions taken on these decisions ; of g Bi – business networks manage data in real-time, support anywhere, anytime and any device connectivity and provide the appropriate information to users across and beyond the enterprise (business users, partners, suppliers, customers). 15
  16. 16. • The Internet serves as a platform for a Business service-oriented approach that changes the Networks way of enterprise computing. With BI- applications moving to the web, the Internet emerges as a global SOA that is referred to as an Internet of Services. The IoS serves as the The basis for business networks. Future Internet • The new BI requires technologies that integrateInt rvic multiple data sources, address business needs Se ern es in a dynamic way and have a short time to ta et Da deployment. of g Bi • Contrary to large scale application development of traditional BI, the new BI moves towards smaller and flexible applications that can adopt quickly and are supported by a service-oriented architecture. 16
  17. 17. • SOA is an architecture whereby business applications use a set of loosely coupled and reusable services that can be accessed on a network.• Often implemented by Web services, a SOA is a building block for flexible access to multiple data sources and the very nature of services that can be reused and integrated with each other allows business processes to be adopted in an agile way to adjust to changing market conditions and to meet customer demands.• With cloud computing, this service model is delivered on demand. The delivery model is no longer installed software but services. 17
  18. 18. Internet of Services and BIUser empowerment / Self-service Cloud computing Embedded BI Cloud computing emerges as a new Users expect to have access to deployment model of BI by thebusiness information in the same way BI moves into the context of business adoption of a service-oriented as they use the Internet and search processes and transforms from a architecture and drives a the Web. Self-service BI is the reactive to a proactive decision- transformation in application implementation of this service- making tool by monitoring architectures through using “the Web orientation at the end-user level. performance and the prediction of as a platform” for interoperable future events. This change in the use applications and services. and delivery of software is guided by the adoption of a service-oriented approach. 18
  19. 19. The New Normal in BI2. Big Data 19
  20. 20. 20
  22. 22. Major sources of « big data » 22
  23. 23. The evolution of the Internet and the proliferation of dataData 3V The Cloud The Web The Internet Semantic Web Social Web Desktop/PC era Static Web Internet of People Internet of People and Things producer generated content user generated content. system generated content time 23
  24. 24. • As connectivity reaches more and more devices, the volume, variety and velocity of data from clickstreams, social networks and the Internet of Things (through which the physical world itself becomes an information system) creates a new economy of data.• Traditionally, BI applications allow users to acquire knowledge from company-internal data through various technologies (data warehousing, OLAP, data mining). However, the typical pattern of cleaning and normalizing proprietary information through an ETL process into a data warehouse is challenged by the transition to big data that is marked by greater accessibility, interoperability and 3rd party leverage of online data.• For businesses to become responsive to market conditions, it is necessary to look at the whole ecosystem by connecting internal business data with external information systems. BI- applications must access data from disparate sources inside and outside the firewall, consider qualitative and quantitative data and include structured as well as semi-structured and unstructured data. 24
  25. 25. • Data from the Web is feeding BI applications : – BI applications no longer limit their analysis to data inside the company, but also source data from the outside, especially data from the Web. The Web is a data repository. – An important challenge is the extraction, integration and analysis from hererogeneous data sources.• BI applications move to the Web : – BI applications are increasingly accessible over the Web : BI is consumed as a service from the cloud. – The challenge here is the development of Web-based applications that access and analyze both historical enterprise data and real-time data, especially from the world wide market and making the information available on a variety of devices. 25
  26. 26. The increasing volume and complexity of data The 3 V’s represent the common has forced organizations to look at new data dimensions of big data, but the real management and analytic tools to optimize challenge lies in extracting actionable performance, improve service delivery and insights from it. discover new opportunities. Variety Database TechnologyVelocity Analytics Volume Services 26
  27. 27. • Heterogenous datasets are no longer manageable by a traditional relational database approach.• Requirements for next-generation BI-tools include : – connect directly to the underlying data sources to capture distributed data ; – schema-free : relationships between data are discovered dynamically ; – anytime, anywhere access with multiple devices ; – real-time visibility of what is happening now is needed and analytics must be used in the stream of business operations. 27
  28. 28. • New approaches such as in-database analytics, massive parallel processing, columnar databases and « No SQL » will increasingly be used for the analysis of structured as well as unstructured data. 28
  29. 29. • Traditional RDBMS and SQL-based access languages are unfit to the new world of unstructured information types.• NoSQL (« Not only SQL ») is a database management system that is more versatile than traditional database systems. – Map Reduce and Hadoop, for example, are currently the most widely known NoSQL approaches. – Data is stored without a pre-defined schema and big data sets are analyzed in parallel by assigning them to different servers. – Results are then collected and aggregated and can be further used in conjunction with relational database systems. 29
  30. 30. • BI has evolved from historical reporting to the pervasive analysis of (real-time) data from multiple data sources. Transactional data is analyzed in combination with new data types from social, machine to machine and mobile sources (e.g. sentiment, RFID, geolocation data). 30
  31. 31. • Organizations that embrace a « socialization of data »-approach by incorporating and converging disparate data sources into their BI-platforms, acquire a holistic view that provides them with the opportunity to derive actionable insights, e.g. – analytics of real-time customer sentiment and behaviour yield indicators of product or service issues ; – geospacial information of customers can be combined with transactional data to make targeted product or service offerings ; – combining internally generated data with publicly available information can reveal previously unknown correlations.• In its focus on the user experience, BI embraces Web 2.0-technology that focusses on intuitive user interfaces. Organizations must master visualization tools that let business users interactively manipulate data to find tailored insights that can be shared with other stakeholders (customers, partners, suppliers). 31
  32. 32. The New Normal in BI3. Cloud Computing 32
  33. 33. # apps / # users ING PUT COM UD CLO GE O MA OTC  virtualized connected ET/D RN environment INTE  Internet-based data VER access & exchange SER  eCommerce NT- CLIE  « as a service »-  service-oriented paradigm architecture  networking PC  Web 2  office automation  data warehousing INI E/M INFRA M  desktop computing MA  centralized automation 1970s 1980s 1990s 2000s 2010 & beyond 33
  34. 34. • As the competitiveness of businesses increasingly depends on adapting to changing market conditions, companies outsource tasks and processes to external providers.• This trend can be linked to the creation of business ecosystems in The Future Internet with vendors offering their services.• Software-as-a-Service (Saas), for example, is a type of cloud offering for software delivery. Applications are hosted by a provider and made available on demand.• Cloud computing is the backbone for the Internet of Services and provides resources for on demand, networked access to services. Infrastructure as a service Platform as a service Software as a service Data as a service ERP Analytics as a service 34
  35. 35. “Cloud computing is enabling the consumption of IT as a service. Couple this with the “bigdata” phenomenon, and organizations increasingly will be motivated to consume IT as anexternal service versus internal infrastructure investments”.The 2011 Digital Universe Study : Extracting Value from Chaos, IDC, June 2011 35
  36. 36. • Cloud computing alters the way computing, storage and networking resources are allocated. Through virtualization, the traditional server-centric architecture model in which applications are tied to the underlying hardware is altered to a service-centered cloud architecture. Applications are decoupled from the physical resource which implies that services (computing resources, e.g. processing power, memory, storage, network bandwidth) in a cloud computing environment are dynamically allocated to on demand requests.• In addition to a better utlization of IT resources, hardware cost reduction and greener computing, cloud computing provides an agile infrastructure to respond to business needs in a flexible way. 36
  37. 37. The commoditization of analytics The trend towards the hosting of services, leads to the commoditization of analytics. As a result, the creation of a competitive advantage depends on 2 factors.The management of large Analytics in itself don’tdata volumes (data integration, guarantee a competitivedata quality). As data fuels advantage. The insights,analytic processes, big data communications and decisionsbecomes increasingly important.. that follow analysis become more important. This stresses the role of self-service and collaboration. 37
  38. 38. In the pre-cloud world, the implementation of data warehouses needed serious upfront costs and designing database schemas was time consuming. Moreover, database schemas have their limitations because some data types (e.g. unstructured) don’t fit the schema. Combined with the need to manage big data volumes new database technologies (e.g. NoSQL) are used. For example, in the case of a Hadoop cluster that runs in parallel on smaller data sets, multiple servers are needed. Making use of cloud computing services in a pay-for-use formula is appealing. Furthermore, a service-oriented cloud architecture is ideally suited to integrateCloud computing data from various sources (e.g. « mash up » enterprise data with and big data public data). 38
  39. 39. Cloud computing gives a new meaning to the consumerization of IT. The convergence of cloud computing and connectivity is changing the way technology is delivered and information is consumed. Cloud applications are available on demand and developed to meet the immediate needs of users. Cloud computing is an important catalyst for self-service BI. Users do not need to be concerned with the technical details of software and hardware when using services. User-friendly interfaces and visualization capabilities make the generation, sharing and acting on information in real-time easier. This permits faster and better decision-making as well as greater collaboration internally and Cloud computing outside the firewall.and self-service BI 39
  40. 40. The New Normal in BI4. Embedded BI 40
  41. 41. As the market changes faster and faster, BI has to adopt to support decisions in day-to-day operations. The role of BI has changed beyond its original purpose of supporting ad hoc queries and analysis of historical information. With changing market dynamics there is aThe Need for Agile BI growing need to monitor performance using the latest data available and to predict future events. The new BI delivers information to users within the context of operational activities. Rather than reporting on the business, BI moves into the context of business processes. Data is analyzed in the flow of transactions to produce real-time metrics, alerts, recommendations and predictions for action. BI transforms from a reactive to aProcess Orientation proactive decision-making tool. Operational BI is related to the subject of real-time processing. Through the Internet of people (e.g. social media) and the Internet of Things (e.g. RFID and other sensored data), information becomes available that helps enterprises to improve business EMBEDDED BI processes. 41
  42. 42. • The consumerization of IT and the need of business decisions to be made on relevant information are drivers for placing reporting and analytics in the hands of more decision-makers and to apply analytics in real-time to production data.• A broader user adoption of BI results from : – faster and easier executive access to information ; – self-service access to data sources ; – right-time data for users’ roles in operations ; – more frequently updated information for all users.• The business benefits are : – improved customer sales, service and support ; – more efficiency and coordination in operations and business processes ; – faster deployment of analytical applications and services ; – customer self-service benefits. 42
  43. 43. Next-generation business applications will be more people- and process-oriented and have the computing power to proactively generate information that supports operational decisions.PEOPLE PROCESS Next-generation applications areSelf-directed analytics give users theability to navigate through and not static but interactive,visualize business data, allowing allowing users to couple the rightthem to generate views and reports actions based on the insights thatrelevant to their job function. are delivered. For example : Business - analytics on browser-based BI applications allow the mobile Analytics workforce to take actions ; - in an inventory application, proactive decision-making is supported through real-time information about which items are running low in inventory. TECHNOLOGY New approaches such as in-memory processing, in-database analytics, CEP, etc. contribute to the broader adoption of BI. 43
  44. 44. BI delivery framework(adapted from Eckerson, 2011) 44
  45. 45. tofrom service-oriented architecturemonolithic applications 45
  46. 46. 1 changes in the nature of BI : from1 2 3 stand-alone applications to embedded applications 2 changes in the function of applications : from dedicated applications to composite applications 3 changes in the way data is accessed : from data as an isolated resource to data as a service Source : SINGH KHALSA, R.H., REASON, A., BIERE, M., MEYERS, C., GREGGO, A & DEVINE, M., 2009. 46
  47. 47. SOACompanies move away from large-scale monolithic application development and turn to service- oriented architectures that represent the technological foundation of the Internet of Services. Web Services SOA’s are based on the principle that applications can be created as a composition of loosely coupled and reusable services. Open standards and the implementation of SOA’s through Internet-based technologies as Web services represent a new way of computing. 47
  48. 48. The Internet of Services allows for the personalisation of services, tailored to the user’s needs. Example : mashups (combining data from different sources into an integrated application)Web services are an important tool fordata integration from multiple sourcesand provide access to real-timeinformation that can be fed into Open access makes BI-functionalityoperational applications. accessible across and beyond the enterprise. Web services are user-centric because information is provided in the context of day-to-day activities. 48
  49. 49. Mashups and customer serviceAn obvious implementation area for enterprise mashups applies to customer service.CRM implies multiple processes (customer contact, sales, billing, support). Very oftenthe delivery of a process like that of customer service relies on end-users accessingmultiple applications. A major drawback is that customer-facing personnel (e.g. callcenter agents, sales representatives) lack a unified customer view which causes a poorquality of the customer experience. On the other hand, applications require a highinvolvement of IT in the lifecycle of each application.Therefore, enterprise mashups can provide a solution by the integration of disparatedata sources into a composite application. End users can use and reuse applicationbuilding blocks as “mashable” components to construct user-centric solutions. This notonly reduces the cost and time to build and maintain applications, but also allowsbusiness users to create applications that are mapped with processes. Customer serviceprocesses are optimized because employees are able to service customers moreefficiently. 49
  50. 50. Mashups and social media analyticsSocial media is empowering customers to reveal their thoughts and preferences throughthe Internet. This also enables businesses to look for competitive advantage bymonitoring and managing the many conversations that take place in the social mediaworld. Social media content can be tagged to look for pieces of information that can befurther structured to provide aggregate customer data revealing customer service issues,consumer attitudes and brand-related topics. Furthermore, sentiment analysis thatextracts the semantics of user-generated content allows for the creation of mashups thatidentify trends in unstructured data.For example, dashboards can use sentiment measures as key performance indicators tomonitor product performance. Consumer sentiment can serve as an indicator of theperformance of a new product that is introduced in the market. Sentiment measures canreveal the importance of product features and key customer needs. Retailers canestimate demand for products based on expressed satisfaction of discontent withproducts. 50
  51. 51. • Another implementation area of mashups is data visualization that integrates location intelligence in a composite application.• Data streams within the enterprise can be joined with virtually any data source that can be accessed from the Web. Web-based visualizations spacially represent the inherent relationships between the underlying data.• An example is Visual Fusion, data visualization software of IDV Solutions ( that unites data sources in a web-based, visual context for better insight and understanding. Commercial applications include the monitoring of inventory through RFID systems, field service management, sales and marketing analysis, supply chain management, and more. 51
  52. 52. view all suppliers for several auto assembly plants, a manufacturer developed an applicationthat visualizes suppliers on a map. Supply lines show which suppliers support which plants andcan be color-coded based on key information such as deliveries in progress and KPI data. Viewscan be analyzed, sorted, filtered and collaborated upon to show how a selected supplier performscompared to others via KPI-based charts and graphs. 52
  53. 53. reach the long tail of the application spectrum user-drivencloud adoption real-time data view incorporate social & collaborative agility computing features 53
  54. 54. The New Normal in BI5. User-Empowerment / Self-Service 54
  55. 55. • A confluence of factors (including ubiquitous broadband, a growing technology-native workforce, the adoption of social networking tools tools, mobile apps) is driving a trend called the consumerization of IT.• Enterprise application development is driven by the need for interactive access to disparate data, self-service capabilities that offer a flexibility for personalization and end-user customization. BI shifts towards the self-service delivery model that accomodates knowledge workers to search, access and analyze data from a variety of sources and available on a range of devices.• Empowerment of users is an important trend in BI. Business users generate their own reports and analysis and are no longer dependent on IT to deliver them. The ownership of BI shifts from IT to the business.• By incorporating collaborative features, BI environments are getting social. These enhancements facilitate the creation of user-generated content that can be shared with stakeholders across and beyond corporate boundaries, enabling the networked enterprise and optimized decision-making. 55
  56. 56. Traditional BI The New BI based on open standards and looselyclient server, closed, coupled services that can beproprietary architecture reconfigured easilystructured data (data gathering data of any source is useddepends on data warehousing (structured, semi- and unstructuredmethodology) data data)analytics and presentation no separation between analytics andare separated ; data-centric analytics presentation ; decision-centric 56
  57. 57. Traditional BI The New BI deliver relevant data, ensurecreate data models, control security and scalability, enableof data and applications IT role self-servicefocused on standard reports ; focused on interactive analysispredefinied reports to answer by end-users ; used to derive newpredefined questions BI-delivery insights (“business discovery”)on premise, desktop and on premise and on demandserver deployment type (cloud, SaaS) 57
  58. 58. traditional report-centric approach data discovery approach monolithic applications intuitive applications close coupled enterprise loose coupled services architecture « app-ification » IT-driven user-drivendata warehousing infrastructure Web-based (cloud-)infrastructure STRUCTURED DATA (RDBMS) STRUCTURED & SEMI-/UNSTRUCTURED DATA 58
  59. 59. technological innovations Consumerization are user-driven and increasingly of IT outside central IT-control self-directed analytics business discovery long tail solutions reusability infrastructure Traditional IT data governance securityAdapted from Hinchcliffe, 2011. 59
  60. 60. Drivers of the consumerization of IT CoIT UBIQUITOUS CONNECTIVITY 60
  61. 61. User-generated contentPower shift from expert-generated touser-generated content. Becausemarkets are more volatile, businessesseek greater agility to respond fasterto market requirements. Thedemocratizaton of BI is driven bottom-up and top-down. Users wantcustomized tools, while the ability tomine data is critical for businesscompetitiveness, which causesinformed decision-making to be CoIT Crowdsourcing. Architecture of participation.extended across more roles. UBIQUITOUS CONNECTIVITY Big data. The googlization of BI. Data and desktopBI as a service virtualizationThe cloud as a delivery Accessing data and applicationsmechanism for self-service BI. from any location, on any device, at any time. 61
  62. 62. interactive data visualization (business discovery) in-memory Web-baseddata management delivery(processing large amounts (delivery to a variety of data) of devices) self-service, fact-based decisions, agile BI 62
  63. 63. The BI-landscape is reshaped by the model of the consumer Web. user-driven analysis, open standards, intuitive user loosely coupled servicesinterfaces, easy to use, work from browser, culture of sharing real-time, and collaboration zero wait, app-driven, multiple devices 63
  64. 64. Collaboration is more than distributing and sharing ofBusiness users are empowered to documents ; it implies bringinggain insights into data (through context to analytics : differentexploration, visualization) people track the relevancy of analytics and the decisions that will be based on it The result is faster and better decision-making Value created from data can be shared internally within the company and externally with customers and partners 64
  65. 65. The New Normal in BI6. Collaborative BI 65
  66. 66. • The idea of collaborative BI is to extend the processes of data organization, analysis and decision-making beyond company borders.• While Web 2.0-technologies are migrating into the enterprise, consumer-oriented social media tools do not provide the necessary components for collaborative BI. Collaborative BI requires the principle of information sharing to be incorporated into day-to-day workflows.• A difference also exists between analyzing social media on the one hand and collaborative BI on the other hand.• Social media provide a new source of data that complements traditional data analysis to help organizations capture market trends, better understand customer attitudes and behaviour and uncover product sentiments.• Collaborative BI uses web-based standards to connect people (enterprise users, partners, suppliers, customers) to build dynamic networks that share information and analysis results to enable timely decisions that drive actions. 66
  67. 67. • Collaborative BI correlates with the analysis of big data and self-service BI.• Big data involves the analysis of ever-increasing volumes of structured and semi- or unstructured data. In the context of always changing business requirements, organizations need to act quickly and decisively on business and consumer trends derived from petabytes of data.• Closely related to the expectations of users to access applications anaywhere, at any time on any device are self- service features that allow them to interact with data in a flexible way. Accordingly, technologies as advanced data visualization, embedded BI and in-memory analysis rank high in preference lists.• The pervasive use of BI that is stimulated through these technologies is a necessity to enable analytic agility and responsiveness. 67
  68. 68. Contrary to the traditional linear nature of data processing, collaborative BIincorporates various feedback loops at different places in the analysis cycle. Applied to BI, collaboration frameworks can be built that enable teams to interact and socialize on data analysis-related topics. 68
  69. 69. « The world is rapidly turning into a network society. … The need to quickly adapt to this changing environment is evident. The new paradigm in innovation is joining forces in an online environment and activily working together. If we collaborate, we can co-create and grow our ideas together, which ultimately leads to better, and higher value Innovation ». A McKinsey study gives evidence that the application of Web 2.0- technologies to increase collaboration fosters the creation of networked organizations. Enterprises that connect employees to forge close networks with customers, business partners and suppliers become more competitive and show improved performance in the areas of market share gains, market leadership and margins. Through the use of collaborative tools, information flows become less hierarchical and access to expert knowledge is facilitated. Operational costs and time to market for new products/services are reduced.The rise of the networked enterprise : Web 2.0 finds its payday, McKinsey Quarterly, spring 2011. 69
  70. 70. • The business value of Web 2.0 for collaborative BI can be situated from the eight core patterns of Web 2.0. 70
  71. 71. Web 2.0-features focus on the user experience. Thecustomer-centric focus of Web 2.0 has created a demandfor applications that move from the traditionaltransaction platform to a model that is more accessibleand personal for the user.Web 2.0-applications represent an opportunity for BI tobuild Web-based collaboration. Reports can be publishedin blogs and wikis, which help construct a knowledge baseto share interpretations. Users will learn to useinformation more dynamically which allows thegeneration of « crowd-sourced wisdom ». Besidesreporting and analysis, decisions are part of the BIdelivery mechanism. Gaining insights from data to drive better decisions is no longer constrained by the limits of internal data. The open access to information in the Web 2.0-space allows users to combine existing information with consumer- generated content from the social networking spectrum like blogs and wikis. Social media analytics presents a unique opportunity to threat the market as a « conversation » between consumers and businesses. Companies that harness the knowledge of social networks compile enterprise data with streams of real-time data from Web 2.0-sources to better access marketplace trends and customer needs. The adoption of Web 2.0-technologies and applications can help businesses to expand the reach of BI and improve its effectiveness. 71
  72. 72. The New Normal in BI7. Social Media Analytics 72
  73. 73. • An important BI trend is the incorporation of the growing streams of data generated by social media networks in BI- applications.• Social BI is a type of intelligence that focuses on data that is generated in real-time through Internet-powered connections between businesses and the public.• Social media analytics give companies insights into the mindset of their (prospective) customers, help them improve media campaigns and offerings and accelerate responses to shifts in the marketplace. 73
  74. 74. Drivers for social media analytics 74
  75. 75. The spectrum of available data has been enlarged with new soures, esp. social mediadata streams. 75
  76. 76. The explosion of social media drives the need to analyze andget insights from customer conversations. 76
  77. 77. The mobile and social media explosion empowers customers and through the rapid growth of digital channels, the customer experience takes on a new meaning. The objective of social media analytics is to analyze social media data in context and generate unique customer experiences across channels. interaction datadescriptive attitudinal data data behavioral data 77
  78. 78. Examples of the use of social media analytics in day-to-day operations :• Baynote ( provides • Wise Window ( distills recommendation services for websites. social media content automatically and in real- Websites using Baynote recommendations time into industry-specific taxonomies. The deliver relevant products and personalized approach that Wise Window calls « Mass content that create an intuitive user Opinion Business Intelligence » (MOBI) does not experience. focus on individual behavior but the type of syndicated research that Wise Window performs is aimed at giving a broader• Baynote applies « interest mining ». It understanding of consumer sentiments and attempts to cluster consumers to provide behavior in the market at large. product or content recommendations that are based on a broader understanding of consumer behaviour. Baynote goes beyond • MOBI discovers leading indicators with data the clickstream by examining the words derived from social media to make associated with the clicks the user makes. organizations more agile and responsive. Combining the clickstream and the semantic Application fields include simple mindshare stream reveals the communality of cluster analysis, discovering new products and niches, members above a pure statistical or spotting fast movers, performing constituent demographic cluster approach. The resulting analysis and predicting demand. « integrest graph » is used to personalize product and content recommendations that lead to maximum engagement, conversion 78 and lifetime value.
  79. 79. The New Normal in BI8. Predictive Analytics 79
  80. 80. Traditionally, BI systems provided a retrospective view of the business by querying data warehouses containing historical data. Contrary to this, contemporary BI-systems analyze real-time event streams in memory. Analysis In today’s rapidly changing business environment, organizational(Why did it happen ?) agility not only depends on operational monitoring of how the business is performing but also on the prediction of future Reporting outcomes which is critical for a sustainable competitive position. (What happened ?) Predictive analytics leverages actionable intelligence that can be integrated in operational processes. HISTORY FUTURE PRESENT Monitoring Predictive Analytics (What is happening now ?) (What might happen ?) 80
  81. 81. Potential growth vs. commitment for analytics options advanced analytics (e.g. mining, predictive) data marts for analytics advanced data visualization predictive analyticscommitment enterprise data warehouse (EDW) analytics processed within EDW statistical analysis data mining OLAP tools real- time reports or dashboards analytic database scoring outside the EDW in- database analytics accelerator (hardware or software based) hand- coded SQL data warehouse appliance text mining DBMS for data warehousing in- memory database sandboxes for analytics column oriented storage engine visual discovery private cloud DBMS for transaction processing closed- loop processing mixed workloads in a DW MapReduce, Hadoop, Complex Event Processing extreme SQL in- line analytics public cloud Software as a Service -30 -15 0 15 30 45 potential growthGraphic based on survey results reported in Big Data Analytics, TDW Best Practices Report, Q4 2011, pp. 23.Potential growth is an indicator for the growth or decline of usage for big data analytics over the next three years.Commitment is a cumulative measure representing the percentage of respondens (N= 325) who selected using today and/or using in three years. 81
  82. 82. Current trends affecting predictive analytics : 82
  83. 83. Standards for data mining and model deployment : CRISP-DM • A systematic approach to guide the data mining process has been developed by a consortium of vendor and users of data mining, known as Cross Industry Standard for Data Mining (CRISP-DM). • In the CRISP-DM model, data mining is described as an interative process that is depicted in several phases (business and data understanding, data preparation, modeling, evaluation and deployment) and their respective tasks. Leading vendors of analytical software offer workbenches that make the CRISP-DM process explicit. 83
  84. 84. Standards for data mining and model deployment : PMML • To deliver a measurable ROI, predictive analytics requires a focus on decision optimization to achieve business objectives. A key element to make predictive analytics pervasive is the integration with commercial lines operations. Without disrupting these operations, business users should be able to take advantage of the guidance of predictive models. • For example, in operational environments with frequent customer interactions, high-speed scoring of real-time data is needed to refine recommendations in agent-customer interactions that address specific goals, e.g. improve retention offers. A model deployed for these goals acts as a decision engine by routing the results of predictive analytics to users in the form of recommendations or action messages. • A major development for the integration of predictive models in business applications is the PMML-standard (Predictive Model Markup Language) that separates the results of data mining from the tools that are used for knowledge discovery. 84
  85. 85. 85
  86. 86. PMML represents an open standard for interoperability of predictive models. Most development environments can export models in PMML. As analytics increasingly drive business decisions, open standards like PMML facilitate the integration of predictive models into operational systems. The deployment of predictive models in an existing IT-infrastructure no longer depends on custom code or the processing of a proprietary language.Besides the flexible integration of predictive models into businessapplications, continuous analysis is key to enable business processoptimization. The broad acceptance of the PMML-standard furtherstimulates the exchange of predictive models. Open standards likePMML contribute to the wider adoption of predictive analytics andstimulate collaboration between stakeholders of a businessprocess. In a similar vein, the increased use of open-sourcesoftware can profit from PMML. Open-source environments canvisualize and further refine predictive models that were producedin a different environment. 86
  87. 87. Structured and unstructured data types• The field of advanced analytics is moving towards providing a number of solutions for the handling of big data. Characteristic for the new marketing data is its text-formatted content in unstructured data sources which covers « the consumer’s sphere of influence » : analytics must be able to capture and analyze consumer-initiated communication.• By analyzing growing streams of social media content and sifting through sentiment and behavioral data that emanates from online communities, it is possible to acquire powerful insights into consumer attitudes and behaviour. Social media content gives an instant view of what is taking place in the ecosystem of the organization. Enterprises can leverage insights from social media content to adapt marketing, sales and product strategies in an agile way.• The convergence between social media feeds and analytics also goes beyond the aggregate level. Social network analytics enhance the value of predictive modeling tools and business processes will benefit from new inputs that are deployed. For example, the accuracy and effectiveness of predictive churn analytics can be increased by adding social network information that identifies influential users and the effects of their actions on other group members. 87
  88. 88. Predictive modeling Advanced visualization multidimensional Self-service view of data business discovery in an interactive wayData-as-a-servicemaking multipledata sources Social mediaavailable for analysis analytics analyze customer Text mining sentiment pattern detection in unstructured data Collaboration adding context to decision making Real-time dashboards monitor KPI’s 88
  89. 89. Advances in database technology : big data and predictive analytics• As companies gather larger volumes of data, the need for the execution of predictive models becomes more prevalent.• A known practice is to build and test predictive models in a development environment that consists of operational data and warehousing data. In many cases analysts work with a subset of data through sampling. Once developed, a model is copied to a runtime environment where it can be deployed with PMML. A user of an operational application can invoke a stored predictive model by including user defined functions in SQL- statements. This causes the RDBMS to mine the data iself without transferring the data into a separate file. The criteria expressed in a predictive model can be used to score, segment, rank or classify records.• An emerging practice to work with all data and directly deploy predictive models is in-database analytics. For example, Zementis ( and Greenplum ( have joined forces to score huge amounts of data in-parallel. The Universal PMLL Plug-in developed by Zementis is an in-database scoring engine that fully supports the PMML-standard to execute predictive models from commerial and open source data mining tools within the database. 89
  90. 90. Data is partitioned across multiplesegment servers and each segmentmanages a distinct portion of theoverall data.The Universal PMML Plug-in enablespredictive analytics directly withinthe Greenplum Database for high-performance scoring in a massivelyparallel environment. 90
  91. 91. Predictive analytics in the cloud• While vendors implement predictive analytics capabilities into their databases, a similar development is taking place in the cloud. This has an impact on how the cloud can assist businesses to manage business processes more efficiently and effectively. Of particular importance is how cloud computing and SaaS provide an infrastructure for the rapid development of predictive models in combination with open standards. The PMML standard has yet received considerable adoption and combined with a service-oriented archirtecture for the design of loosely coupled systems, the cloud computing/SaaS model offers a cost-effective way to implement predictive models.• As an illustration of how predictive models can be hosted in the cloud, we refer to the ADAPA scoring engine (Adaptive Decision and Predictive Analytics, ADAPA is an on demand predictive analytics solution that combines open standarfds and deployment capabilities. The data infrastructure to launch ADAPA in the cloud is provided by Amazon Web Services ( Models developed with PMML-compliant software tools (e.g. SAS, Knime, R, ..) can be easily uploaded in the ADAPA environment. 91
  92. 92. Since models are developed outside the ADAPA environment, a firststep of model deployment consists of a verification step to ensurethat both the scoring engine and the model development environmentproduce the same results. Once verified, models are executed eitherin batch or in real-tile. Batch processing implies that records are runagainst a loaded model. After processing, a file with the input andpredicted values is available for download. Real-time execution ofmodels in enterprise systems is performed through Web servicesthat are the base for interoperability. As new events occur, a requestis submitted to the ADAPA engine for processing and the results ofpredictive modeling are available almost simultaneously. 92
  93. 93. • The on-demand paradigm allows businesses to use sophisticated software applications over the Internet, resulting in a faster time to production with a reduction of total cost of ownership.• Moving predictive analytics into the cloud also accelerates the trend towards self-service BI. The so-called democratization of data implies that data access and analytics should be available across the enterprise. The fact that data volumes are increasing as well as the need for insights from data, reinforce the trend for self- guided analysis. The focus on the latter also stems from the often long development backlogs that users experience in the enterprise context. Contrary to this, cloud computing and Saas enable organizations to make use of solutions that are tailored to specific business problems and complement existing systems. 93
  94. 94. • PMML represents a common standard for the representation of predictive models.• PMML eliminates the barriers between model development and model deployment.• Through PMML predictive models can be embedded directly in a database.• PMML-models can score data on a massive scale through parallel processing or in the cloud. 94
  95. 95. BI has evolved from performance reporting on historical data to thepervasive use of real-time data from disparate sources.To respond faster to market conditions, a much broader user baseneeds data access to interactively explore and visualize informationsources and share insights to make faster and betterinformed decisions.In the era of big data, a Web-based platform enables businessdiscovery and data as well as analytics are consumed as servicesin the cloud. 95
  96. 96. ReferencesBOHRINGER, M., GLUCHOWSKI, P., KURZE, Chr. & SCHIEDER, Cgr., A business intelligence perspective on the future Internet,AMCIS 2010 Proceedings, Paper 267.COUTURIER, H., NEIDECKER-LUTZ, B., SCHMIDT, V.A. & WOODS, D., Understanding the future Internet, Evolved TechnologistPress, New York, 2011.ECKERSON, W., BI delivery framework 2020, Beye NETWORK, march 2011.GUAZZELLI, A., STATHATOS, K., ZELLER, M., Efficient deployment of predictive analytics through open standards andCloud computing, SIGKDD Explorations, 11, issue 1, pp. 32-38.HINCHCLIFFE, D., Next-generation ecosystems and its key success factors, Dachis Group, 2011.MICU, A.C., DEDEKER, K., LEWIS, I., MORAN, R., NETZER, O., PLUMMER, J. & RUBINSON, J., The shape of marketing research in2021, Journal of Advertising Research, 51, march 2011, pp. 213-221.RUSSOM, Ph., Big data analytics, TDWI Best Practices Report, Q4 2011.SINGH KHALSA, R.H., REASON, A., BIERE, M., MEYERS, C., GREGGO, A & DEVINE, M., A convergence in application architecturesand new paradigms in computing. SOA, composite applications and cloud computing, IBM, january 2009.SINGH KHALSA, R.H., REASON, A. & BIERE, M., The new era of collaborative business intelligence, IBM, march 2010. 96