Vendor comparisons: the end game in business intelligence


Published on

The winners among vendors will have exceptional capability in implementation of large projects pulling together capabilities in reporting and querying, multidimensional analysis, analytics, data management, visualization and business process management

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Vendor comparisons: the end game in business intelligence

  1. 1. VENDOR COMPARISONS: THE END GAME IN BUSINESS INTELLIGENCEA great deal of innovation in the business intelligence industry, in the past, will culminate with theemergence of players who will dominate the industry for years to come. The winners will becompanies which are able to reduce the latencies in data gathering, analysis and decisionmaking. Since information for decision making requires multi-dimensional data, users have to beable to aggregate information from diverse sources; the information should include not onlyquantitative information but also qualitative information such as conversations, notes, images andvideos. The patterns in the data have to be discerned and understood as quickly as possible sothat action in taken before an opportunity is lost.Feedback from customers provides clinching evidence that ease of integration is the most valuedattribute for customers. Inter-linked transaction systems allow companies to aggregate data fromtheir CRM, SCM, production and financial systems. All this data has to be free from errors andthe definitions have to be consistent across all sources. The extraction of patterns of data is aidedby machine learning systems and comprehended quickly if it is vividly visualized. Decisionmaking and its implementation involves the broad majority of employees in a company who needto be able to compare their data with agreed standards of performance before they can takeaction. All employees have to be able to share the same data and access in user friendly form. Inorder to take action, the processes of companies have to be flexible enough to respond tosituations as they happen.The winners among vendors will have exceptional capability in implementation of large projectspulling together capabilities in reporting and querying, multidimensional analysis, analytics, datamanagement, visualization and business process management. Pure play business intelligencevendors, such as Cognos, Business Objects, Hyperion, as much as ERP vendors likeOracle/Siebel, IBM and advanced analytics players, such as SAS and SPSS, and upstarts likeQliktech are all looking to provide suites of business intelligence functionalities. While pure playvendors have an edge in consolidating products, the ERP players have accumulated competencein integration of applications, business processes and data, the advanced analytics group ofvendors has strengths in enhancing the value of data by drawing insights for decision makingwhile upstarts continue to tap disruptive new technologies.Enterprise scale business intelligence suites are the preferred flavor in the business communityeager to reduce complexity and costs. Pure Play business intelligence vendors have respondedto competition from ERP companies by tightly integrating their products. They are also agnosticabout the databases and applications and are more inclined to use service-oriented architectureto be able to access any source of data on their network. BusinessObjects, for example, hasintegrated its Crystal Enterprise and BusinessObjects products as a single suite which can beoperated with a common set of administrative tools thereby lowering the costs of installation. Inaddition, users are able to take advantage of the composite of business intelligence functions
  2. 2. including reporting, ad hoc queries, OLAP and dashboards. Cognos 8 has unified its OLAP(PowerPlay), Visualizer, Metrics Manager and NoticeCast, into the services-oriented architecturethat under grids its operational reporting tool ReportNet in a web based environment. In addition,Cognos has improved access to data from the entire enterprise as a result of integration with itsDecisionStream ETL tool which can now be managed by ReportNet. As a result of thepartnership with Composite software, Cognos 8 users have the ability to query data from anydatabase such as Oracle, DB2, etc.On the other hand, database and ERP vendors such as Oracle, Teradata and IBM areconsolidating their products by integrating the business intelligence functions into their databaseswhich considerably lowers the latencies in the transfer of data for analytical purposes. Microsoftbundles its online analytic processing (OLAP) with SQL Server and has added data mining and areporting server. Oracles10g database incorporates several of the routine business intelligencefunctions into its database. SAP, Oracle, Siebel and Microsoft all offer products with automatedbusiness processes; an update on a table triggers processes within the database, setsapplications and business processes in motion, causes updates in other databases, initiatescommunication with users, and even trigger remote procedures in external systems. A tightintegration of Siebel Analytics package into its CRM applications helps to steer workflow andreceive real-time information.All this is much harder with pure play business intelligence vendors who have to partner or makerisky acquisitions to achieve the same objective. These capabilities are important to lower thelatencies between the time a decision is taken and relevant actions are executed.The other major advantage the traditional ERP and database vendors have to offer are theirplatforms that support the broad range of functions such as composite applications, businessprocesses and data integration technologies. In the web services and SOA environment,platforms are particularly useful to join myriad services. Within an SOA framework like ProjectFusion of Oracle or mySAP of SAP, Microsoft’s SQL Server 2005, diversity of functionalities canbe incorporated rendering business intelligence packages irrelevant.REAL TIME PREDICTIVE ANALYTICS AND DATABASE PROVIDERSOne instance of the advantage database providers have in the business intelligence domain istheir ability to build in predictive analytics required for operational purposes. Several decisions arerecurring in nature yet they require up-to-date data for sound decisions. Financial institutionshave to be able to make judgments about credit worthiness, increasingly in real time, before theycan accept credit card applications from retail customers. Similarly, customers are often swayedby fads when they choose colors for their cars or clothes or specific models. Other times somecombinations of products sell well and they are better displayed next to each other. Customerchurn, cross-selling and pricing policies are other problems that need to be addressed in real
  3. 3. time. Sales people have to be able to make impromptu decisions about their stocking policy assuch information flows into their transaction databases.In the world of data warehouses and cubes, any kind of data analysis is preceded by an elaborateprocess of cleaning and reformatting data before it will be ready for analysis. In operationaldecisions, moreover, data volumes become overwhelmingly large and data warehouses much tooclunky to cope with the pressures of real time decision making. Increasingly, database providersare looking to build-in canned models for the analysis of data required for decisions in routineprocesses.The change has come with the advent of the Predictive Model Markup Language (PMML), anopen standards XML based language, which facilitates the transfer of models created in oneenvironment, such as SAS, and transfer it to a relational database. The XML tagging helps todescribe data inputs into data mining models, the transformations used in preparing data for datamining, and the parameters defining the data mining models so that the data mining algorithmscan be transferred to any environment whether it is CRM, SCM or production data.IBM, for example, has a partnership with SAS, to create scoring models and transfer them to therelational context of its DB2 database. While SQL is not meant to address complex queries, it hasthe ability to find answers to relatively simple questions of most operational staff. Once thepredictive models are embedded in relational databases, they can be accessed and modifiedusing SQL. The data for such purposes is drawn directly from transactional databases and doesnot have to be processed in a data warehouse. Similarly, Microsoft is building equivalentcapabilities for its SQL Server 2005.The pure play BI vendors, on the other hand, are at a disadvantage as they have traditionallyused cubes for their analytical routines. Microstrategy is one exception among them with its longstanding ROLAP capabilities and is using embedded DBMS models to provide predictivemodeling capabilities for report generation. Hyperion has incorporated predictive analytics in itsEssbase product where the predictive model is added as another dimension.Intelligence in the language I understandThe ideal that customers want to achieve in integration is to find information as close to humannatural language as possible and in a form that reflects their thought processes. This impliespulling together data in any form whether structured or unstructured. They want to be able tosearch information that matches concepts rather than specific queries. XML has helped to breaksome barriers by affording an ability to describe data. Web Services and SOA architecture hashelped to join information from a variety of sources. Semantic metadata has built the foundationfor natural language searches.When information is available close to natural language, decision makers are better able tovisualize a scenario before they can make decisions. In order for these decisions to beactionable, decision makers need levers to implement their decisions. Integration, in other words,
  4. 4. is not simply a question of inter-linking applications and business processes as EnterpriseApplications Integration does. Similarly, integration is more than linking all data sources asEnterprise Information Integration does. With the integration of data sources, enterprises canbegin to use metadata and federated queries to parse data spread all around their network. BothEAI and EII have proved to be expensive so vendors are turning increasingly to grid computing tolower costs. In the final analysis, integration is the sum total of integration of all sources of data,metadata, applications and business processes.Large scale vendors with their scalable platforms are best positioned to unify the diverseelements that can help to extract information and present it in a form that is intelligible to decisionmakers. Companies like Oracle, IBM, and Microsoft have long had strengths in applicationservers that can help to bring together the composite of services required for businessintelligence purposes. Pure play business intelligence vendors, on the other hand, havecollaborated with companies, such as Composite Software, which provide integration servers.Vendors have progressively moved from exposing legacy software as services and integratethem with software of more recent vintage to increasingly a portfolio of services or compositeapplications to department wide integration of exposed services to tentative efforts at enterprisewide services-oriented architecture. One example of the early attempts at creating tools forenterprise scale services architecture is SAPs Enterprise Services Architecture which providesthe tools to create services for use across an enterprise, to weave business processes with theservices using the SAP Composite Application Framework and to implement those processes onthe NetWeaver application server. BPEL-based service orchestration is used to integrateenterprise services with SAP and non-SAP applications, including their business processes. Thedistinctive aspect of SAP’s SOA strategy is that it exposes its ERP applications to services thussaving its customers a major overhaul of their architecture. Siebel has also redesigned its CRMso that it can be exposed as a service.Oracle is able to integrate web services with its "Oracle Fusion Middleware," centered around itsOracle Application Server 10g, web services orchestrated on J2EE (Java 2 Enterprise Edition)Application Server Web services infrastructure, ESBs (enterprise service buses) and integrationserver. At the base of the Fusion stack lie Oracles version of the database grid supportedclusters of several computers. Although Oracle initially expected customers to choose its ownapplication server, it has increasingly been willing to integrate the products of other companiesincluding WebSphere.Additional tools are available for business process management and activity monitoring tools,business intelligence tools and enterprise portals as well as Oracles data hubs and the OracleCollaboration Suite. Oracle also uses BPEL based business process integration; unlike SAP, itextends the scope to all processes. Fusion middleware is open system architecture for integratingservices including those from other vendors such as IBM.
  5. 5. The action plansThe acid test for competing vendors would be the ability to orchestrate business processes withapplications and data flows. Flexible business processes are pivotal to translating strategy intoactions. Business users should be able to change business processes with ease using visualtools. They need to be also monitor business processes and the performance parameters in orderto determine where efficiencies are possible.A more flexible approach to management of business processes is now possible with theemergence of Business Process Execution Language (BPEL). Standards based languages, suchas BPEL, enable companies to avoid vendor lock-in and facilitate widespread adoption. Thislanguage is intuitive enough to let business users set up a flow of business activity; a series ofprocesses can be automated when the first of them is initiated. So a customer could request aticket for travel which will trigger a series of actions such as checking for availability, selecting aseat, issuing a ticket, receiving a payment and depositing the money in a bank. More complexbusiness process integration will record the revenue in accounting software. All of this can bedone as a seamless flow of activity if none of these processes are embedded in any specificapplication.While integration of a few of the business processes has happened already, the more complexintegrations are beginning to happen with the entry of the larger vendors. Comprehensiveintegration enables a company to optimize and simulate to extract efficiencies. Companies canuse data from their business processes to take decisions on improving operational efficiency.Additional benefits follow when automated business processes respond to a new event. Forexample, a retail store may find that inventories are lower than expected and its businessprocesses will initiate action to replenish them.The emergence of business process management tools which can be operated by businessusers, without the assistance from IT, is illustrated by Microsoft’s BizTalk Server. Businessprocesses can be programmed with the use of Visual Studio .NET 2003 developmentenvironment. Alternatively, the server has the ability to insert Visio for business users to alterbusiness processes as they see fit.The integration of business intelligence software with business processes paves the way forbusiness activity monitoring as well as event based monitoring of workflows. Business activitymonitoring compares planned performance to the actual. Events management software sets upalerts so that managers receive warnings when the exceptions are experienced. One of theleading players in this segment is Teradata with its Active Warehouse. Oracle and IBM have alsointroduced sophisticated programs for business process management.Master data: Navigating complex information systemsMaster data management provides consistent definitions of data in a services-orientedarchitecture where heterogeneous applications have to co-exist. The availability of consistent
  6. 6. definitions helps to considerably improve efficiencies by smoother cross-flow of processes. In thepast, each application had its own way to define business process and logic. When theseapplications were integrated, the data stored with these applications had inconsistent definitions.In addition, the data was duplicated in several applications and created confusion when it wascombined. Master data management systems provide a centralized library of business operationssuch as querying customer information. The availability of master data management systemshelps to solve problems of data quality. However, applications have to be able to call informationfrom master data management systems before they can be utilized.The extent of standardization of definitions can vary across different business intelligenceproviders. Ideally, enterprises would like to see a master data management system iscomprehensive to include both unstructured and structured data and include all types of businessprocesses such as CRM, supply chain management and manufacturing data. For vendors, thecost and complexity of Master Data Management systems grows as they aggregate moreinformation. The MDM solution offered by SAP, for example, is focused on transaction processingwhile Hyperion MDM product is oriented toward business intelligence.The usability of master data management systems depends greatly on how well they areintegrated with transaction and business intelligence systems and the ability to use them at run-time. SAP, for example, has integrated its master data into its Enterprise Services Architecturewhich enables its applications to look up data at run time. Siebel Systems has a market leadingmaster data management system in its Siebels Universal Customer Master (UCM) managementand Universal Application Networking (UAN) systems which can work together to call datadefinitions at run-time. IBM is also incorporating its Master Data Management System in itsservice-oriented architecture so that data definitions can be called when applications need them.Mining structured dataIncreasingly, enterprises are looking for data mining solutions that provide analysis in real time tofeed into decision here and now. The typical problems they want to solve are to anticipatecustomer churn or determining the credit worthiness of their customers. Analysis in such a shortperiod of time implies that data has to be extracted, cleaned and prepared for analysis in a shortenough intervals for decisions to be made. Some new entrants have found an opportunity in thislargely unaddressed segment of the market. KXEN, for example, offers Analytic Frameworkproduct which reduces the time to define, develop and run a model. KXENs Consistent Codermodule automatically transforms raw, inconsistent data into clean, uniformly formatted data readyfor modeling.Several other vendors have offered solutions for the improvement of business processes in realtime. BusinessObjects XI has added BusinessObjects Process Tracker and BusinessObjectsProcess Analysis that embed analytics in their business processes. The data on performancemetrics are linked to alerting capabilities so that managers can take action in real time. Cognos
  7. 7. has software, e-Applications, for supply chain management processes such as procurement,sales and inventory. The tool allows customers to keep track of the performance metrics of theirsuppliers and respond to alerts about events.Mining unstructured dataUnstructured data is available in much larger volumes and is of greater interest in operationalsituations where qualitative information, such as the lifestyles of customers, is more relevant.Information from text documents can be extracted in a variety of ways including categorization,classification, information extraction, summarization, identifying themes or topics, concepts,information visualization and responses to questions.Information extraction looks for key phrases such as “tourists vacationing in San Francisco tendto come from China and other East Asian countries” in order to find data on the travel behavior oftourists in California. When conducting topic searches, text mining tools search for broad themessuch as “the Japanese stock market performance” to cull out information most relevant for thetopic, text summarization tools scours for words like “in short” or “in conclusion” to find relatedinformation that tells the gist of the text, categorization can be accomplished by looking at thefrequency of usage of words and their synonyms; the recurrence of a word such as mining wouldsuggest that the document is about decision support analysis tools. Similarly, clustering methodscomb through text to find frequently occurring words and themes; a text on business intelligencewould have clusters of predictive mining, decision-making, etc. It is also possible to extractinformation related to a specific concept; a doctor could be looking for information on allergiesand would like to obtain related information on weather conditions, food habits and lifestyles. Textmining tools are also capable of visualizing information in the form of link tables that help tounderstand the mass of information. Finally, text mining can be used to answer specific questionsand the words used would suggest the required information.Vendors can be differentiated by their inclination to use text mining tools for knowledge extractionand real time operational needs of companies. Text mining tools, such as those offered by SASand SPSS, have strengths in information or knowledge extraction and categorization while themore recent entrants are focused on addressing the real time text analysis requirements whichwill focus more on clustering, summarization and visualization. SAS Text Miner has capabilities ininformation extraction, categorization and concept linkage. SPSS has strengths in informationextraction, categorization and information visualization. Megacomputer, canned software likeCognos on the other hand, has capabilities in summarization, clustering, answering questionsbesides categorization and information extraction.For the broader category of unstructured data, IBM Omnifind promises to leader in themarketplace with its capabilities in searching content including e-mails. The search engine in thesoftware indexes the information and lays the ground for queries.
  8. 8. Intelligent data cleaningData cleaning software has a number of tools such as data profiling which finds inconsistencies,parsing which identifies different types of data and places them in the relevant fields,standardization which brings consistency into data from a variety of sources and verification toolsfor comparing data against a universal master such as the U.S. Postal Service, matching whichlinks interrelated files and consolidation which eliminates duplicate entries. The Master Data usedby most of such software does not take into account the context, the connotations, the overtoneor the undertone in much of human expression. Inevitably, a large majority of the conversions thathappen are prone to error and require inordinate human effort to make the corrections.Increasingly, vendors are looking to semantic metadata to do the translations of data from onesource to another. Such semantic metadata is conscious of the context in which language isused. Some of the newer technologies, such as those offered by Silver Creek Systems, are ableto improve the efficiencies in data cleaning.Pricing pressuresThe launch of suites and the entry of ERP players in the business intelligence market haveintensified the pressure to lower prices which will work to the disadvantage of pure play BIvendors. By all accounts, the recognized price leader in the market is Microsoft SQL Server.Microsofts SQL Server 2005, with its Analysis and Reporting Services, is priced at $80,000 for1,000 users while most BI suites have a list price in the range of $450,000 to $700,000. However,pricing data about business intelligence packages has several layers of complexity as strategicpricing is the norm and it is not always possible to make apples-to-apples comparisons.In a review of the data revealed during the anti-trust investigation conducted when Oracle made abid for PeopleSoft, the many caveats that have to be added when comparing prices wererevealed. Oracle’s well kept secret that spilled out that it willing to price out Microsoft at almostany cost—with as much as 90% discounts. Customer acquisition has a lucrative reward in themaintenance earnings that both companies can earn estimated at 25-30% of the licenserevenues.Mass adoptionReal time decisions have to be necessarily complemented by collaboration across the enterprise.The broad majority of employees in organizations can participate if their familiar tools, such asspreadsheets, are embedded in business intelligence tools. In the future, however, spreadsheetscannot be used in isolation and have to be incorporated in enterprise wide systems; they have tobe able to migrate from desktops to servers. In the past, spreadsheets also allowed individualusers, often highly educated business analysts, to manually configure their spreadsheets to suittheir analytical needs. The formulas and the data were often lost when an employee left. In abusiness intelligence environment, users have to be able to share their data and analytical
  9. 9. techniques with the larger community. It should be possible for all users, whatever their skill level,to reuse the formulas somebody else might have created.In the past, the data from business intelligence tools was, at best, exported to Excel sheets whereit could be manipulated in intractable ways and the final results were not imported back.Increasingly, customers are looking for tools that integrate Excel spreadsheets with corporatedatabases, relational or multi-dimensional, consistent with the format and architecture of theirbusiness intelligence systems. The data should be available across all information systems andall users should be able to trace back the methods used in analysis.The players that stand-out in their integration of excel spreadsheets into business intelligencesystems are Hyperion, Actuate, Information Builders, Business Objects Outlook Soft, SAP andlately Oracle. Hyperion was one of the earliest among leading Business Intelligence players andSRC Software, later acquired by Business Objects, was among the first to offer an Excelinterface.The value of integration of Windows Office is potentially more than the usability of a familiarinterface. Much greater benefits can be reaped when the Office applications integrate with theapplications, data and business processes of enterprises. Business intelligence vendors areincreasingly trying to gain an edge over their competitors by linking inter-related processes,applications and data with a convenient Office interface. The joint product “Mendocino”, createdby the partnership between SAP and Microsoft typifies the competitive trends in industry; theprocesses that were earlier integrated by APIs is increasingly done on a SOA platform and helpsto realize much larger gains in productivity.Reporting toolsTwo main types of reporting are available with business intelligence tools and these areproduction reporting and management reporting. Production reporting generates routinedocuments such as invoices, bank statements which are repeatable. Management reports, on theother hand, are ad hoc in nature and extract data to decision related questions such as how manycustomers bought goods worth more than $2000 in the Christmas season. Increasingly, vendorsseek to gain competitive advantage by building in the capability to generate more intelligentreports. Users of production reports soon begin to ask questions such as the reasons forexceptionally high debits recorded in their bank statements. Ad hoc reporting is meant for thebusiness analysts in companies. Over time, vendors have discovered a larger market for staticreports, with pre-defined templates and drill-down capabilities, for a much larger client base in theoperational staff of companies which is usually satisfied with simple queries most relevant fortheir roles.The leaders among the group of vendors who focus on the reporting space are recognized to beCognos which was offering Cognos 7, recently upgraded to its eighth edition recently with greaterintegration of multidimensional cubes and the reporting engine, and Actuate 8. Cognos has been
  10. 10. a strong enough player to provoke Business Objects to acquire Crystal Reports to match itsreporting capabilities. Cognos stands out for its capabilities in ad hoc queries and a web interface.Actuate 8 has gained considerable recognition for its ability after its acquisition of Nimbletechnologies improved its ability to integrate with a diversity of data sources using EIItechnologies. In addition, Actuate has its eSpreadsheet interface.The visual big pictures in the detailBusiness intelligence vendors see in interactive visualization a means to gain an edge byproviding customers a means to extract insight from large data stores. Several differentapproaches are available to achieve this objective including geographical data, interactivity,animation, super-imposing objects on data and dimensionality of the graphics.Mapping of geo-spatial data is one of the means of relating data to location to understand trendsin terms of who, where and how determined them. A typical example could be the mapping ofconcentrations of population to understand the impact store location could have on thepurchasing behavior of customers. Insights can be extracted by visually estimating the time itwould take customers to reach the store location. Additional insights could be extracted if storelocations are compared with the centers of crime in the city. Vendors seek to gain a competitiveedge by integrating business intelligence and location information so that they can be juxtaposedon graphs which can be depicted without getting bogged down in tedious processes of dataextraction. SAS, one of the leaders in combining geographical information and business data,now offers SAS/GIS which integrate business and location data that it draws from ESRI, a longtime market leader in location information.Interactive visuals are another means to gain insight. While exploring information, users ofbusiness analytics software want to view data from a variety of angles and want to portrayinformation as their thought process evolves. They want to see not just pretty pictures butrelationships which would require them to slide, move and juxtapose components of their visualsto compare, contrast and highlight to bring into relief patterns and trends. They want to shuffle thevisuals to ask “what if” questions. In a typical application involving balanced scorecards, theywant to compare the planned and the actual performance. Infommersion, recently acquired byBusiness Objects, provides these very features relevant for decision-support analyticalpresentations.Users can gain better understanding of their data if they have the ability to conduct visual querieswhich enable them to select their data and their visuals to address the specific questions theyhave in their mind. Tableau, a start-up, has pioneered visual queries using business intelligencedata. Instead of slicing and dicing data, users are able to flip visuals to spot any anomalies in theirdata, noticeable trends or patterns that would be elusive especially in large data sets. The sameproduct has been licensed and renamed as Hyperion Visual Explorer.