The uses of pervasive intelligence


Published on

As it incorporates a gamut of functions from business activity monitoring to performance management and business planning, business intelligence attracts a growing number of companies who earlier specialized in individual functions

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

The uses of pervasive intelligence

  2. 2. Business Intelligence is an evolving industry with growing market potential which has attracted aswarm of players who are serving their customers in several different novel ways. It has grownfrom a technology focused on decision support and performance management in somedepartments to increasingly ubiquitous tool that spans operations management across theenterprise. As it incorporates a gamut of functions from business activity monitoring toperformance management and business planning, business intelligence attracts a growingnumber of companies who earlier specialized in individual functions. Customers have a dauntingtask of choosing from a rich crop of innovative packages; they will have to make a judgmentabout the products that will become the standard for the industry in the future.The growth of the business intelligence industry has been propelled by convergence of severalfactors which will continue to fuel rapid expansion and innovation. Regulatory compliance isinescapable and the need to monitor operations risks is the baseline for additional applications toreduce costs, fraud detection, accelerated responses to market changes, better targeting ofcustomers, detection of latent opportunities for product development and a personalizedapproach to selling among several other applications.Keeping an eye on all activitiesThe process of selection from several different offerings will be influenced by enterprises’preference for tactical or strategic goals or a combination of them. While decision-supportanalysis or performance management in selected functional areas are possible in isolation withbusiness intelligence tools alone, companies will need a comprehensive strategy, includingbusiness process management, content management and performance management, when theyweave business intelligence into the fabric of their daily business activity for rapid responses tomarket changes. Dell, for example, has embedded intelligence into its daily business and is ableto set in motion its business processes all along the supply chain, once a configuration for acomputer is spelt out by a customer, including orders for its contract manufacturers and shippingcompanies.In the past, spreadsheets, OLAP tools, management report generation and query tools were thepreferred tools as they catered to decision-support and performance management in selectivefunctional areas. Enterprises will need more of dashboards and performance management toolsas they look to monitor and evaluate business activity across the enterprise. They also need toable to collaborate and communicate rapidly for a coordinated response to contingencies facingthe enterprise based on the information received from performance metrics. In general, they needto aggregate, analyze and act in near real time.For their tactical or their operational needs, companies need information on all the metrics thatgovern their performance metrics and the parameters that form the co-ordinates of theirdecisions. In the past, companies could barely integrate information from a few of their
  3. 3. operational data stores and create standard reports to make decisions quickly; they looked fortools where they could drill down and view from different angles. Increasingly, enterprises willneed to inter-connect all their sources of data to take actions that are consistent with theirenterprise strategy.One instance of the use of performance measures is La Suisse Insurance which set targets forits sales force based on a multi-dimensional view of data. The company was losing up to $50,000per salesperson each year by paying monthly allowances to salespeople who wereunderperforming. An OLAP tool helped in gaining multiple views of sales performance — bysalesperson, branch, and region— which uncovered opportunities for raising productivity. Datawarehouses can extend the capability and include the analysis of impact of prices, advertising,etc. on the results achieved.Alternatively, companies can focus on decision-support tools which involve ad hoc queries andmake considerably greater demands on the analytical and data management capabilities ofbusiness intelligence software. This kind of query requires much larger data sets for cross-referencing across several dimensions of data as well as over time. Companies need to be ableto integrate data from several transaction data bases, usually in a data warehouse environment,and need the IT horsepower to conduct such complex queries. The typical applications of ad hocqueries are customer segmentation and response modeling.Strategic decisions, unlike operational decisions, take a longer time as companies need to beable to parse current and historical data before they can come to decisions. They involve the useof statistical and data mining tools to consider alternative scenarios, predict future financialperformance, conduct customer segmentation for product positioning and make decisions aboutthe choice of their channels. This is typically done with data stored in data warehouses andupdated periodically, typically overnight or on the weekends, in order not to interrupt analysisduring the day.A growing number of companies favor using business intelligence tools to align their strategy andtactics. They want to be able to make course corrections to ensure that their tactical initiatives arein line with their strategic direction including coping with contingencies which could roil their bestlaid plans. With the advent of active data warehousing and rule based decision engines, they areable to compare actual figures with benchmarks to come to decisions about routine decisionssuch as inventory replenishment, yield management or pricing decisions. This kind softwareallows companies to embed rules that prompt decisions when an expected event happens.Integration of decision and business process management for linking strategic and tacticalmanagement is rare (not impossible) but immensely beneficial for those who succeed. PSS WorldMedical based in Jacksonville, Fla. has a sprawling network of 44 distribution centers sellingmedical equipment and supplies. Each of the centers is tied, by a dashboard, to its own P&Lstatement and to the performance goals of each individual who are rewarded according to the
  4. 4. achievements of their metrics. An integrated network business processes and information flowsensures that the drivers of performance are transparent to the senior corporate management. Inthe past, performance measures relied on one measure such as productivity often at the expenseof quality. In a business intelligence context, balanced scorecards help to assemble a groups ofmetrics which best represent the correlation between individual performance and the companyoverall.Teradata’s Active Data Warehousing is one instance of a data warehouse concept which updatesdata, in real time, for operational and strategic decision making. Companies are then able torespond to significant events affecting their financial performance. In the past, batch processes indata warehouses prevented quick responses to events. An alternative method of achieving thesame objective is the standards based Service-Oriented Architecture (SOA) which piecestogether components of business processes, data services and applications and each of themcan be invoked by a web service.An instance of the use of event-based decision making is Land’s End which progressed fromusing dashboards for reporting to more event-based decision making. In the first stage ofimplementation of a BI solution, it began to provide figures of inventory as well as supplies in thepipeline and compared them with incoming customer demand to make decisions about orders tobe placed with vendors. Now it has embedded triggers in its systems so that alerts areautomatically sent out whenever inventory runs low.Event based monitoring does not stop at single or discrete situations but extends to responses toa sequence of events. Thus, the monitoring of the supply chain would involve the movement ofgoods from the vendor to the transportation company and then to the buyer. The events along theway can have consequences for inventory holdings and financial payments. Delays can occur onthe way due to weather changes or congestion on the roadways or the vehicle carrying the goodscould meet with an accident. Based on preset rules, the BI software can make simultaneousdecisions for alternative means to replenish inventories as well as financial payments totransportation companies.An example of responses to complex events management is the experience of American ElectricPower Company which often made duplicate payments to its numerous suppliers who submittedinvoices under different numbers and in several different formats such as e-mail, snail mail, etc.The complex event management software helps to detect duplicate payments by looking ataddresses, dates, amounts submitted and names of vendors to check for overpayments.Business Activity Monitoring provides the infrastructure to monitor events, compare metrics tostandards and sends out alerts when action is required. In the daily routine of business,companies need to keep track of their supply position. Often, supply will either exceed or runshort of requirements. BAM tools enable companies to compare the actual situation withthresholds and send out alerts if the situation warrants action by decision makers.
  5. 5. BAM has been widely adopted in the financial services, logistics and the telecommunicationsindustry and its acceptance in the mainstream is expected to be completed by 2008. In thefinancial services industry, it has been readily accepted as a large majority of trading decisions istriggered by news feeds about financial events. In the logistics industry, a large number ofscheduling decisions are prompted by information on progress of shipments. Similarly, thetelecommunications industry uses BAM to monitor the observance of service level agreements.One example of a product that servers such needs is Celequests ActivityServer suite whichprocesses data flowing continuously from transaction systems. It compares standards ofperformance and the actual metrics to determine when production yields are lower than the normor alerts store manager when inventory is running low. The standards of performance aredetermined by comparing historical data in data warehouse with the actual achievements.Brocade Communications uses Celequest’s Activity Server to monitor product yields achieved byits contract manufacturers.Enterprises would rather be nakedIn order to test their strategic assumptions and to be able to see how they play out in practice,company managements need greater visibility into their business processes before they canevaluate the impact of their actions. A recent survey of 300 business-technology executives foundthat close to 60% of them want greater visibility into their business processes while nearly 80% ofthem are interested in data on performance metrics. The implication of an interest in visibility oftheir business processes is that companies are looking for ways to monitor the ebb and flow ofbusiness activity and take preventive measures if adverse situations are encountered.For a closer integration of operational management and analytical capabilities for strategicmanagement, companies need to lower the information and decision latencies to be able torespond to situations in real time. Business Activity Monitoring helps to lower informationlatencies as it monitors current event data and uses BI to compare it to expected performance.Business Process Management (BPM) software has the tools to make corrections rapidly. Forexample, customer satisfaction will be affected by call wait time before customer representativescan respond to customer queries. Business Process Management has to be able to use thisinformation to reroute calls to another call center or representative. This would involve the tasksof finding the most efficient alternative route (modeling), ensuring that the traffic flow throughalternative routes to proportionate to capacity and the pathways are interconnected (integration).(BPM), which is designed to work across departments, enables organizations to alter automatedand coded processes, without major re-investments in IT, prompted by analysis of eventinformation. At its efficient best, companies will be able to automate marketing by definingcustomer data in a manner that is consistent with business decision rules and any turn in eventswill change the approach to marketing campaigns. Software such as Fair Isaacs Blaze Advisor,
  6. 6. Ilogs JRules or Pegasystems PegaRules are designed to read customer data and change theoffers made based on the customer information received.BPM is intelligible to business users and helps to manage workflows and the process design. Thecenterpiece of a BPM implementation is the process engine which monitors processes as well asoperational and business metrics, watching for exceptions that may require human intervention.One of the benefits of rapid feedback from tactical decisions is business activity monitoring (BAM)or the ability to take impromptu actions to remain on course to reach strategic objectives. Portfoliomanagers, for example, have to be able to makes adjustments in their holdings based on newsfeeds received. Delta Air Lines monitors the impact of unexpected events, such as weatherconditions and gate changes, to alert its employees and to update its gate display systems. Thiswould not be possible unless the analytical and operational systems are not integrated.Increasingly, business process management software provides a graphical view of the design ofworkflows as well as progress achieved all on a web site where it is transparent to the entirecompany. Blue Rhino, a gas distribution company, used BPM to optimize the flow of shipments offull gas tanks, inventories and empty tank returns among the companys 200 distributor locations.An interlinked inventory tracking system continuously updates data on stocks and obviates theneed to manually reconcile the numbers and to issue paper bills of lading. The more advancedBPM software also allows event management and triggers responses as exceptions are recordedsuch as the actions asset managers need to take following interest rate changes.IBM’s WebSphere Business Integration Monitor and TIBCO’s BusinessFactor have thecapabilities to use transaction data and conduct analytics to find out whether they meet standardsand they assess the ability of business processes to facilitate the expected performance metrics.These products can also retrieve historical data from a data warehouse to provide thebenchmarks. The more recent acquisitions of these two companies, Alphablox product by IBM,and TIBCO’s OpsFactor, have capabilities to monitor the impact of current events. This wouldenable companies to analyze the impact of events such as delayed shipments on their financialhealth.Business intelligence software needs to be seamlessly integrated with operational datamanagement, business process management and visualization to meet operational objectives. Inone survey of users of BI software, it was found that 32% of the respondents consider pre-packaged integration of BI software with existing enterprise applications as one of the mostsought after attribute. One instance of such an approach to product offerings is the case of SiebelBusiness Analytics platform which has web services interfaces that enable it to insert analyticalresults into Java or .Net and display the results in any application. Furthermore, its analytics areclosely intertwined with its CRM packages and directs the workflows for real time informationflows.
  7. 7. Visualization is essential for rapid collaborative decision making as illustrated by the case of SunNational Bank, a regional bank, which needed to compete with much larger banks. It created asingle data warehouse for all its branches and provided analytical software to monitorperformance metrics such as the fastest growing products, regional variations in revenue, allvisualized as charts and graphs, on dashboards for rapid decision making.A larger number of business and operational users need analytical tools that cater to their specificneeds. Consequently, analytical software needs to be scalable and designed for role-basedcustomization. A case where a customer switched from reporting tools to more scalableenterprise software is the Ministry of Tourism in Bahamas which had used an OLAP tool, withinits offices, for its processing power and analytical capabilities. Later, the ministry needed tocommunicate with 400 local hotels and regional tourist boards and switched to an enterprisereporting tool with server-based computing environments. Lately, OLAP providers have upgradedtheir scalability.As information is drawn from a growing number of functional departments, business intelligencesoftware should be able to draw data from a variety of repositories. Information Builders product,WebFocus 7, for example, provides access to more than 200 data sources and data formats,including relational and legacy data.An all-encompassing use of business intelligence is that companies find themselves looking atboth structured and unstructured data. Regulatory compliance, as required by Sarbanes Oxley,requires detailed monitoring of controls which often means finding a record among million others.Similarly, fraud detection in the corporate sector often requires sifting through thousands of e-mails which would be impossible with manual methods. It would be hard for companies touncover pain points in customer experience unless they are able to mine call centerconversations. Similarly, companies invariably experience delays in understanding the causes ofwarranty claims while shipments of the same product continue and later add to the costs ofrefunds or product recalls. It is important to be able to correlate structured and unstructuredinformation to complete the analysis. Attensity is one company which has products that can readas well as store structured and unstructured information in a relational form. Whirlpool uses itsRelational Extraction Server to extract information and develop insight about warranty claims,customer feedback and service records.One of the pioneers of an integrated use of structured and unstructured data is EDS, a systemsintegrator for over 9,000 corporate customers. Its large client base implies that it buys countlessservers, PCs, networking gear, software and services from numerous suppliers who sell to itsoffices spread in 65 countries. The contracts with its suppliers are a valuable source ofinformation to find ways to lower prices, as well as to evaluate suppliers based on their responsetimes and other metrics.
  8. 8. The benefits of integrating unstructured data can extend to streamlining the business processesincluding increasing the efficiencies in the supply chain. Motorola has used EII to integrate itsinformation flow to be able to see the order status at every stage of its supply chain network.Modern text mining technology has progressed beyond keyword searches of familiar searchengines and increasingly looks for items that fit a pattern. Searching by keywords often throws upirrelevant results because the meaning of words can change with context. This is achieved byusing taxonomies, while mining textual information, which can ferret out results that are inter-related. The significant difference in searching by taxonomies is illustrated by the experience ofChelsea & Scott Ltd., a retailer of childrens products under the OneStepAhead and Leaps andBounds brands. Customers could not often find the relevant products because they were notaware of their existence. They could, however, describe the needs of their babies such as theirneed for comfort. Installation of a natural language search engine change matters and a searchby comfort yields results of all the products that meet this need.An all encompassing view of the enterpriseThe integration of analytical and business processes requires additional choices involvingtechnologies that help to integrate them. Applications, content and data repositories, analyticalsoftware and workflows management software have existed independently as discrete processes.Increasingly, they will operate as a seamlessly integrated, continuous process that can bemonitored from an all embracing GUI or an enterprise portal. The analytical tools are integratedby “portlets” or web services. These portals are available from companies like BEA Systemswhich earlier specialized in application servers or players like Vignette which earlier specialized inenterprise content management or from companies like Plumtree, the only remaining companyfrom the former ‘pure-play’ portal providers.Alternatively, companies can also decide to integrate their analytical software with theiroperational applications such as Customer Relationship Management Software. In such asituation, they expose their Application Programming Interface to inter-link BI software orcomponents.Similarly, relational data base companies have increasingly integrated analytical applications ontheir databases. Microsoft, for example, now supports analytical applications on its ‘Yukon’ SQLServer which includes support for OLAP, statistical and data mining functions besides databasequeries.The integration of business and analytical applications has to extend to linking contentrepositories; this was not possible with the available integration technologies. With the advent ofweb services, XML and a gamut of object-oriented, component based technologies enablesintegration of structured and unstructured data. IBM has bolstered the prospects of thetechnology by acquiring Venetica and uses its content integration middleware into its DB2Information Integrator suite.
  9. 9. Depending on their business needs, companies can decide on the kind of package they want tobuy. If they would rather focus on specific departments, they can decide to buy packagedanalytical applications such as especially the finance department. On the other hand, they couldopt for integrated packages which include data management and integration, analytical tools andapplications and collaboration software. Companies can also decide to customize their businessintelligence software and elect to buy analytical development environment which allows them touse components to build applications for their needs.The overlapping functions of business intelligence and enterprise management have alsoattracted many different types of vendors from the enterprise resource planning, performancemanagement and business process management space besides best-of-breed innovators whoexcel in some segments of the process. Buyers, therefore, have to weigh the benefits of stayingwith their familiar ERP vendors who increasingly have the ability to package business intelligencepackages versus choosing the richer functionality of ‘pure-play’ business intelligence vendors orthe niche ‘best-of-breed’ innovators. While the increasing availability of integration tools, with theadvent of web services, XML and Java, afford the ability to conveniently extend ERP packages,management of a diversity of vendors does increase costs.The sum of the parts is higherBuyers of business intelligence software have to weigh the often conflicting needs ofstandardizing the source of their software to realize cost economies as against acquiringinnovative software from ‘best-of-breed’ companies who often do not have the ability to provide apackage of transaction data software, ETL and business intelligence tools. The quality ofmiddleware has vastly improved in recent years with the advent of web services but companiesstill prefer a solution from a single vendor as we will discuss later. While business transactiondatabase companies are increasingly incorporating business intelligence software, they are stillnot functionally as rich in business intelligence functions.At this point of time, the industry is divided between vendors who excel in implementation ofbusiness intelligence software, as assessed by Gartner, and many of them are cash-richcompanies from the mature segments of the enterprise software sector. On the other hand, thepure-play business intelligence software companies have stood out for their visions; they areemerging companies which are relatively weak in their capabilities to develop the infrastructurefor business intelligence. None of the companies have both the capabilities for users to be able tomake a clear choice of a vendor.The players from the traditional segments of the enterprise software industry, i.e., EnterpriseResource Planning, RDMS, Customer Relationship Management and Supply Chain Managementcompanies have added Business Intelligence functions in their offerings. They stress their proventrack record and familiarity while the ‘pure-play’ BI companies berate this ability as commonplacein an environment where web services facilitate convenient integration with existing applications.
  10. 10. According to estimates of Gartner, the users who decide to opt for ERP companies, who are alsoable to meet the application requirements of customers including BI, will incur costs that are 25%lower than those who decide on ‘best-of-breed’ or more specialized innovative companies by2005. The best of breed companies are more innovative and find new ways to compete but thefact remains that the older companies are either buying them or are able to incorporate or embedtheir software within their package and lower costs of servers and license fees.The merits of these contradictory claims can be judged from a recent survey of BI and DWprofessionals who are members of TDWI. The data from 552 BI and DW professionals indicatesthat 56% of them use BI solutions from their transaction database providers either exclusively orjointly with third-party solutions. On the other hand, 61% of them are using either third-partysolutions exclusively or in combination with the transaction database providers. Users are morelikely to prefer transaction database providers when they need to access information from theirsoftware and when the data is used for routine analytical functions like fraud detection andprofiles of customers for contact management. Also, they are more likely to use software fromtransaction database providers when they also provide the ETL solution as well saving them thejob of extracting the data. On the other hand, third-party solutions are likely to be preferred foradvanced analytics like predictive analysis or ad hoc queries. Also, companies have a lesscompelling need to stay with their transaction database providers when they already have legacysoftware installed and need to use integration technologies. In all, companies are more likely touse transaction database providers as they increasingly use BI for improving operationalmanagement as is the case today.The leap towards enterprise wide intelligenceBusinesses have to often choose between data marts implemented in some departments incontrast to the bolder alternative of investing in a data warehouse or an enterprise wide real timeintelligence infrastructure from the very beginning. A smaller data mart can be a test case and ismore likely to be accepted by a minority of technology buffs in an organization while a datawarehouse or enterprise intelligence is more efficient. If companies opt for a data warehouse,they will encourage standardization of data definitions and planning for business processmanagement across the enterprise which is not a pressing concern when data marts areimplemented in some divisions. Furthermore, data marts require a different kind of technologywhich is rendered obsolete later when enterprises decide to accept data warehouses orenterprise wide real time intelligence. Also, data marts, duplicate data which has to be cleanedlater when companies switch to data warehouses.One case where a company had to abandon its investments in a data mart is GE Real Estatewhich has investments in over 8,000 properties around the world. The management of such aportfolio required information that was often available in the individual countries where theoperating units were located. GE decided against the ideal solution of a web based data
  11. 11. warehouse solution primarily because reconciliation of a variety of data definitions used by itsdepartments seemed much too daunting. After the modest success of its data marts, GE had toentirely abandon its Microsoft server which could not scale for a data warehouse and had toinstead change to a UNIX server with an Oracle database.A more comprehensive survey of 150 technology executives, who have implemented businessperformance management (BPM) across the enterprise, shows that a step-wise process hasbeen more successful for most companies. Twenty percent of the respondents have implementedBPM and 60% of them implemented them for particular problems before expanding their scope tothe entire enterprise.The decision to buy software for a specific department or the company as a whole is not simply atechnical decision. While the more skilled or “power users” are avid consumers who relish newsoftware, the risk of provoking a more political response from other employees is much greaterespecially if they are either wedded to the tools they have used in the past (typically, Excel) or arenot sophisticated users of some arcane business intelligence software. The choice to use moreintegrated software should be preceded by finding a sponsor with the clout to implement newbusiness intelligence software. Also, companies have to find the means to customize the softwareaccording to the roles of each individual in the enterprise.On the other hand, the benefits from enterprise business intelligence yield the greatest benefit.Companies can more effectively align their strategies with the resources and tactics of theircompany. They can also monitor performance in real time to ensure that their actions are able toachieve their goals.Intelligence for all decisionsAnalytical software is not any longer a stand-alone application that is the preserve of power userswho have special skills to use arcane tools. Instead, analytical applications permeate theenterprise and they have to be adapted for users who are pre-occupied with solving businessproblems or just daily operations and do not want to bog down in mastering the technical nuancesof sophisticated tools. While power users require data exporting, cube and data modeling options,business users need the ability to manipulate data, at a granular level, locally in Excel. Casualusers prefer dashboards and canned reports.The business users are also interested in sharing information with their team members. Tools likeCrystal Reports and Actuate Corp.s e-reporting disseminate information to many casual users and high-level decision makers.Finally, analytical tools have to be seamlessly integrated with business process management aswell as tools for evaluation of results of actions taken based on the initial analysis. An example ofthis kind of software is Customer Power 5.0, a tool developed by NCR, which is targeted atcustomers in the retail, financial and catalog marketers. They are able to use customerinformation and analytics for improving the effectiveness of marketing campaigns. One clothing
  12. 12. company is able to use transaction information to create catalogs specially designed for its bestcustomers.Enterprise users look for tools that have an intuitive interface and are customized for theirindustry and functional area of expertise; generic analytical tools are less likely to meet theirneeds. The need to expand the functionality of analytical tools stretches the technical capability ofvendors. In the past, analytical software companies specialized in a limited range of functionalexpertise. Some companies like Hyperion excelled in performance management while others likeSAS and SPSS distinguished themselves in statistics. In their efforts to meet a more diverse setof needs of customers, analytical software vendors are now entering a risky terrain where theyhave to acquire companies to provide a portfolio of products including integration with businessprocess management software. Furthermore, they have to be able to gain domain knowledge of avariety of industries where they have customers. Finally, customer needs are evolving asknowledge workers in enterprises find ingenious ways to use information. It is unlikely that allcustomers will be able to find the entire range of functionality they need.An alternative possibility is that customers can buy a package of tools and have the ability to buildnew applications using development tools and a platform. Custom applications, developed bycustomers themselves, are more likely to take advantage of the domain knowledge than vendors.Customers would prefer component based application development with graphical tools so thatthey don’t have to invest inordinate amount of resources in programming. A typical product of thisnature which bundles a package of software tools and a development environment is Oracle’s10g which combines business analytic and reporting tools, Oracle Discoverer for queryinganalysis and dashboard features; Oracle Spreadsheet Add-In, which allows direct access toOracles online analytical processing (OLAP) from within Microsoft Excel spreadsheets; OracleWarehouse Builder with extract, transform and load (ETL) capabilities; and Oracle BI Beans, aset of custom developing tools.The advantage of custom applications is that they readily adapt to the existing infrastructure ofthe company including the data warehouse. In addition, companies have already implementedtheir data categories, dimensions and the data model and their customized applications will adaptbetter to this ensemble while packaged applications would require considerably greateradjustment within the enterprise including changes in the data warehouse.Weaving the information fabricThe latest research, based on feedback from users, indicates that ease of integration is the singlemost important criterion for selection of Business Intelligence software. More than 75% of theusers consider this to be the most important factor in the selection of BI software. The high levelof significance of integration is illustrated by the case of Alaska Airlines which preferred SiebelAnalytics for the ease of integration of the software in its heterogeneous environment especially
  13. 13. the data warehouse. The metadata that came with Siebel Business Analytics is especially usefulin the integration process.Data warehouses have been the lynchpin of strategic applications of business intelligence; theyenabled companies to coalesce information from disparate sources. In addition, the datawarehouse stores data only after it has been cleaned for inconsistencies and when it conforms tostandard definitions. Data is prepared for use in data warehouses by ETL (extraction,transformation and loading) tools which extract data from operational data stores, transform thedata so that data definitions are consistent and duplicated data is removed and the output isloaded into data warehouses in line with its metadata. The data warehouse updates the sameinformation. The downside of the data warehouse is that it can update information only whenanalytical functions are interrupted while data can be updated usually on weekends or beyondwork hours. As the volume of data grows, the time spent on loading increases to an extent whereit would conflict with data analysis functions. Also, real time analytics is really not feasible with thelatencies that are inescapable with the data warehouse.The alternatives to ETL technologies are the EAI (Enterprise Application Integration) and EII(Enterprise Information Integration) technologies which can access data from a variety ofsources. The EAI integrates applications and helps to access data from them. One example ofhow a company leverages such a technology is Virgin Mobile which needed to rapidly expand itsservices in the USA without sinking investments in an elaborate IT infrastructure. It struck anagreement to use Sprint’s operating infrastructure, for the phone service, which was integratedwith its CRM and financial software using process integration software.The problem with simple integration of applications is that their data definitions or the metadatamay not be the same so that it would be hard to extract data without standardizing theirdefinitions. On the other hand, EII helps to both integrate the applications and uses XML to matchthe taxonomy of the data extracted.For real time applications, BI software will have to work with both historical and currentinformation. Enterprise Information Integration software is able to draw information from both thedata warehouse as well as operational data stores in real time as transactions happen. When thedata is drawn directly from operational data stores, it is not cleansed of inconsistencies in datadefinitions or duplication. A BI tool, such as Cognos ReportNet, uses Composite Software’sComposite Information Server, to extract information from data repositories throughout anenterprise and create a single, more comprehensive data source.One application of the EII technology is the case of Owens Corning which combined its EIIsoftware and the BI software to generate reports on gross margins earned every day. The datawas extracted from its numerous ERP databases. In the past, the same exercise took about amonth. The reduction of the lag time in reporting has helped Owens to make mid-coursecorrections.
  14. 14. An increasingly preferred means of integrating software is the Services-Oriented Architecture(SOA) which reduces the monolithic applications of the past into components or services whichconstruct the larger applications. These services or applications are mounted on a single userweb based user interface. Users can choose the services they need instead of buying a packagewhich often has components that are not relevant for them. Each of these components are inter-connected by web services which use XML tags to build an interface which can be used toaccess services using web technology. Web services can be used to provide access to businessprocesses, applications, BI cubes, reports, queries and data integration functions, databases etc.The web services are indexed in a UDDI registry. In order to connect to a web service, anapplication or portal simply queries the UDDI registry, finds the service and then dynamicallyconnects to it by sending it an XML message known as a SOAP (Simple Object Access Protocol)message. The Service Oriented architecture is a more cost effective since it does not require ITpeople to inter-connect applications.An example of a SOA implementation is Siebel 7.5.3, released in mid-2003, which is expected tocrystallize with the Siebel 8.0, due in 2006. The complete Web-services-oriented software willinclude workflow management tools.The cross-point of all dataAt the heart of efforts at integration is the task of creating taxonomy and metadata or descriptionsof data which helps to describe data across any application, repository or database. Mostmetadata is sui generis and hard to relate to other descriptions of data associated with anotherapplication.Data is fragmented for a variety of reasons. Structured and unstructured data is hard to correlatebecause the former is stored in a relational data base and the latter in a content repository.Content itself is divided between document, web and digital asset management repositories. It ishard to find related structured and unstructured information in a relational database and a contentrepository because the former uses SQL to extract data and the latter uses a search engine. Thedescriptions of the data vary depending on the function of the person who is using it. A marketingperson is concerned about customer acquisition while a sales person is interested in prospects,orders and closures. The information systems they use will differ; a marketing person will use aCRM system while a sales person will typically use Both will enter customerreference data often with different conventions about recording first names and second names,address entries, etc. When the units of a company are spread across the world, each nationalsubsidiary uses a different language which often means they also use another informationsystem.Increasingly, companies need to be able to correlate information. They need to be able to seeinter-related information about customer purchases and their behavioral traits which are bestdescribed in text. Similarly, they need to correlate information about customer acquired and the
  15. 15. sales earned from them. They also want to share content created in one subsidiary with anotherin another country and translate and use it rapidly.A real time enterprise needs not only gain access to all this information in a cohesive manner butalso be able to do this quickly. The advent of XML has considerably simplified the process ofcorrelating disparate information by described its components at a more granular level with thehelp of tags. Once content descriptions are available at this level of detail, some extracts can bereused rapidly. With the use of XML, it is now possible to search simultaneously in relationaldatabases as well as content repositories. Companies are now able to translate content in arepository in one country for use by another country without requiring the intervention of skilledtranslators.MacDonald is a classic instance of a company that operates across national boundaries and hasto work with repositories located in several different countries. In addition, it has numerousfranchises working independently and loose time when they recreate the same content. In orderto collate information from its subsidiaries, MacDonald has to be able to translate the contentfrom several different languages and draw them from several different information systems. Italso benefits from centralizing content creation and redistributes it to local units where it can becustomized. All the content can be accessed from a portal.Golden Gate Software’s Data Synchronization solution is an example which helps to use datafrom disparate sources in real time. It has been used in the health care industry by hospitals tocollate information from individual hospitals and patient care centers in a single repository. Withsynchronized clinical and administrative data, one of its customers is able to produce detailedanalytic reports to track patient metrics, length of stay, and other care management data.CONSIDERATIONS ABOUT THE FUTURERFIDRadio Frequency Identification is used to identify, track and sort objects. A tag is inserted in anobject for transmitting information about the product identification; the data is transmitted by radiowaves and received in a digital form for storing in computers. The tag is a tiny antenna with theability to send radio signals to a reader or a mechanism that is able to receive radio signals. RFIDis presumed to have potential applications in supply chain management which is expected togenerate enormous data for BI analysis purpose even though applications development isuncertain.Wal-Mart’s declared its plans to use RFID with much fanfare but the effort petered off after thesuppliers balked at the idea. The aggregation of the RFID data will be a complex task ofintegration as the data will be received from a plethora of often small companies involved inlogistics management.Yet, some companies have found ways to use RFID data to their advantage. Graniterock, aWatsonville, California based company, has installed a BI system to monitor time spent by trucks
  16. 16. in its quarries. The clerical tasks of identifying incoming and outgoing trucks don’t need to bedone any more, as their movements are recorded electronically, so that Graniterock saves time.The customers benefit as they are able to monitor the costs they incur on the time spent byindividual trucks and the relative efficiency of each company.RFID has potentially profitable applications in several different industries. Tracking baggage inthe airlines industry is an enormous cost and pioneers like Delta Airlines is already using RFID totrace lost baggage. The drug industry also has an intractable problem of controlling the diversionof drugs for intoxication.Complex Business Event MonitoringInvestment in Information Technology infrastructure creates new possibilities. CRM paved theway for business analytics. In turn, business analytics afforded opportunities for event basedbusiness activity monitoring. Even as companies are implementing simple event based businessactivity monitoring technology, conditions have been created for more complex event basedbusiness activity monitoring which enables decision making including pre-emptive actions, basedon predictions, which help to avert a crisis or an adverse situation from happening.An example of more advanced event based monitoring is the action that can be taken inresponse to news feeds in the financial services industry. A simple event based business activitymonitoring would be a stop-loss routine which triggers a sale whenever the price of a stock fallsby a given percentage amount. A relatively more complex business activity monitoring would beto monitor trading activity to determine whether the patterns in bids by traders in an auctionsuggest that they are colluding and are violating SEC regulations. This could go further and rulescould be written which help to predict the future performance of the stock and investmentdecisions are triggered when expected events actually occur.The use of complex event monitoring is expected to rise with the use of RFIDs when a lot moredata is expected to be gathered. Several different applications are possible with the datagathered from the scanning of the codes embedded into products. The obvious application ofRFIDs is to monitor trends in inventory depletion. The more important applications would be tomonitor the time of delivery by different transportation companies which could be affected by theconditions along the route, the quality of maintenance of vehicles or by the driver’s efficiency.Companies can begin to analyze the data and find better means for routing, loading andunloading techniques as well as data on congestion along the way.A recurring issue with retailers is out-of-stock items (OOS) which lead to lost sales opportunities.According to a study conducted by Andersen Consulting, 53% of OOS situations happen as aresult of poor stocking decisions. Another 8% of OOS situations happen as a result of lags inmoving inventory from the warehouse to the shelves. Some pioneers, like U.K. supermarket chainTesco, have taken the lead in adopting RFID technology to monitor shipments of high-valuenonfood goods to avoid OSS.