I D C            M A R K E T                        S P O T L I G H TUnified Information Access: Bridging the SilosDecembe...
If organizations are to avoid the risks inherent in too much scattered information, and if their workersare to perform eff...
Currently, IDC sees unified information access platforms commonly used for the following types ofapplications:   Quick-ch...
   Recognition of the value of information aggregation — having all related information in one place    spurs understandi...
As Figure 1 illustrates, unified information access platforms offer a lot more capability than traditionalsearch or busine...
predictive analytics model when mobile phone customers are likely to churn, enabling a mobileoperator to offer special pro...
When handling rich media, especially audio, organizations should look for technologies that canprovide speech and image re...
    Finance: Fraud detection, voice of the customer, upsell and cross-sell with predictive analytics,     trend spotting ...
Upcoming SlideShare
Loading in...5

IDC Market Spotlight: Uniformed Information Access: Bridging the Silos


Published on

HP sponsored this Market Spotlight that explores the trends associated with information management and discusses the importance of unified information access platforms as organizations contend with big data management issues.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

IDC Market Spotlight: Uniformed Information Access: Bridging the Silos

  1. 1. I D C M A R K E T S P O T L I G H TUnified Information Access: Bridging the SilosDecember 2012Adapted from Unified Access to Information: Less Seeking, More Finding by Susan Feldman andHadley Reynolds, IDC #227780, and Worldwide Search and Discovery Software 2012–2016 Forecastby David Schubmehl, IDC #235480Sponsored by HPInformation is the fuel that powers todays organizations, and it comes in many forms, including textfiles, databases, social media, rich media, click streams, sensor data, images, email, and more.Over 85% of this information is unstructured or semistructured, including email, documents,Web pages, and audio, image, and video files. Effectively handling the waves of information thatwash over and through organizations is at the same time an opportunity and an ever-increasing risk.An integrated approach to capturing, finding, managing, retaining, and delivering information isimperative if an enterprise is to protect its central business processes while still encouragingappropriate use of information both inside and outside the organization.Increasingly, organizations are finding it difficult to manage the wide variety and the sheer volume ofdata, but they realize that by not doing so, they are missing opportunities and insights based on newanalytics capabilities. New unified information access platforms are emerging to help solve theproblem of scattered and diverse information. Unified information access platforms have begun tosurface as technologies that go beyond traditional search systems to collect, aggregate, and connectinformation across multiple heterogeneous sources of unstructured data; these technologies deliverinformation to users and assist them in deriving meaning from it.Big data adds a new dimension to information management. In response to the unique demandspresented by big data, organizations are reconsidering their information access and analysiscapabilities. The big data explosion is a huge driver propelling the need for unified informationaccess. IDC believes that the time is ripe for an investment cycle in better information access andanalysis. External market forces such as the requirements for understanding customers, forautomating knowledge work, for faster time to market, for better risk management and compliance,and for increased revenue and decreased costs all point to investments in new informationmanagement and analysis technologies.This Market Spotlight explores the trends associated with information management and discusses theimportance of unified information access platforms as organizations contend with big datamanagement issues.IntroductionThe extreme difficulty users have in integrating information from digital sources to complete tasks andto make decisions has driven businesses to the point of diminishing returns in their investments ininformation technology. The ever-increasing numbers of point solutions designed to digitize virtuallyevery enterprise business process, the proliferation of platforms for infrastructure and for applications,the increasing use of digital collaboration and social software, and the tremendous stream of text,data, and rich media from Internet sources present difficult challenges for IT departments today.IDC 1435
  2. 2. If organizations are to avoid the risks inherent in too much scattered information, and if their workersare to perform efficiently, IT must solve users issues regarding handling the diversity and volume ofinformation ranging from internal memos and email to audio and video. In essence, this is the humanfactors dilemma of this decade. The productivity of an organizations workforce and the timelinessand accuracy of business decisions are affected directly by the inability of traditional systems to offera unified information access environment, particularly now as organizations contend with a widevariety of data types and data sources as defined by big data.At the same time, newer generations of information access systems have evolved to meet theaforementioned challenges. Unified information access platforms have been evolving in the searchworld for some time. The convergence of demand and available products has launched a newgeneration of information access software that differs fundamentally from its database/businessintelligence (BI) and search progenitors.Unified information access platforms provide a single point of access to multiple heterogeneoussources of information, including text, structured data, audio files (including voicemail), and other richmedia. These platforms are highly scalable and are built to accommodate quickly changinginformation through real-time or near-real-time updating and analytics. These platforms combineelements of structured/unstructured data integration, database, analytics, BI, and search anddiscovery technologies to make information access intelligent, secure, dynamic, and immediate forbusiness users.Unified information access platforms are capable of handling, indexing, and integrating large volumesof unstructured, semistructured, and structured information in a wide variety of forms into a unifiedenvironment for information gathering, analysis, and decision support. Like portals and searchapplications, unified information access platforms offer rich and interactive user interface designs.Like BI applications, they integrate visual analysis tools. These platforms promise to boostinformation worker productivity because they enable ad hoc queries and quick, easy access to bothdata and content for business decision makers as well as trained data analysts. Capabilities such asautomatic categorization, metadata enrichment, and guided navigation give the business user simpleyet quick ways to locate and relate information from a wide variety of sources. Unified informationaccess platforms provide the infrastructure for building search-based applications, or InfoApps,IDCs term for a new category of software applications that provide an integrated workspace that isdedicated to a specific task or workflow.The market for unified information access solutions is just beginning to evolve in conjunction with thegrowth of unstructured information available to organizations internally and externally via Web-basedsocial media and traditional media. Unified information access platforms form the foundation forInfoApps. These applications provide a unified view of a particular domain, subject, or topic. InfoAppsare typically built on a unified information access and management platform to create comfortableworkspaces that: Are tailored to fit a specific task or workflow Combine multiple technologies and tools, particularly search, collaboration, authoring tools, content management, and analytics Integrate information from multiple sources Incorporate domain- and organization-specific term lists, taxonomies, and knowledge bases Hide technical complexity below an easy, compelling UI that may have dashboard-like qualities 2 ©2012 IDC
  3. 3. Currently, IDC sees unified information access platforms commonly used for the following types ofapplications: Quick-changing data, and for ad hoc access to current data, often in real time or near real time (Marketers, business developers, product managers, and customer service managers gain an instant snapshot of their business across internal and external structured and unstructured sources.) Litigation and eDiscovery support within organizations developing an internal early case assessment (ECA) strategy Better customer understanding and service, by merging external social media information streams such as Twitter or Facebook with internal customer records Unified view of patients in healthcare Faster decision support and strategic decision making Dynamic data integration interfaces (e.g., on-the-fly generation of facets for browsing and/or clusters to suggest information themes of interest) Sifting through high-volume streams of heterogeneous data from multiple sources and finding relationships among the information fragments in government intelligence applications Making projects in consulting and professional services firms run more efficiently by finding the most recent boilerplate for contracts and proposals, by locating appropriate experts based on their profiles, and by locating all pertinent information, no matter where it is stored Combining geolocation data with unstructured and structured sources to locate retail establishments, improve logistics applications for trucking or car rental companies, or help sales and marketing analyze the state of their businesses Government situational awareness surveillance applications requiring a combination of multiple sources of structured, unstructured, and geolocation data, such as intelligence gathering and evidence support using text, audio, and even raw videoIDC believes that these new unified information access platforms are gradually replacing featurespreviously found separately in traditional data integration, search, digital asset management (DAM),and BI software with a single information access platform that does not distinguish betweeninformation types by either source or format. Many of these platforms already have moved beyondintegrating information because they also integrate the tools as interdependent modules.There are major technical challenges to merging these information types, yet the need for moving toa cross-enterprise approach to information access has begun to drive adoption. Organizations arebeginning to see the benefits of adopting a unified information architecture, including lower costs,increased ROI, and quicker user adoption, as well as some of the business benefits noted previously.A number of recent developments have spurred interest in and adoption of unified information accessplatforms: The continuing explosion of data quantities inside the enterprise and the understanding that there are nuggets of valuable information that need to be discovered within the information repositories Compliance requirements and the cost of litigation, especially as they relate to dealing with rich media, which has traditionally been ignored but is fast becoming an important data type in litigation©2012 IDC 3
  4. 4.  Recognition of the value of information aggregation — having all related information in one place spurs understanding of trends and relationships (The time saved in having a single access point to all information sources is also significant.) The rise of social media as a new source for business analytics, customer relationships (including "voice of the customer"), and support, marketing, and advertising programs Requirements for companies to combine internal data with extensive external data to derive analyticsDefinitionThe following criteria distinguish unified information access platforms from their search and BIpredecessors. Unified access platforms: Provide a single point of access to a wide variety of sources and types of structured and unstructured information that can surface relationships across data sources, including rich media types such as audio and video Combine structured data and data operators with text and semistructured operations and analytics within a single architecture, which is not dependent on predefined schemas or federated operations (These platforms typically include tools for semantic understanding, including fuzzy matching and a range of search and text analytics routines, as well as structured data operations.) Interpret queries appropriately for each type of data and then merge and analyze the results to uncover relationships across sources (e.g., bring together all of the information about a customer from transactions, emails, and customer call records) Provide appropriate tools to prepare, merge, analyze, and present information from multiple sources Process queries faster and more flexibly than traditional database and/or BI applications, although they often depend on these repositories as sources Scale to petabytes of information or billions of key value pairs and, depending on the requirements of the solution, offer incremental updates in near real time Provide situational awareness and context by linking related information "nuggets" together across disparate sets of dataIn addition, unified information access platforms include the following features: Flexible "model as you go" data preparation tools that apply to both text and data sources, are somewhat automated (beyond traditional ROLAP or data modeling tools), and dont require normalization to a single schema Smart connectors to major information sources that can recognize new data and help normalize information Presentation and reporting tools that are BI-like and that feature visualizations as well as tabular data or lists of references 4 ©2012 IDC
  5. 5. As Figure 1 illustrates, unified information access platforms offer a lot more capability than traditionalsearch or business intelligence systems. One of the real keys for unified information access platformsis "ETL for Text/Media," where knowledge from unstructured information is extracted, normalized,and fed into knowledge bases and information warehouses. Figure 1 Unified Information Access Architecture Reference Diagram Applications InfoApps UI, Dialogue Manager Output Workflow P Query Converters l Browse/Navigation Visualization Message Distribution CEP a Text Mining Data Mining Search Query Analytics Rules BPMS t Data Store, Data Warehouse, Search Index, Content Repository, Rule Base, f Model Manager, Knowledge Base o Input Workflow: Pre-processing ETL for Text/Media ETL for Data Real-Time Streaming r Tokenize, normalize, extract entities, relationships, time, sentiment, categorize Cleanse, normalize, extract, transform, load Enterprise Service Bus, Message-Oriented Middleware m Smart Connectors Unstructured Content Semistructured Data Structured Data Source: IDC, 2012In a larger sense, unified information access applications are hybrid applications. They combinetraditional search, knowledge management, and informational awareness with workflows that canprovide additional functionality such as adding or modifying content, real-time collaboration, or eventaking actions based on the unified information provided.For example, a customer service agent in a bank can use a unified information access application tosee a holistic view of a customer, including loan applications, credit history, bank accounts, and evensocial media posts, in order to act upon customer requests with full knowledge of the customers pastinteractions with the bank. Another example is a 911 dispatcher alerting the authorities to amultihouse fire near a street intersection. Live video from the video camera above the traffic light iscombined with audio from several callers, tweets from passersby, and information on the houses fromthe citys real property database. All of this information is fed to the fire chief on the scene, includingdemographic records of who lives in the houses to ensure that everyone is accounted for. Yetanother example is that of a voice of the customer application that combines mobile phone call data,service records, and social media (Twitter and Facebook posts) to determine and predict from a©2012 IDC 5
  6. 6. predictive analytics model when mobile phone customers are likely to churn, enabling a mobileoperator to offer special programs to retain existing customers or to attract new customers.There are also examples in research and development such as combining research, lab notebooks,patent filings, and other types of scientific information to create next-generation knowledgemanagement and sharing applications or intellectual property management solutions such as whitespace analysis to determine where to invest research dollars.BenefitsUnified information access platforms enable organizations to realize a number of benefits, includingthe following: Accommodate rapid increases in information types and volume Ensure that information is available when needed and that the source of the information is transparent to the user Minimize the administrative burden on IT and business organizations Deliver security and data protection the information needs in order to be available, accessible, and secure Allow organizations to extract relevant information nuggets from their untapped unstructured information stores and leverage that information in their decision-making process Transform vast amounts and varieties of data into actionable insight based on content and business analyticsUnified information access platforms can be combined with other systems, such as contentmanagement and workflow applications, to provide a closed loop process where information can befound, examined, analyzed, and then acted upon. In this type of scenario, unified information accesscan be seen as a focal point of evidentiary information for decision management systems, early caseassessment systems, or product life-cycle management systems.ConsiderationsThe unified information access market is still maturing. Several companies that offer traditionalenterprise search are moving in this direction, and a host of smaller companies are moving into thisarea as well. Thus, buyers need to closely consider varying levels of capabilities and offerings beforemaking a purchase, including: Maturity of the software stack being deployed Ability to handle the wide variety of data types, especially audio and video files Ability to scale beyond departmental solutions to enterprisewide applications, including the ability to successfully ingest and handle big dataThe development of unified information access solutions will require services to be properlyimplemented. A ratio of 1:1 or even 2:1 for services versus software license would not be unusual.Organizations should try to implement departmental point solutions where the initial ROI is greatest inorder to demonstrate the value of the technology. They should look to areas such as litigation supportor customer service where savings or benefits can be easily identified. 6 ©2012 IDC
  7. 7. When handling rich media, especially audio, organizations should look for technologies that canprovide speech and image recognition as part of the offering because this capability will provide a richlevel of metadata for indexing and later search.Market TrendsThere are a number of paths to implementing unified information access today, and they vary widelydepending on the vendor and the use case. As the data and content worlds have converged, bothsearch and BI vendors have acquired significant pieces of the other, parallel universe. Depending onthe vendor, the slant and combination of tools in the unified information access platform will differ.IDC does not believe that these new platform offerings will usurp existing search or BI/data warehouseapplications; however, it is clear that new purchases of information access software increasingly requirethe ease of use of search combined with the analytic and reporting capabilities of BI.According to IDC research, there is an increase in applications built on unified information accessplatforms, serving as a "front end" to legacy applications. These applications take in queries, modifythe queries for target source applications, and then merge the results for the user.In many cases, these types of systems will become the next generation of knowledge sharing andmanagement applications within organizations. As such, these systems will unify legacy content siloswithin a task and workflow–based set of applications that allow users to locate and use legacyinformation as well as current information and to determine relationships and connections in order toadvance decision making.ConclusionMany of todays popular technologies address only a subset of relevant information, whether inside oroutside the enterprise. Organizations face growing demands to monitor, mine, analyze, andunderstand complex information-based business processes. Newer approaches and technologies arebreaking down the barriers between these current technologies. IDC data shows that adoption ofunified information access is accelerating. Users dont care how information is packaged and where itcomes from; they just want the information they need, when and where they need it. They want itpromptly and in one place. Vendors that can deliver on the promise of unified information accessplatforms will provide leadership in a new and fast-growing segment in enterprise software.Organizations that deliver these types of solutions to their users will see increased value from theinformation that they have accumulated and because of this will be able to distance themselves fromtheir competition.Search and discovery (including unstructured data analytics) software and technologies are thefoundation for an entire spectrum of new, innovative solutions that are arriving on the market today.These solutions help users and businesses address the unsolved problems they face in attempting tofind and make sense of the embarrassment of riches that digital information has become.As noted previously, unified information access will be used for a wide variety of use cases, including: Publishing: Smart content Manufacturing: Voice of the customer, social media monitoring Marketing and politics: Social media monitoring Government: Fraud and terrorist detection; voice of the citizen, emergency event monitoring Healthcare: Diagnosis support, predict recurrence of hospital visits, fraudulent claim detection©2012 IDC 7
  8. 8.  Finance: Fraud detection, voice of the customer, upsell and cross-sell with predictive analytics, trend spotting for investment bankers Pharmaceuticals: Drug discovery, cause and effect for treatments, finding influencersIDC expects that the search and discovery market will continue to grow at annual rates in excessof 5% that are comparable to or somewhat exceed overall enterprise software market growththrough 2016. However, the growth for unified information access platforms will exceed this ratesubstantially. In addition, the current market has already begun to diversify, embedding full textindexing, rich media handling, content analytics, automatic categorization, and fact extraction inInfoApps (search-based applications), in big data applications, in reputation monitoring applications,and, especially, in unified information access platforms. Search and content analytics are also anecessary component in a new generation of decision support applications. These new uses willensure the vitality of the market for search and text analytics software that is central to unifiedinformation access platforms.A B O U T T H I S P U B L I C A T I O NThis publication was produced by IDC Go-to-Market Services. The opinion, analysis, and research results presented hereinare drawn from more detailed research and analysis independently conducted and published by IDC, unless specific vendorsponsorship is noted. IDC Go-to-Market Services makes IDC content available in a wide range of formats for distribution byvarious companies. A license to distribute IDC content does not imply endorsement of or opinion about the licensee.C O P Y R I G H T A N D R E S T R I C T I O N SAny IDC information or reference to IDC that is to be used in advertising, press releases, or promotional materials requiresprior written approval from IDC. For permission requests, contact the GMS information line at 508-988-7610 or gms@idc.com.Translation and/or localization of this document requires an additional license from IDC.For more information on IDC, visit www.idc.com. For more information on IDC GMS, visit www.idc.com/gms.Global Headquarters: 5 Speen Street Framingham, MA 01701 USA P.508.872.8200 F.508.935.4015 www.idc.com 8 ©2012 IDC