Big Data’s Big Impact on Businesses


Published on

The web-conference hosted by CRISIL Global Research & Analytics on “Big Data’s Big Impact on Businesses” on January 29, 2013, saw participation from senior officials of global multinationals from 9 countries. The presentation described how data analytics is helping businesses make “evidence-based” decisions, thereby creating a positive impact. It also spoke about the opportunities opening up in the Big Data space in India and across the globe.
Hosted by:
Sanjeev Sinha, President, CRISIL Global Research & Analytics
Gaurav Dua, Director & Practice Leader (Technology, Media & Telecom), CRISIL Global Research & Analytics

Published in: Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Companies worldwide are turning their attention to Big Data as they scramble to derive insights from the deluge of information generated from various sources. In the past few years, the global marketplace has seen exponential growth in data volumes, created and consumed by a diverse cross-section of stakeholders. The term “Big Data” signifies large data sets in multiple formats, growing at an enormous rate and posing problems for traditional storage and analytical platforms. Big Data is distinct from large existing data stored in various relational databases, as it warrants a more advanced mechanism for both storage and analysis. Technologies such as NoSQL databases and MapReduce/Hadoop frameworks are at the core of the solutions heralding a paradigm shift. So Big Data is characterized by three attributes of data: volume, variety and the velocity at which it is generated.Traditional analytics on transactional or structured data have helped data-driven organizations gain insights from various enterprise data. As data from weblogs, social media posts, sensors, images, e-mails, audio and video files emerge as sources of insights, it presents a huge competitive opportunity for businesses. The need to derive predictive and actionable insights from this data for improved business operations and better decision-making is what drives Big Data analytics.
  • Data volume is the primary characteristic of Big Data. With data becoming an indispensable part of every economy, industry, organization, business function and individual, it is being actively captured by companies to better understand their customers, suppliers, partners and operations. Large data sets yield more information and hence improved analysis compared to limited records of data, leading to better competitive advantage and business operations. This data is being generated at a rapid pace: around 2.5 billion GB of data is generated every day, and more than 90% of the data available today has been created in the past 3-4 years. According to IDC, data generated globally is expected to witness a 41.0% CAGR between 2009 and 2020 to reach 35.0 zetabytes. Moreover, the technological landscape has changed with innovation in both managing and storing large data. As companies move away from the traditional data storage systems such as file systems and databases to newer technologies such as cloud-based storage and open source software, data storage and management costs are seeing a downward trend. According to IDC, storage costs have plummeted from US$18.9/gigabyte in 2005 to US$1.6/gigabyte in 2011, and are expected to further decline to 0.7/gigabyte by 2015. Apart from storage costs, the evolution of several open-source analytical tools and platforms has made data analytics flexible, reliable and relatively affordable for Big Data.
  • Organizations worldwide are increasingly realizing that unstructured data, if analyzed, can provide a competitive edge. While structured data is transactional and can be stored in rows and columns with an identifiable structure, unstructured data such as audio, video, and social media messages is raw or semi-structured. This data is generated in several forms such as web clicks, e-mails, phone conversations, weather data, audio and video files, location co-ordinates and pictures. Moreover, unstructured data is highly dynamic and does not have a particular format, i.e., it may be in different languages, have several terminologies, and may exist in the form of x-ray sheets, voice mails, digital photographs, or phone conversations.Companies are overwhelmed by the volume of unstructured data and are looking at ways to manage and analyze them in a systematic manner. As a result, one of the key focus areas for companies wanting to leverage Big Data is to handle unstructured data and adopt new technologies to deal with them.It is imperative to develop technologies that can enable storage of such huge data as well as maintain transactional consistency between structured and unstructured data. Newer technologies such as NoSQL databases to store unstructured data and processing methods such as Hadoop and massively parallel processing are gaining prominence in the area of Big Data and Big Data analytics.
  • The proliferation of the Internet and the mobile era has increased the rate at which data is created and stored; hence, there is a need for tools and technologies to analyze data at an equal speed. The shelf life of data has dropped from months to hours and seconds. The ubiquitous nature of the Internet, coupled with massive computing power and accessibility, has transformed data processing from an auxiliary function into an essential mechanism that enables organizations to transform their businesses. Big Data service providers are increasingly leveraging technologies such as streaming processing and in-memory computing that mitigates the shortcomings of batch processing and enable faster storage and data processing.Earlier, these technologies were popular in verticals considered more critical, such as the financial and government sectors. However, as the criticality of analyzing data in real time emerges, several other industries are also adopting solutions based on these technologies.
  • Big Data analytics is an evolving and multifaceted area for analytics players. The key differentiating factors between traditional analytics, advanced analytics and Big Data analytics are:Big Data analytics differs from advanced analytics in terms of different data formats and structures, and new application requirements for Big Data.While traditional analytics performs rear-view analysis on structured data, advanced analytics and Big Data analytics provide a progressive view, enabling organizations to anticipate and deal with future opportunities i.e. Big Data analytics has a definitive predictive end-result in its use.Big Data analytics has enabled cross-channel analytics and real-time insights at greater speed, access and collaboration. For example, detection of consumer emotions on a call on mentioning a competitor or conversion of a service call into an opportunity by leveraging Big Data analytics are more relevant in real time rather than after the interaction ends.
  • The Big Data ecosystem includes multiple elements from the data that is analyzed using the IT infrastructure that supports it and the applications that enable its analysis and usage. Elements of Big Data include:Data Management refers to systems where the data resides. It comprises the legacy systems as well as Hadoop-based systems and NoSQL databases. Legacy systems include databases that store and manage structured data, i.e., RDBMS to store and analyze structured data, and MPP systems to scale up for large structured data sets. Hadoop is an open-source software framework to support applications that enable analysis of petabyte- and xetabyte-sized data. Given Hadoop’s popularity and wide adoption, several other open-source projects have become associated with it, adding new functionality and enterprise-ready features to make it a compelling enterprise solution. These sub-projects include Hadoop Distributed File System (HDFS), Hbase, Hive, Mahout, Pig, ZooKeeper, Avro, Cassandra, and Chukwa. Once Big Data is collected and processed, it becomes operational data, i.e., it represents Big Data outcomes or serves as an input data for analytics. Big Data Analytics includes the technologies and tools to analyze the operational data and generate insight from it. After the data is analyzed, it becomes available for business users through various visualization techniques.Data Consumption involves enabling the Big Data insights to work in Business Intelligence (BI) and end-user applications IT Services enable integration of Big Data framework with the traditional business intelligence infrastructure.
  • North America and Europe, the two major data hubs of the world, account for a substantial portion of the global demand potential for Big Data analytics. Big Data service providers and leading IT players have significantly ramped up their capabilities in these developed regions that embraced the concept of Big Data, particularly in data-intensive industries such as digital media, manufacturing, healthcare, retail and financial services. While North America and Europe are poised to drive the growth of Big Data for the next two-three years, developing economies such as India and China are expected to catch up soon riding high on the rapid expansion of multimedia content, increasing popularity of social media, and proliferation of mobile devices. Further, while developed economies are likely to continue to be the major Big Data contributors in terms of revenue opportunity, emerging economies, particularly India, are all set to emerge as the preferred Big Data analytics and associated IT service providers.
  • Tools and technologies required to manage and analyze Big Data present a growth opportunity for startups to innovate and come up with new products. New companies across the Big Data technology stack have been thriving on the back of some robust investments anticipated in the Big Data space. The centerpiece of Big Data technology innovation, the Hadoop distribution, has been put to commercial use by many startups such as Cloudera, HortonWorks, Zettaset, and MaPR, with some customization of the open source software. Furthermore, the business environment is witnessing a slew of startups in the non-Hadoop systems such as NoSQL, Next Generation (MPP) Data Warehousing like CouchBase, Splunk, and VoltDB. The industry also has many startups emerging in the analytics platforms and cloud-based applications as well as in the advanced data visualization space. While the past two-three years have mainly seen new companies coming up in the data management space, analytics applications is the impetus for growth in the next few years. Some of the startups in this field include Karmasphere, Kognitio, 1010Data, Revolution Analytics, and QlikView.The Big Data technology space is witnessing a lot of venture capital activity, with funding in Big Data startups reaching ~USD 2.5 billion in 2011, compared with ~USD 1.5 billion in 2010. These startups are innovation hubs that are gaining importance across industry verticals. Most of these companies are witnessing high-double-digit revenue growth driven by the huge demand for their solutions. Moreover, many startups are being acquired by larger IT players given the growth opportunities and the need to build Big Data capabilities. For instance, IBM has acquired Tealeaf Technologies, Vivisimo and Varicent; Teradata acquired eCircle, and EMC2 acquired Greenplum.
  • The Big Data space is witnessing a string of M&A driven by the need to quickly ramp up capabilities and also to have a complete set of capabilities to service clients who are keen to have Big Data implementation. Leading technology players such as Oracle, IBM, SAP, and EMC are aggressively acquiring smaller independent software vendors (ISVs) and data analytics firms to strengthen their Big Data portfolio. IBM is in the forefront of this phenomenon through multiple acquisitions over 2010-12 in the Big Data space. It acquired Vivisimo and TeaLeaf Technology in 2012, i2 Limited in 2011 and Coremetrics and Netezza Corporation in 2010, for bolstering its Big Data capabilities. Further, HP acquired Autonomy for more than USD 10 billion, making it the largest deal in the Big Data industry. HP aims to cater to the Big Data market by leveraging Autonomy’s pattern-matching technology that recognizes and processes Big Data.
  • Financial services is considered to be a very data-intensive sector, with more data per million of revenue/operating expenditure or per employee, than almost all other sectors. Within the sector, structured and unstructured data is available from a variety of sources such as customer and transaction data from various channels such as branch, kiosks, mobile and web; social media; emails; credit cards data; insurance claims data; stock market data; statistical data, PDF & excel files, news, videos, and government filings. With the industry facing a multitude of challenges such as higher customer expectations, uncertain operating environment, strict regulations, stiff competition, and slowing economic growth, Big Data analytics can help banks, capital markets and insurance companies by providing tools to reduce costs and improve productivity. Increasing regulatory compliances and the need for collecting every piece of data and standardizing them is driving the growth of Big Data analytics. Several areas within the financial services sector are expected to gain from Big Data technologies. They include:Banking:Credit reward program analysis: Banks are increasingly using unstructured data to understand customer profile and introduce successfulcredit cards with innovative rewards programFor E.g. A national bank used a Big Data solution to analyze data from sources such as call centers, customer service emails, and social media conversations to create a credit card offering with a rewards program to attract a young, professional demographic. This helped in providing information to the marketing department to create a targeted promotion campaign, including strategically placed social messaging and monitoring.Capital Markets:Trading surveillance: The financial sector leverages Big Data to monitor trading activities and identify abnormal trading patterns. In surveillance, Big Data analytics allow on-line access to trade-by-trade history for investigation, trending, and discovery to be combined with real-time data to provide a real-time and historical context to behavior.For E.g. Companies combine data about the parties that participate in a trade with the complex data that describes relationships among those parties and how they interact with one another. The combination allows the bank to recognize unusual trading activity and to flag it for review.Insurance:Insurance companies are increasingly using unstructured data to predict client longevity, along with examining the prospective client’s medical status by analyzing their general comments, visits to particular websites, and enquiry about some specific products.Using weather and calamity information for managing claims exposures and losses based on unstructured data from weather measurements, and soil observations.E.g. An insurance company sells Total Weather Insurance, which pays local farmers when they are impacted by weather events that affect their profits. The company uses a cloud-driven Big Data analytics service to predict the possibility of extreme weather, along with the potential impact. It prices its insurance policies accordingly, based on 2.5 million daily weather measurements, 150 billion soil observations, and 10 trillion scenario data points to build and price their products.Big Data is being extensively used across all domains of the financial services for risk management, fraud detection, compliance and customer relationship management:Risk management: Predictive modeling of customer behavior and scoring techniques enable financial sector companies to access and minimize default risks at an individual level and make customized offerings, in line with the customer’s risk profile. E.g. A large bank wanted to use 12 years of monthly account-level credit card data, credit bureau information and bank account information to better assess the risk before granting loans or raising credit limits. Ideally, it wanted this information in real time. To speed the computing, it used an in-database Big Data approach, which helped the bank to calculate risk 70 times faster.Fraud detection: Big Data technologies give financial service companies the ability to run exploratory modeling and discovery on data, thereby increasing the accuracy of fraud detection models. The faster processing capability enables companies to quickly build or refresh fraud detection models, and also helps in detecting fraud in real time by analyzing and streaming transaction data.Compliance and regulatory reporting: Increased oversight and scrutiny of the companies’ operations, funding and investment portfolio has led financial services companies to adopt sophisticated Big Data technologies to store and process vast amount of data to simplify and streamline their regulatory and compliance reporting.For E.g. Reserve Bank of India (RBI) has directed all Indian banks to standardize their regulatory reporting by following an automated data flow (ADF) approach – to ensure 100% accuracy and zero human intervention in every stage of reporting: right from data extraction from source systems to the actual submission of returns. Firms that could not utilize complete information and firms that believed reporting did not really require management attention are increasingly focusing on Big Data analytics.Customer relation management: Big Data analytics also help financial service companies in acquiring new customers and cross-selling their offerings to existing customers by using Big Data to identify the most profitable customers and run effective marketing campaigns. The large volume of unstructured data from social media is combined with the CRM systems to study customer behavior and optimize customer experience. Apart from customer acquisition, companies can improve customer retention by using predictive analytics to detect early signs of disengagement.Financial service companies are gaining business advantage by mining and analyzing Big Data to stay ahead of the competition, improve customer service, detect fraud, accurately calculate risks and maximize operational efficiencies, along with adhering to stringent regulations and compliances.Indian service providers are enabling Big Data analytics in the area of fraud detection, client behavior analysis, trading pattern analysis, risk calculation on large portfolio of loans, and improved and targeted marketing campaigns. Further, Indian financial sector companies are increasingly favoring Big Data analytics to tackle terabytes of unstructured data:YES Bank is finding out solutions to handle the increasing pile of unstructured data from mobile devices and social media networks, customer transaction starting from withdrawal of money from bank, and ATM. The bank feels the regulatory requirement of storing internally generated data is driving banks to adopt Big Data.
  • The Big Data phenomenon has led to an increasing demand for ‘data scientists’ – professionals conversant with both the business context and data analytics – who play a crucial role in extracting insights from large data sets, analyzing these and then presenting the value-added information to business users or non-data experts. Big Data needs a new breed of professionals with a deep expertise in statistics and machine learning, as well as managers and analysts who can leverage insights for Big Data. The shortage of such talent is a significant challenge that companies need to address for successful Big Data implementation. According to McKinsey, the US alone faces a shortage of 140,000-190,000 analysts and 1.5 million managers who can analyze Big Data. To address the shortage, companies have embarked on initiatives to train their existing employees and develop new talent. Companies such as EMC2, Oracle, and IBM are partnering with universities to offer courses on various elements of Big Data. Internally, enterprises are creating organizational cultures that are favorable for data-driven decisions by hiring employees from academic fields such as statistics, and mathematics, as well as through on-the-job training on emerging technologies in the Big Data space.
  • As enterprises undertake pilots for Big Data implementation and large IT companies and startups compete for market share, the global Big Data market is expected to grow by about 46% to more than USD 25 billion by 2015. The IT & IT-enabled services, including analytics, are expected to grow the fastest, at a rate of more than 60%), with their share in the total Big Data market expected to increase to ~45% in 2015 from ~31% in 2011. Big Data analytics is likely to be driven by the near-ubiquitous nature of the data and proliferation of technologies and applications such as mobile sensors, smart phones and social networking, along with the growing realization of the benefits of Big Data by enterprises. While Big Data could add momentous value in the coming years, it might have to overcome certain roadblocks. Though early movers are formulating Big Data strategies, mass adoption may be hindered by the lack of best practices and the significant cultural change organizations require for sharing data. However, as companies leverage large datasets from within and outside, Big Data is likely to continue to grow as an area which can deliver substantial benefits. Finally, the aggressive efforts of service providers – both large IT companies and niche startups – to demonstrate their domain expertise and ability to derive valuable insights from Big Data would be an enabler to this opportunity.
  • India’s Big Data outsourcing opportunity is likely to grow by about 83% annually to ~US$1.0 billion during 2011-15. India is expected to be the preferred destination for analytics and IT services for Big Data due to its pre-eminence in IT/BPO services, knowledge services outsourcing and analytics as well as for its intellectual pool of talent. The share of analytics in the overall Big Data opportunity is expected to rise from ~16% in 2011 to 25% in 2015. The key drivers for India include the efforts of service providers to develop talent and increase their domain expertise and breadth of services. Moreover, a number of Indian service providers are leveraging partnerships with Big Data technology players to facilitate delivery of Big Data solutions. Finally, while the current demand for Big Data analytics is generated from global clients, domestic demand in India is also gaining traction. For example, Asian Paints and Star India have leveraged Big Data analytics to track and analyse large datasets.
  • India’s Big Data outsourcing opportunity is likely to grow by about 83% annually to ~US$1.0 billion during 2011-15. India is expected to be the preferred destination for analytics and IT services for Big Data due to its pre-eminence in IT/BPO services, knowledge services outsourcing and analytics as well as for its intellectual pool of talent. The share of analytics in the overall Big Data opportunity is expected to rise from ~16% in 2011 to 25% in 2015. The key drivers for India include the efforts of service providers to develop talent and increase their domain expertise and breadth of services. Moreover, a number of Indian service providers are leveraging partnerships with Big Data technology players to facilitate delivery of Big Data solutions. Finally, while the current demand for Big Data analytics is generated from global clients, domestic demand in India is also gaining traction. For example, Asian Paints and Star India have leveraged Big Data analytics to track and analyse large datasets.
  • As Big Data technologies become mainstream, the vendor landscape is evolving rapidly. Data management includes vendors of Hadoop-based solutions, other MapReduce technology suppliers as well as cloud and data center providers. The increased demand for Big Data analytics has changed the competitive landscape for the Big Data analytics service providers. In addition to the incumbent IT/BPO/knowledge service players, there are now more pure-play analytics players, some of whom provide sector-specific analytics solutions. Some of the larger organizations have set up captives, which provide data analytics solutions to the other divisions and subsidiaries of those organizations. Even the breadth of the services provided by analytics companies has substantially increased from data storage and management to delivering real-time insights and end-to-end data analytics services.Big Datamanagement and storage: Many new companies have emerged as providers of Apache open source Hadoop distributions, with various levels of proprietary customization for data management. Cloudera and Hortonworks are the major players for Hadoop distributions. While Cloudera contributes significantly to Apache HBase, the Hadoop-based non-relational database that enables low-latency, Hortonworks mainly offers next-generation MapReduce architecture. Other pure players include MapR, Hadapt, and Zettaset. Moreover, mega IT vendors have also entered the Big Data market through acquisitions. The Big Data warehouse market is mainly led by four players – IBM Netezza, EMC2Greenplum, HP Vertica and Teradata Aster Data. Non-Hadoop vendors are also significantly contributing to the Big Data market opportunity – Splunk, HPCC Systems, and Datastax are some of the key players.Big Data analytics: With the deluge of data, it has become pertinent to have applications and platforms that leverage the underlying Hadoop infrastructure for data analytics. Some of the key players in this segment are: Karmasphere, which offers an analytical development platform to perform ad-hoc queries on Hadoop-based data via an SQL interface; Datameer, which provides a Hadoop-based business intelligence platform that leverages a spreadsheet-like interface to analyze data; and service providers such as QlikView, Revolution Analytics, Informatica, 1010data, and ClickFox which offer cloud-based Big Data applications and services. Big Data use: Big Data analytics engage with large data sets which may be difficult to understand for business users. A number of companies such as Amazon Web Services, Google, and Intellicus are launching new user applications which facilitate the usage of Big Data analytics.Additionally, the landscape for Big Data IT services is growing exponentially, with established service providers such as Oracle, IBM, and CSC building their Big Data service portfolio. Moreover, Indian IT/BPO players such as TCS, Infosys, and Wipro are also bolstering their capabilities in Big-Data-specific software development and implementation.
  • Big Data’s Big Impact on Businesses

    1. 1. Big Data’s Big Impact on BusinessesTo join the call, please dial the below toll-free phone line for your country: – USA 1 866 746 2133 – UK 0 808 101 1573 – Singapore 800 101 2045 – Hong Kong 800 964 448 – India 1 800 200 1221 – Australia 1 800 053 698 – Poland 00 800 112 4248 – Netherlands 0 800 022 9808 – UAE 800 017 5282 – Argentina 0 800 444 1557 – China 10 800 140 1383 – South Korea 003 081 32 503 – Sweden 020790997If you are not based in any of the above locations, you can dial the following numbers to participate in the discussion.Primary number: 0013233868721Secondary number: 00442031067123 1
    2. 2. Big Data’s Big Impact on BusinessesWebconference : Jan 29, 2013
    3. 3. Key Takeaways Slide 3 Introduction to Big Data Slide 5 Global Landscape and Trends Slide 12 The Big Data Opportunity Slide 20Big Data’s Big Impact on Businesses
    4. 4. Key Takeaways Big Data market opportunity is expected to witness strong growth in the next 5 years – Expected to touch US$25 billion globally; the ‘BIG’ opportunity for India lies in the IT & IT-enabled Services space, which is likely to be ~US$ 10-11 billion market globally in 2015 – India is likely to garner a ~10% share of the ~US$ 10-11 billion global Big Data IT Services Market by 2015 – Data-related regulations like Dodd-Frank and Basel III to impact Big Data implementations Initially, North America & Europe are likely to drive the Big Data opportunity since over 85% of the world’s data is today residing in these 2 regions New database architectures and innovative analytics tools & techniques to facilitate Big Data implementations By end of 2012, around 90% of Fortune 500 companies had some initiatives underway related to Big Data Key verticals driving demand for Big Data analytics: Financial services, Retail, Telecom, Healthcare and Manufacturing Key risk – potential shortfall of 1.5 million Data-Savvy Managers and 140,000-190,000 Data Scientists in the US by 2018Source: CRISIL GR&A analysis 4
    5. 5. Key Takeaways Slide 3 An Introduction to Big Data Slide 5 Global Landscape and Trends Slide 16 The Big Data Opportunity Slide 23 Definition of Big Data Big Data ecosystem Benefits of Big Data to enterprises Key applications for end consumers
    6. 6. Big Data is Defined by Volume, Variety and Velocity What is Big Data ? Big Data relates to rapidly growing, Structured and Unstructured datasets with sizes beyond the ability of conventional database tools to store, manage, and analyze them. In addition to its size and complexity, it refers to its ability to help in “Evidence-Based” Decision-making, having a high impact on business operations Speed, Accuracy and Complexity of Intelligence 3Vs 1 Large quantity of data Small Data Sets Big Data which may be enterprise- Volume specific or general and public or private Advanced Big Data analytics analytics 2 Diverse set of data being created, such Variety as social networking Small Data Sets Big Data feeds, video and audio files, email, sensor data and Traditional Traditional other raw data analytics analytics Velocity 3 Speed of data inflow as Gigabytes Terabytes Petabytes Zetabytes well as rate at which this fast-moving data needs to Size of Data be stored Source: CRISIL GR&A analysisSource: CRISIL GR&A analysis 6
    7. 7. The Global Data likely to Grow at a CAGR of 41% Growth of global data, 2009-2020 Zetabytes CAGR 35.0 (2009-2020) 41.0% 7.9 1.9 0.8 2009 2011 2015 2020 Implication for an organization  Need for large storage capacity and quick retrieval of data  Enable informed decision-making effectively, leveraging large data sets – Turn 12 TB of Tweets created each day into improved product sentiment analysis – Convert 350 billion annual meter readings to better predict power consumptionNote: ZB stands for Zetabytes;Source: IDC; CRISIL GR&A analysis 7
    8. 8. VolumeToday 80% of Data Existing in any VarietyEnterprise is Unstructured Data Velocity Introduction  Resides in formal data stores – RDBMS and Data Structured Data Warehouse; grouped in the form of rows or columns  Variety of sources from where data is being  Accounts for ~10% of the total data existing currently generated has also undergone a shift RDBMS (e.g., Data Microsoft Project  The types of data being created has changed from ERP and CRM Warehousing Plan File structured to semi-structured to unstructured data  A form of structured data that does not conform with the Implication for organization Semi- formal structure of data models Structured Data  Need to manage broad range of data types  Accounts for ~10% of the total data existing currently  Process analytic queries across numerous data types  Comprises data formats which cannot be stored in row/ Unstructured column format like audio files, video, clickstream data, Data Solutions required  Accounts for ~80% of the total data existing currently Weather Location  Need to extract meaningful analysis from this data Video Audio Text message Blogs patterns co-ordinates has led to several technologies to gain traction  Examples include NoSQL databases to store unstructured data as well as innovative processing Web logs & Sensor data/ Geospatial methods like Hadoop and massive parallel clickstreams M2M Email Social media data processing (MPP)Source: Industry reporting; CRISIL GR&A analysis 8
    9. 9. VolumeBig Data will Enable Real Time Analytics Variety Velocity Big Data velocity enabling real 600+ time use of data videos on YouTube 200  Big Data is also characterized by 1,500+ blog posts million+ velocity or speed i.e. frequency of emails sent data generation or the frequency of data delivery 2  New age communication channels 7,000+ million+ photos on Google such as mobile phones, emails, social flickr search networking has increased the rate of Data queries information flows velocity per Examples: minute 700,000+ 400,000+  Telcos adopting location based minutes of Facebook Skype marketing based on user location updates sensed by mobile towers calling  Satellite images can help monitor and US$ 3500+ analyze troop movements, a flood 300,000+ ticks per are spent minute in plane, cloud patterns, or forest fires on online 400,000+ securities shopping trading  Video analysis systems could monitor tweets on Twitter a sensitive or valuable facility, watching for possible intruders and alert authorities in real timeSource: Industry reporting; CRISIL GR&A analysis 9
    10. 10. Big Data Analytics is Application of Advanced Techniques on BigDatasets; Answers Questions Previously Considered Beyond Reach Evolution of analytics Big Data analytics Behavioral analytics Advanced  Big Data analytics is analytics where advanced Stochastic Analytic analytic techniques Complex optimization database  Why did it Prescriptive are applied on Big event happen? functions analytics Data sets processing Constraint  When will it Optimization  The term came into Extreme SQL based BI happen play late 2011 – early Visualization again? Level of Complexity 2012 Predictive Social network analytics  What modeling caused it to Predictive Semantic analytics Forecast happen? analytics - ing  What can be Time series analysis Statistical done to analysis Natural Language Processing avoid it? Multivariate statistical analysis Alerts Online analytical processing (OLAP) Descriptive Query Data mining analytics drill down Adhoc Basic analytics reports  What happened? Standard  When did it happen? reports  What was the its impact ? Late 1990s 2000 onwards Time Analytics as a separate value chain function In-database analytics Source: CRISIL GR&A analysis 10
    11. 11. Big Data Management, Analytics, IT Services & Applicationsare the Key Constituents of Big Data Ecosystem Four key elements: What does the Big Data Ecosystem Constitute ? 1. Big Data Components of Big Data Ecosystem Management & storage: End users  Data storage Big Data Analytics application and use Data analytics & its infrastructure Applications and technologies Developer Environments Analytics (mobile, search, web) (Languages (Java), products (SI,customization, consulting, system design) 2. Big Data Analytics Environments (Eclipse & NetBeans), programming (Avro, Apache BI  Includes the Thrift) interfaces (MapReduce)) &visualization technologies and tools tools to analyze the data and generate Input data insight from it Business analysts 3. Big Data’s Data IT services Application & Use Sources Big Data Operational Data  Involves enabling Data management & Unstructured Data Architecture NoSQL the Big Data data Hadoop/ Big Data NoSQL MPP insights to work in (Text, web tech’y framework Hadoop RDBMS storage BI and end-user pages, social (MapReduce etc.) based DW applications media content, 4. IT services including video etc.)  System Integration Structured Data administration tools  Consulting data ETL & Data Workflow/  Project System (stored in integration scheduler management and tools MPP, RDBMS products products customization and DW*) *MPP – Massively parallel processing; RDBMS - Relational Data Base Management Systems; DW – Data warehouse Source: CRISIL GR&A analysis 11
    12. 12. Key takeaways Slide 3 An Introduction to Big Data Slide 5 Global Landscape and Trends Slide 16 The Big Data Opportunity Slide 23 Big Data – Geographic Analysis Market Trends & Developments
    13. 13. North America & Europe Drives the Big DataOpportunity with over 85% of the World’s Data As North America and Europe account for the lion’s share of the world’s data the initial opportunity of both Big Data implementations and analytics lies in the these geographies i.e. developed economies  Key verticals: Healthcare,  Key verticals: Technology, Financial services, Manufacturing, Retail, Digital Oil & Gas, Utilities, Manufacturing Marketing  Demand trend: European MNC’s are still in  Demand trend: High demand the early stages of the adoption cycle North of Big Data analytics America Europe >2,000 Japan >400 >3,500 China Middle East >250 >200 India >50  Key verticals: Manufacturing, Telecom, Health & Life Sciences  Demand trend: Demand for BI  Demand trend: Current demand to derive operational efficiency appears to be limited, however, South America lack of skills may drive  Key verticals: Telecom, Bioinformatics, >40 outsourcing of Big Data analytics Retail  Demand trend: Industry is in nascent stage with demand catching up, particularly in retail  Low awareness levels  Key verticals: Telecom, Retail, Banking  Demand trend: Still embryonic; most organizations have wait and watch approach Data generated: High to low Amount of new Big Data stored (Petabytes), 2010Source: McKinsey Global Institute; CRISIL GR&A analysis 13
    14. 14. Emergence of Niche Startups and Large IT Players Enhancingtheir Big Data Capabilities are key enablers for the Industry Market Trends and Developments 1 Emergence of niche Big Data startups driving technological innovation 2 Large IT players leveraging M&As to add Big Data capabilities to their service portfolios 3 Financial Services, Retail and Telecom are likely to be the early adopters in the Big Data space 4 Talent shortage is one of the biggest challenges of the Big Data spaceSource: CRISIL GR&A analysis 14
    15. 15. Emergence of niche Big Data start-ups to boost 1technological innovation A new class of companies, specializing in Big Data technologies have emerged, to capitalize on the opportunities in the Big Data domain Technology Area Players* Big Data start-ups – Key characteristics Specialized in niche Big Data technologies like Hadoop distributions Hadoop, NoSQL systems, in-memory analytics, 1 multiple parallel processing, and analytical platforms Non Hadoop Big Data Majority of start-ups generate revenue less than Platforms 2 USD 50 million and exhibit double digit revenue growth annually Most start-ups raising funding by private ventures Analytic Platforms 3 and Applications or being acquired by large IT players Cloud-based Big Data Applications*Indicative list of playersSource: Industry reporting; CRISIL GR&A analysis 15
    16. 16. Large IT Players Leveraging M&As to add Big Data 2Capabilities to their Service Portfolios Target Area Acquirer Date Deal value Rationale Company Data Oct. 11 USD 1.1 billion  Develop a comprehensive platform to analyze Big Data Management USD 263  Strengthen position in data warehousing market through Mar. 11 million expertise in SQL and MapReduce-based analysis  Extend Smarter Commerce suite with qualitative analytics Jun. 12 N.A. software  Leverage data navigation technologies for Big Data by May. 12 N.A. automating discovery of through innovative index and search capabilities May. 12 N.A.  Addition of sales performance analytics Advanced analytics May. 12 N.A.  Enhance Big Data marketing analytics Apr. 12 N.A.  Acquisition of spend and procurement analytics Mar. 12 N.A.  Accelerate development of Big Data analytic applications Mar. 11 N.A.  Enhance real time business analytics for Big DataKey highlights  M&As in the Big Data space had tripled in  M&As with bigger deal value are happening in data the first half of 2012 management  Acquisition targets are mainly innovative Big Data start-upsN.A. is not available. Source: Industry reporting; CRISIL GR&A analysis 16
    17. 17. 1. Retail: Sears is leveraging Big Data analytics internally and 3 is also keen on offering analytics services externally Sears Holding is a leading integrated retailer with ~4,000 full-line and specialty retail stores in the US and Canada. It operates through its subsidiaries including Sears, Roebuck and Co. and Kmart Corp. Challenge/Business Solution Benefits Need IT need • Leverages its global In-house center in Across IT environment • Manage Increasing volumes of data Pune, India for Big Data Analytics • Utilization of 100% of collected data like customer personal information, • Implemented a Big Data architecture against 10% utilization earlier PoS data, online purchases, etc., using Hadoop • Ability to run price elasticity posing a challenge • Used MapReduce algorithms to analyze algorithms in one week, as opposed • Capacity run-out on its mainframe, and data and feed results back into the to eight weeks previously adding more capacity proving to be mainframe, on individual customer expensive activity, across all 4,000 locations • Cost-savings of USD 600,000 per year Business need Across business • The need to set prices quickly and in • More relevant and personalized real time customer communications and offers to • The need to drive customer loyalty an active customer base (~80 million) • Increased shopping and higher spend per transaction by active members Looking at the current and potential benefits of Big Data analytics, Sears aims to expand into newer areas and sell its data management and analytics services technology to other companies, through its subsidiary MetaScale*Massively Parallel ProcessingSource: Industry reporting; CRISIL GR&A analysis
    18. 18. 2. Financial Services: Witnessing increased adoption of Big Data 3analytics, to reduce risk and uncover new market opportunities • The need to meet growing regulatory compliances, detect fraud and create new market opportunities is driving the growth for Big Data analytics in the financial services sector • Customer & transaction data from multiple channels like branch, kiosks, mobile and web; social media; emails; credit cards data; insurance claims data; stock market data; statistical data, PDF & excel files, videos, government filings, etc. are key Big Data sources Big Data application across Financial Services sub-sector Capital Banking Markets/ Insurance Trading Predict client longevity, Credit line optimization Trading surveillance along with analyzing perspective clients Credit reward program medical status Intraday analysis analysis Using weather and calamity information for Trading pattern analysis managing exposures and losses Pre-trade decision support analytics Risk management/assessment Fraud detection Portfolio analytics Compliance & regulatory reporting CRM,, Entering new marketsSource: Industry reporting; CRISIL GR&A analysis
    19. 19. Potential Shortfall of 1.5 million Data-Savvy Managers and 4~150,000 Data Scientists in the US in 2018 Demand-supply gap for data scientists* Requisite educational Role in Ecosystem Other expertise in US, 2018 qualifications 440K-490K  Big Data analytics  Advanced degree like  Expertise in data Data  Business intelligence M.S. or Ph.D., in analytics skills to extract mathematics, statistics, data, use of modeling & 300K Scientists  Visualization economics, computer simulations science or any decision 140K – 190K  Multi-disciplinary sciences knowledge of business to 50%-60% find insights gap relative to supply  Knowledge of statistics  Project management  Advanced business and/or machine learning Data-savvy across the Big Data degree such as MBA, M.S. or managerial to frame key questions Managers ecosystem and analyze answers diplomas – Consulting2018E Supply 2018E Demand services  Conceptual knowledge of – Implementation business to interpret and – Infrastructure challenge the insights Demand-supply gap for data-savvy management managers* in US, 2018 – Analytics  Ability to make decisions using Big Data insights 4.0 million  Technical support in  Having a degree in  Possessing data Technical hardware & software computer management knowledge 2.5 million across the Big Data science, information Engineers ecosystem for: technology, systems  IT skills to 1.5 million – Data architecture engineering. or related develop, implement, and 60% gap disciplines maintain hardware and – Data relative to administration software supply – Developer environment – Applications 2018E Supply 2018E Demand*Analysts with deep analytical training; **Managers to analyze Big Data and make decisions based on their findings; Source: McKinsey Global Institute; CRISIL GR&A analysis 19
    20. 20. Key Takeaways Slide 3 An Introduction to Big Data Slide 5 Global Landscape and Trends Slide 16 The Big Data Opportunity Slide 23 Forecasted market size Future outlook
    21. 21. Global Big Data market to reach ~USD 25 billion by2015,with a 45% share of IT & IT-enabled services  The global Big Data market is expected to grow by about a CAGR of 46% over 2012-2015  IT & ITES, including analytics, is expected to grow the fastest, at a rate of more than 60% – Its share in the total Big Data market is expected to increase to ~45% in 2015 from ~31% in 2011  The USD 25 billion opportunity represents the initial wave of the opportunity. This opportunity is set to expand even more rapidly after 2015 given the pace at which data is being generated. Global Big Data Market Size, 2011 – 2015E Global Big Data Market Size, 2015F US$ billion ~US$25 billion 25.0-26.0 Opportunity for India lies in capturing the Big Data analytics & slice of IT services that US$ 10-11 IT & IT-enabled includes Big Data billion services analytics and IT & IT- enabled services US$ 7-7.5 Lion’s share of the Big 8.0-8.5 Software billion Data hardware and software market is 5.3-5.6 expected to be occupied by IT giants US$ 6-6.5 like Hardware billion IBM, HP, Microsoft, SA P, SAS, Oracle, etc. 2011E 2012E 2015F 2015Source: Industry reporting; CRISIL GR&A analysis 21
    22. 22. Conclusion Big Data market opportunity is expected to witness strong growth in the next 5 years – Expected to touch US$25 billion globally; the ‘BIG’ opportunity for India lies in the IT & IT-enabled Services space, which is likely to be ~US$ 10-11 billion market globally in 2015 – Data-related regulations like Dodd-Frank and Basel III to impact Big Data implementations New database architectures and innovative analytics tools & techniques to facilitate Big Data implementations Key verticals driving demand for Big Data analytics: Financial services, Retail, Telecom, Healthcare and Manufacturing Key risk – potential shortfall of 1.5 million Data-Savvy Managers and 140,000-190,000 Data Scientists in the US by 2018Source: CRISIL GR&A analysis 22
    23. 23.
    24. 24. Appendix
    25. 25. India’s ‘BIG’ opportunity is in IT andIT-enabled services India Big Data outsourcing opportunity, 2011 – 2015E India Big Data outsourcing opportunity, by US$ billions category, 2015F, Percent 100%= ~US$1.1 billion 1.1-1.2 24%-27% Pure-play Analytics firms Integrated IT/ BPO players ~0.2 ~0.1 73%-76% 2011E 2012E 2015F Source: CRISIL GR&A analysis Source: CRISIL GR&A analysis  India’s Big Data market is expected to grow at a 83% CAGR over 2011-2015 to reach ~US$ 1.1-1.2 billion  India’s share in the ~USD 10-11 billion global Big data IT and IT-enabled services market is expected to be ~10% in 2015 , where: – In 2015, integrated IT and BPO players will dominate the US$1.1 billion opportunity with close to 73-76%Source: Industry reporting; CRISIL GR&A analysis 25
    26. 26. Key Players Across the Traditional and Big DataTechnology Stack Key players in BI/Traditional Analytics vs. Big Data Analytics technology stack Big Data Analytics BI/Traditional Analytics Big Data Use E-commerce, Search , Social gaming Traditional BI suites and OLAP End-user applications IT Services – Data Management Big Data Analytics Basic visualization apps. Advanced visualization apps. MapReduce Programs Visualization tools Traditional Analytics Advanced Analytics Analytical tools Parallel Relational Hadoop Big Data RDBMS NoSQL Databases Database SAP HANA Data management systems Conventional file systems HDFS Infrastructure & storage systems Monolithic Hardware Distributed HardwareNote: This is a representative list of playersSource: Industry reporting; CRISIL GR&A analysis 26
    27. 27. Financial Services and Telecom to be the earlyadopters of the Big Data  Indian service providers like Infosys, Fractal are enabling Big Data analytics in the area of fraud detection, CRM Financial and customer loyalty program, trading pattern analysis, risk calculation on large portfolio of loans Services  Key Adopters: JPMorgan Chase, Merrill Lynch, HSBC, American Express, Goldman Sachs, Barclays, Bank of America, Citigroup, and Wells Fargo  Telecom players are increasingly focusing on Big Data to limit churn rates, build loyalty and provide multi- Telecom channel and multi-service applications by proactively analyzing the subscriber and network data  Key Adopters: Airtel , Vodafone  Both brick and mortar as well as online retailers are increasing their adoption of Big Data analytics for real time Retail analysis of purchase behavior and buying patterns, enhanced customer segmentation and customer loyalty  Key Adopters: Walmart & Sears  Indian service providers are enabling manufacturing companies through Big Data analytics in the areas of Manufacturing accurate demand forecasting, optimization of operations, inventory management, open innovation and better analysis of post sales feedback in real time  Key benefits of big data in public sector include: Intelligence to counter national threats, Forecast economic Public Sector events, Traffic management, Environment monitoring, energy/ waste management, etc.  Healthcare players use Big Data Next-generation sequencing and mapping for genomics, analysis of correlation Healthcare between treatments & outcomes and real time data from medical devices for better patient careSource: Industry reporting; CRISIL GR&A analysis 27