Key PointsThree common themes emerged + common “culture”Will cover each in detail and illustrate with examples; now a quick overviewTheme 1: Importance of data Approach to code and evolving technology with industry peersTheme 2:Role of the common platformHow they drive engineering productivity with toolsTheme 3:Approach to innovationRapid release and iterationsCulture:Pervasive elements that we’ll encounter throughout
Big Data for Bigger Decisions & Better Business Soumendra Mohanty Nasscom India Leadership Feb 16th, 2012 Mumbai
There is an Explosion in Data and Real World Events 4 Billion Internet users by 20121.3 Billion RFID tags in200530 Billion RFID today 4.6 Billon Mobile Phones World Wide Capital marketdata volumes grew Twitter process 1,750%, 2003-06 7 terabytes of data every dayWorld Data Centre for FacebookClimate process 220 Terabytes of Web data 10 terabytes of 9 Petabytes of additional data every day data
Data is becoming part of every industry and business function…Big Data is top of mind for virtually every industry, impacting core businessprocesses. Resources Upstream Oil & Gas companies Health monitor 40K sensors per asset Electronic health records, home (combined with 4d seismic imagery) health monitoring, telehealth, and to drive real-time production new medical imaging devices drive operations and maintenance & data deluge in a connected health reliability programs. world. Public Sector Retail USPS applies unique barcodes so it Emerging location based data, group can seamlessly induct and account purchasing and online leads allow for postage. This results in ~1B Retailers to continuously listen, pieces per day, scanned multiple engage and act on customer intent times throughout the supply chain. across the purchasing cycle.Financial Services Communications Pioneers in Big Data, Capital Mobile usage data for Service Markets firms continue to innovate Providers unlock new business around low latency systems to models and revenue streams from unlock trading arbitrage Outdoor Ad placement to medical opportunities. adherence.
The Big Data OpportunityExtracting insight from an immense volume, variety, variability and velocity of data,in context, beyond what was previously possible. Variety: Integrate multiple relational and non-relational data types and schemas Velocity: Streaming data and large volume data movement Variability : Point in time data applying business context and criticality of time Volume: Scale from terabytes to zettabytes5
Business Expectations: Maximize Return on Data Value of Data Return on Data = Cost of Data“Definitive Value of Data” – Data within the firewalls, in the EDWs, Data Marts, Reports,Dashboards, this data is currently used today to help/run businesses; proven, tried, tested!“Perceived Value of Data” – Data outside the firewalls, we have a view that this data is valuable,but it is not proven yet!
Accenture’s view - how ‘Big Data’ will evolve … Current view Future view (i.e. 6-7 years out)Two disconnected worlds and ‘ad-hoc’ Target: hybrid architectures are emerging to supplement RDBMS-based architecture for unstructured data data warehouses with tools for unstructured data End-to-End Data Governance, MDM, DQ Actionable Insights Data Management Insight Generation Ingestion “We believe both the worlds need to co-exist for some time … 1. Data can be voluminous and poly-structured – prime for big data 2. Data can be less on volume but poly-structured – prime for big data 3. Data can be less on volume and structured only – not prime for big data 4. Data can be less on volume and semi-structured – likely to become a big data candidate in the near future
5 Interventions that are necessary to address the emerging data platformIntervention 1 – Next generation Data Warehouse (structured data + In memory databases + columnar data architectures)Intervention 2 – Hadoop and the likes of cassandra etc (unstructured data)Intervention 3 – Making data available from the Source (upload, refining, publishing)Intervention 4 – BI Tooling, Advanced Data Visualization, Real Time Insight GenerationIntervention 5 – Robust Data Management principles covering end to end data governance spanning across distributed systems, this would enable the “feedback loop” from “ingestion” to “insights” to “actions” 5 Actionable 4 Insights End-to-End Data Governance, MDM, DQ Data Management 1 2 Insight Generation 3 3 Ingestion
The Big Data Approach Treat data as a strategic asset, seek to maximize it’s value to the organization Invest in common services, data platforms and tools Rapidly prototype, deliver, and measure value-added data services, evolve over time• Data-driven decision making • Sharing of platform, tools and code• Experimentation and continuous improvement with academic rigor• End-to-end ownership of services Culture