Downloaded 20 times
Uploaded bySti Uff
UFF Tech 2013 - Big Data - Rafael Borges EMC
The document discusses big data and EMC's expertise and strategy around big data. It provides definitions of big data, describes the massive growth of digital data and challenges of managing unstructured data. It outlines EMC's R&D center in Brazil focused on big data and collaboration. It also distinguishes between business intelligence and data science, the importance of predictive analytics, and the role of data scientists. Finally, it outlines EMC's oil and gas strategy and examples of big data applications and technologies across the upstream exploration and production process.
UFF Tech 2013 - Big Data - Rafael Borges EMC
- 1. Big Data Rafael Borges EMCCentro de P&D rafael.araujoborges@emc.co m Copyright © 2013 EMC Corporation. All Rights Reserved.
- 2. EMC Expertise Agregando valorà informação através de mudanças de TI CLOUD BIG DATA Otimização de infraestrutura através de virtualização Transformando negócios com o uso de ferramentas analíticas Copyright © 2013 EMC Corporation. All Rights Reserved. Security Cyber security, compliance e gerência de risco
- 3. Atuação ENTERPRISE APPLICATIONS BIG DATAAPPLICATIONS Pivotal VPLEX VNX VNXe VMAX VMAXe Ionix Isilon Data Domain, Avamar, NetWorker Copyright © 2011 EMC Corporation. All Rights Reserved. 2013 Atmos
- 4. Estratégia de Inovação P&D Aquisiçãode Tecnologias Copyright © 2013 EMC Corporation. All Rights Reserved. 12% Investment 10% Investment
- 5. EMC’s Big DataR&D Center in Brazil Applied Research Center at the heart of the Technology Park in Rio Purpose built facility located with the University Campus / Technology Park Complex Industry Leading Technology Ecosystem: Schlumberger, Halliburton, BG, Siemens, GE, Petrobras, and others 50+ EMC Big Data Scientists Collaborating with 50+ others on Campus Joint projects with leading technology and O&G companies Solution Development and Certification World Class Executive Briefing Center Copyright © 2013 EMC Corporation. All Rights Reserved.
- 6. O que éBig Data, afinal Copyright © 2013 EMC Corporation. All Rights Reserved.
- 7. Big Data RefersTo… • All Data that comes at high Volume • All Data that comes at high Velocity • All Data that comes from a Variety of Sources • All Data that brings Complexity • All Data that challenges existing Information Infrastructure Capabilities • All Data that makes us “Think Different” Today Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 8. IN 2010 THEDIGITAL UNIVERSE WAS 1.2 ZETTABYTES 1,200,000,000,000,000,000,000 Zetta Exa Peta Tera Source: 2010 IDC Digital Universe Study Copyright © 2013 EMC Corporation. All Rights Reserved. Giga Mega Kilo Byte
- 9. Avalanche desta década 2020 2009 0.8Zettabytes CRESCIMENTO DA INFORMAÇÃO 44 X MAIOR 35.2 ZB O NÚMERO DE PROFISSIONAIS DE TI NO MUNDO VAI CRESCER MENOS QUE 50% fONTE: IDC Digital Universe Study, patrocinado pela EMC, 2011 Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 10. Ondas de evolução Computação emnuvem Redes/Computação distribuída PC/ Microprocessador Minicomputador Mainframe Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 11. Com a “Nuvem”,agora estamos prontos para a era da conectividade Copyright © 2013 EMC Corporation. All Rights Reserved.
- 12. O que estácausando essa avalanche de dados? Renderização de Vídeos FACEBOOK CRESCE COM A PUBLICAÇÃO DE 250 MILHÕES DE FOTOS/DIA Sensores móveis Redes sociais Vídeos de vigilância Imagens médicas LER MEDIDORES A CADA 15 MIN GERA 3,000X MAIS DADOS Sequenciamento de gens O CUSTO DE SEQUENCIAR UM GENOMA CAIU DE $100M EM 2001 PARA $10K EM 2011 Smart Grids Exploração Geofísica Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 13. Big Data éum conceito relativo O que é grande hoje… Pode não ser tão grande amanhã…. Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 14. Fontes de DadosCrescem Sem Parar Informações nas empresas irá CRESCER 50X Nos próximo 10 anos Source: 2011 IDC Digital Universe Study Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 15. 90% DO MUNDO DIGITALÉ NÃO ESTRUTURADO Source: 2011 IDC Digital Universe Study Copyright © 2013 EMC Corporation. All Rights Reserved.
- 16. Então… Tudo quepreciso é gerenciar Big Data, certo? Copyright © 2013 EMC Corporation. All Rights Reserved.
- 17. Errado! Copyright © 2013EMC Corporation. All Rights Reserved.
- 18. Big Data IsAbout Copyright © 2013 EMC Corporation. All Rights Reserved.
- 19. Big Data IsAbout Predictive Analytics Copyright © 2013 EMC Corporation. All Rights Reserved.
- 20. E quem faráessa revolução? Copyright © 2013 EMC Corporation. All Rights Reserved.
- 21. BI foca nogerenciamento e reporte de dados existentes para monitorar e gerenciar questões tangentes ao empreendimento. E quem é o Copyright © 2011 EMC Corporation. All Rights Reserved. 2013 Data Science aplica avançadas ferramentas e algorítimos de análise para criar novas informações e inovações que são um desdobramento direto dos dados originais Cientista de Dado
- 22. Qual a diferença? Alto DataScience • Análises Preditivas VALOR DO NEGÓCIO Data Science • E se…? Business Intelligence Business Intelligence • Relatórios Baixo Passado Copyright © 2011 EMC Corporation. All Rights Reserved. 2013 • O que aconteceu? TEMPO Futuro
- 23. Perfil de umCientista de Dados Quantitativo Técnico Cético Copyright © 2013 EMC Corporation. All Rights Reserved. Curiosidade & Creatividade Comunicativo & Colaborativo
- 24. EMC Oil AndGas Strategy ADVANCE LEVERAGE RESEARCH existing EMC oil and gas solutions a robust partner ecosystem inventing the next generation of applied technologies Copyright © 2013 EMC Corporation. All Rights Reserved.
- 25. Big Data Definition:Upstream – Seismic acquisition – Seismic processing – Seismic interpretation – Geological interpretation • Seismic – SEGD, Pre-Stack, Post-Stack • Navigation Volume Copyright © 2011 EMC Corporation. All Rights Reserved. 2013 Engineering and Production Development Exploration – Reservoir Modeling – Reservoir Simulation – Facilities & – Drilling & Test – Production Development Reservoir and Engineering Optimization – Drilling program • Recently Acquired and Historical – – – – Log curves Production data Drilling and test Micro seismic Variety – – – – Tops Lithology Cultural Cores – Production • Real Time – – – – – Flow LWD – Pressure MWD Mud logging Rate of Penetration Velocity
- 26. Data Dependent Compression Visualizationand Collaboration over Distance Integrating Wireless Sensors & Analytics Optimized I/O Platforms for Processing Plug-in Development Analytics for Risers Integrity Management Data Acquisition Seismic Processing Interpretation Drilling Production Data Lifecycle Management Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 27. Scientific Workflow Optimization ComplexData Pattern Recognition Time Series Data Uncertainty Analysis Remote User Data Protection Plug-ins Optimized RTM ; FWI Processing Platforms Mega-survey Mobility and Distribution Edge Data Mobility and Transport Condition Based Maintenance Analytics Logistics Optimization / Incident Management Analytics Data Acquisition Seismic Processing Interpretation Drilling Production Data Lifecycle Management Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 28. Partition Tolerant StorageArchitectures 3D Application Cloud Virtualization Architecture Topside Plant and Process Analytics Production Optimization Analytics HPC Checkpoint Optimization In-memory HPC architectures Analytics for Drilling Intervention / EHS Data Acquisition Seismic Processing Interpretation Drilling Production Data Lifecycle Management Copyright © 2011 EMC Corporation. All Rights Reserved. 2013
- 29. OBRIGADO! Copyright © 2011EMC Corporation. All Rights Reserved. 2013
Editor's Notes
- #3 The message here is to make your enterprise “extraordinary” through IT transformation. EMC helps organizations transform their business through cloud enablement (reduce the 85% maintenance to 60%) - reducing the time/cost of “keeping the lights on” while improving business agility (i.e., basic cloud message). We can expand (or not) on this message and actually link it to Oil & Gas by discussing our leadership in virtualization (first step) - most major Oil & Gas companies are already implementing VMWare, EMC offers leading infrastructure platforms, we are building Oil & Gas solutions with our network of Service Provider Partners, etceterea. B) leading a path to business innovation and competitive advantage by leveraging Big Data solutions (shift your spend from 15% innovation to 40% innovation). Later, we can describe our vision of Big Data for Big Oil through the Volume/Variety/Velocity slide along with our $100M investment at the BRDC. C) My recommendation is to put Security below as the “pillar of trust”. It is a pre-requisite to achieving A) & B) and EMC offers the most secure solutions. The key message is that EMC has an end-to-end offering that allows Oil & Gas companies to transform IT from a cost center to an innovation center.
- #4 EMC has the complete product portfolio for Cloud and Big Data. It begins with a single layer for customers’ backup and recovery (Data Domain, Avamar, and Networker). It also includes enterprise application storage with VNX and VNXe unified storage, and VMAX and VMAXe scale-out block storage. For Big Data, EMC has Isilon and Atmos for storing unstructured and object data. There are also leading hybrid technologies: VPLEX for federation, RSA for security, and Ionix for management. For Big Data analytics, EMC has Greenplum, allowing customers to analyze both structured and unstructured data to bring in data faster than ever with scale-out data ingest, and faster queries with scale-out query execution. To accelerate a customer’s journey to the cloud, EMC has vBlock, a pre-validated, pre-integrated, converged infrastructure of compute, network, storage and virtualization from the market-leaders, EMC, Cisco, and VMware.
- #5 Hiring of 50+ NationalsHiring and sponsorship of just as many interns and grantsFirst EBC in Latin AmericaThis is an artists rendering of what the facility will look like. [click]. This is what it really looks like right now. I like the pretty picture better [click]
- #6 Today, I am making deliberate choices to not spend much time talking about the IT budget dilemma, even though you will soon realize this is an issue, and I will not discuss much about security either…I will instead focus on the Information Deluge issue…
- #7 “The second is the data deluge. In 2010, The Digital Universe study from IDC said the Digital Universe was 1.2 Zettabytes. Zettabyte is a trillion billion bytes. Digital Universe was believed to contain 300 Quadrillion files in 2010.”
- #8 Data from the new 2011 Digital Universe from IDC, sponsored by EMCData growing 44XBut IT staff only growing at 1.5X by the end of the decadeOnly way to stay ahead of the data deluge is to increase the volume of information that can be managed per person, using new technologies and productivity tools
- #9 We think cloud is the next wave of massive disruption in IT that started with Mainframes.It’s disruptive because of the dramatic benefits it delivers to organization in both cost efficiency and agility (bottom line and top line).Its disruptive because it’s built on disruptive technologies and disruptive technology leads to lasting change.In the case of cloud, it’s arguably the most disruptive because we are seeing the IT cloud wave and the consumer cloud wave happen simultaneously.
- #10 Because now we are ready for the connected Era, where our health life is measured by the number of bars you have on your mobile device…
- #11 The data deluge is enabled by the Economies of Cloud, but is driven by the connected era avalanche of devices…The sources of information are expanding. Many new sources are machine generated. It’s also big files (siesmic scans can be 5TB per file) and massive numbers of small files (email, social media).Leading companies for decades have always sought to leverage new sources of data, and the insights that can be gleaned from those data sources, as new sources of competitive advantage.More detailed structured dataNew unstructured dataDevice-generated dataBut big data isn’t only about data, a comprehensive big data strategy also needs to consider the role and prominence of new, enabling-technologies such as:Scale out storageMPP database architecturesHadoop and the Hadoop ecosystemIn-database analyticsIn-memory computingData virtualizationData visualization
- #14 Big data não é novidade alguma
- #15 Explosion in types of data<CLICK>Not just types but size: New digital universe study by IDC, Information will grow 50x before 2021BELOW ARE NOTES ONLY, NOT FOR MONITOR:We see that the sources of data are expanding, and it’s not just the types of data, but the size of this data. Data will grow 50X in the next 10 years super Moore’s Law in growth rate of data.
- #16 New capture, search, discovery, and analysis tools can help organizations gain insights from their unstructured data, which accounts for more than 90% of the digital universe. (Mike, Proof Doc)
- #18 Big Data não é gambiarra
- #19 Mudança na forma como vemos o valor da informaçãoMudança na forma como vemos o gerenciamento da informaçãoMudança na forma como entendemos o contexto no qual a informação é geradaMudança na forma como misturamos o dados e as diversas fontesMudança na forma como vemos o mundoE principalmente...
- #20 A new role—Data Scientist—will play a key role in the Big Data Analytics world, requiring knowledge and skills involving new methodologies, technologies, and tools that go beyond traditional data analytics.While BI looks at historical data, data science enables organizations to look at disparate data sets in real time and draw conclusions that can help better predict future patterns of events. Data Scientists must understand the Business Intelligence world just as Business Intelligence analysts need to understand the Data Science world so they can work together in cohesive teams to ensure the business is gaining optimum value from leveraging big data and data in traditional data warehouses.Conselho do meu pai: meio não fim.Agora inverteu: BD possibilita ser meio e fim.
- #23 As background, it is important to understand that Business Intelligence is different than data science and analytics. BI deals with reporting on history. What happened last quarter? How many did we sell, etc.Data science is about predicting the future and understanding why things happen. What is the optimal solution? What will happen next?For many companies data science is a new approach to understanding the business yet an important one to undertake today. Gartner states that enterprises who are embracing Big Data and Data Science will outperform their peers by over 20% in the next five years.
- #24 Here are 5 main competency and behavioral characteristics for Data Scientists.Quantitative skills, such as mathematics or statistics Technical aptitude, such as software engineering, machine learning, and programming skills. Skeptical…..this may be a counterintuitive trait, although it is important that data scientists can examine their work critically rather than in a one-sided way.Curious & Creative, data scientists must be passionate about data and finding creative ways to solve problems and portray informationCommunicative & Collaborative: it is not enough to have strong quantitative skills or engineering skills. To make a project resonate, you must be able to articulate the business value in a clear way, and work collaboratively with project sponsors and key stakeholders.