Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Introducción a Big Data. HDInsight - Webcast Technet SolidQ

8,898 views

Published on

http://university.solidq.com
Introducción a Big Data. Gracias a la nueva tecnología Microsoft Hadoop, tanto on-premise como en la nube, y a sus capacidades de almacenamiento, procesamiento y escalabilidad, podemos analizar grandes volúmenes de datos del orden de Terabytes o Petabytes de una forma rápida y con buen rendimiento. Es ahora el momento de familiarizarnos con la nueva forma de trabajar en Hadoop a la hora de extraer esa información distribuida por nuestros nodos. De la misma forma que hacemos con SQL Server, en esta sesión aprenderemos a desplegar y almacenar esa información en nuestros nodos, a crear consultas para extraer las partes que nos sean más útiles.

Published in: Technology
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download EPUB Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... Download doc Ebook here { https://urlzs.com/UABbn } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD THE BOOK INTO AVAILABLE FORMAT (New Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THE can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THE is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBOOK .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookBOOK, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, EBOOK, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THE Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THE the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THE Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Introducción a Big Data. HDInsight - Webcast Technet SolidQ

  1. 1. Big Data: La Hora H
  2. 2. Agenda •Introducción: La revolución de los Datos •Escenarios de Business Analytics •¿Y cómo exploto toda esta información?: Microsoft Excel
  3. 3. La revolución de los Datos
  4. 4. Tendencias de la Industria 44x En 2015, servicios de nube pública tendrán 46% de crecimiento neto en gasto de ti
  5. 5. Introducción: La revolución de los datos •Big Data, Data Science y lo que nos pueden proporcionar •Actores de mercado en Big Data •Utilidades de Big Data
  6. 6. ¿Qué es Big Data? • Big data Consists of datasets that grow so large that they become awkward to work with using on-hand DB Management tools. • Wikipedia Big data is when the size of the data itself becomes part of the problem Mike Lukides, O’Reilly Radar It’s not just your “Big Data” problems, it’s all about your BIG “data” Problems. Alexander Stojanovic, Hadoop Manager on Win Azure
  7. 7. Las 4 V’s Volumen Velocidad Variedad Variabilidad
  8. 8. Ejemplos de Big Data 12 Tb día 21 Pb Hadoop cluster 7 Pb mes 1 Tb tweets/dia 75 Million scores/day 14 Tb Hadoop cluster 4 Billion Graph edg/day 7 Tb datos/dia
  9. 9. Ficheros log Datos menos estructura dos Datos públicos Device outputs Texto/ Imágenes New Insights  Datos menos estructurados  Intentar un ETL para transformarlo en relacional  Tiempo de desarrollo elevado  Son datos susceptibles a cambios de estructura  Archivados y Borrados  Acceso caro Entonces…¿cómo obtengo insights? Datos estructurados  Datos estructurados  Bases de Datos relacionales  Bases de Datos analíticas Registros
  10. 10. ¿Qué es Big Data para ? • Datos, datos, datos….. • ¿Big? Hoy es Big, pero dentro de 2-3 años, ¿seguirá siendo big? • Equipos y negocios “centrados en datos” • Big Data: “ Barreras para que una organización o equipo puedan almacenar, procesar y acceder todos los datos que necesitan para operar con eficiencia, tomar decisiones, reducir riesgos, etc.”
  11. 11. Utilidades de Big Data •Vista completa de nuestros usuarios •Seguridad / inteligencia •Operaciones Analíticas (TI, dispositivos,…) •Complemento de DW
  12. 12. Caso de Éxito: Mood Index
  13. 13. MIDAMO
  14. 14. Escenarios de Business Analytics
  15. 15. Escenarios de Business Analytics •Análisis sencillo de gran cantidad de datos no estructurados: Microsoft HDInsight •Análisis sencillo de datos en memoria: Microsoft StreamInsight •Análisis en profundidad: SQL Server y Self-Service BI
  16. 16. ¿Qué es Hadoop? •Open Source •Plataforma de almacenamiento de datos y análisis para Big Data •Optimizado para manejar • Datos masivos a través de paralelismo • Variedad de datos (Estructurados, No-estructurados, Menos estructurados) • Uso de hardware económico •No para OLTP / OLAP
  17. 17. •Escalable • Escala linealmente en capacidad de almacenamiento y computación •Tolerante a Fallos • Proporcionado por el Sistema de ficheros distribuido y el framework de lectura •Procesamiento distribuido • Sigue la estrategia de divide y vencerás ¿Qué es Hadoop?: Ventajas
  18. 18. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  19. 19. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  20. 20. Hadoop Distributed File System (HDFS) •Sistema de ficheros distribuido diseñado para grandes conjuntos de datos •Fiable y con buen rendimiento • Alto rendimiento de acceso: Latencia de disco • Alto ancho de banda Almacenamiento Clustered auto- reparable •Divide los datos entre los nodos en un Cluster • NameNode: Mantiene el mapeo de bloques de ficheros a nodos esclavos • DataNode: Almacena y sirve bloques de datos
  21. 21. Hadoop Distributed File System (HDFS) File File File File File File File File File File File File File File File File File File File File File File File File File NameNode DataNode File File File File DataNode DataNode File File File File File File File Block Size = 64 Mb Replication Factor = 3
  22. 22. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  23. 23. Map Reduce Framework • Motor de planificación parar Procesamiento de carga distribuido • Pares Clave-Valor • Función Map • Función Reduce • Lenguajes de Script : Java, python, Javascript… • Saca provecho de la distribución de datos de HDFS • JobTracker: Planifica los trabajos entre los TaskTrackers • TaskTracker: unidades de trabajo
  24. 24. Map Reduce Framework File File File File File File File File File File File File File File File File File File File File File File File File File JobTracker TaskTracker MapReduce Job TaskTracker TaskTracker File File File File File File File map mapmap Reduce
  25. 25. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  26. 26. Sqoop •Tecnología que sirve de interfaz entre HDFS y los Sistemas de información empresarial •Orígenes de datos relacionales integrados • MySQL, Oracle, SQL Server … •Importación / Exportación (Bidireccional)
  27. 27. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  28. 28. Pig •Lenguaje de flujo de datos de alto nivel y framework de ejecución •Lenguaje de consulta: PigLatin • Posibilidad de join de tablas •Por detrás ejecuta trabajos MapReduce
  29. 29. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  30. 30. Hive •Infraestructura Data Warehouse desde Hadoop •Proporciona • Sumarización de Datos • Consultas Ad-hoc •Lenguaje consulta estilo SQL: HiveQL •Por detrás ejecuta trabajos MapReduce
  31. 31. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … ¿Qué es Hadoop?: Componentes
  32. 32. Otros componentes: Hadoop Ecosystem •Mahout • Minería de Datos y Machine Learning •Pegasus • Page Rank y Graph Mining • Social Network Analysis
  33. 33. HDInsight •Project Isotope •Proporciona Apache Hadoop en • Windows Server • Windows Azure •Active Directory & System Center
  34. 34. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Lucene … Hadoop: Componentes Originales
  35. 35. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Flume HDInsight
  36. 36. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Flume JDBC Hive ODBC Microsoft BI Platform HDinsight
  37. 37. HDFS Map Reduce HivePig Sqoop Mahout Pegasus Flume JDBC Hive ODBC Windows Azure & Windows Server Microsoft BI Platform HDInsight
  38. 38. Características HDInsight • HDFS • Basado en Windows • Compatibilidad con Directorio Activo • Almacenamiento compatible: • HDFS • Azure Blob Storage • Amazon S3 • MapReduce Framework • Compatibilidad JavaScript • Hadoop Streaming con compatibilidad F# y C#
  39. 39. Características HDInsight • Hive • Consola Interactiva • Complemento Hive para Excel • Hive ODBC Driver • Potentes funciones regex • Pig • Consola Interactiva • Sqoop • Driver JDBC para SQL Server y SQL Server PDW
  40. 40. Administrando un cluster HDinsight en Windows Azure Primeros pasos
  41. 41. Escenarios de Integración
  42. 42. ¿Y cómo explotamos esta información?
  43. 43. ¿Y cómo exploto toda esta información?: Microsoft Excel •PowerPivot •Complementos Excel para Minería de Datos •Acceso a Datos Hadoop •Data Explorer
  44. 44. Excel y sus complementos
  45. 45. Big Data: La Hora H

×