SlideShare a Scribd company logo
BIG DATA AND HADOOP 
Big Data History,Hadoop, and Industry Trends 
E S T H E R K U N D I N 
B L O O M B E R G L P
BIG DATA – WHAT IS IT?
WHAT IS BIG DATA?
BIG DATA ORIGINS 
Indexing the web requires lots of storage - Petabytes of data! 
Economic problem – reliable servers expensive! 
Solution: 
 Cram in as many cheap machines as possible 
 Replace them when they fail 
 Solve reliability via software! 
Google publishes papers about: 
 GFS (2000) 
 MapReduce (2004) 
 BigTable (2006)
HADOOP – OPEN SOURCE BIG DATA SOLUTION 
Hadoop, originally developed at 
Yahoo, accepted as Apache top-level 
project in 2008
CURRENT INDUSTRY TRENDS
QUESTIONS? 
Thank you!

More Related Content

What's hot

Open source for customer analytics
Open source for customer analyticsOpen source for customer analytics
Open source for customer analytics
Matthias Funke
 
Eligotech presents @ Data Donderdag on 24 April 2014
Eligotech presents @ Data Donderdag on 24 April 2014Eligotech presents @ Data Donderdag on 24 April 2014
Eligotech presents @ Data Donderdag on 24 April 2014
Paul Broekhoven
 
Big Data Overview/Teaser (6 Aug 2013)
Big Data Overview/Teaser (6 Aug 2013)Big Data Overview/Teaser (6 Aug 2013)
Big Data Overview/Teaser (6 Aug 2013)
orcsab
 
State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...
State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...
State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...
ACTUONDA
 
Big data, Hadoop - lunchtime talk 2015.02.26
Big data, Hadoop - lunchtime talk 2015.02.26Big data, Hadoop - lunchtime talk 2015.02.26
Big data, Hadoop - lunchtime talk 2015.02.26
Robert Gibbon
 
A Brief History Of Data
A Brief History Of DataA Brief History Of Data
A Brief History Of Data
Damien Dallimore
 
From BigTable to HBase and back again
From BigTable to HBase and back againFrom BigTable to HBase and back again
From BigTable to HBase and back again
Leonardo Gamas
 
How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...
How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...
How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...
DevOps.com
 
Lessons learned building a big data analytics engine, from proprietary to ope...
Lessons learned building a big data analytics engine, from proprietary to ope...Lessons learned building a big data analytics engine, from proprietary to ope...
Lessons learned building a big data analytics engine, from proprietary to ope...
J On The Beach
 
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
orcsab
 
MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...
MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...
MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...
MongoDB
 
Oracle Data Science Platform
Oracle Data Science PlatformOracle Data Science Platform
Oracle Data Science Platform
Oracle Developers
 
Hadoop
HadoopHadoop

What's hot (13)

Open source for customer analytics
Open source for customer analyticsOpen source for customer analytics
Open source for customer analytics
 
Eligotech presents @ Data Donderdag on 24 April 2014
Eligotech presents @ Data Donderdag on 24 April 2014Eligotech presents @ Data Donderdag on 24 April 2014
Eligotech presents @ Data Donderdag on 24 April 2014
 
Big Data Overview/Teaser (6 Aug 2013)
Big Data Overview/Teaser (6 Aug 2013)Big Data Overview/Teaser (6 Aug 2013)
Big Data Overview/Teaser (6 Aug 2013)
 
State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...
State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...
State of the Podcast Indutry 2018 by Triton Digital @ MAC PodcastINOUT / Webi...
 
Big data, Hadoop - lunchtime talk 2015.02.26
Big data, Hadoop - lunchtime talk 2015.02.26Big data, Hadoop - lunchtime talk 2015.02.26
Big data, Hadoop - lunchtime talk 2015.02.26
 
A Brief History Of Data
A Brief History Of DataA Brief History Of Data
A Brief History Of Data
 
From BigTable to HBase and back again
From BigTable to HBase and back againFrom BigTable to HBase and back again
From BigTable to HBase and back again
 
How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...
How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...
How to Gain a Competitive Edge with an Open Source, Purpose-built Time Series...
 
Lessons learned building a big data analytics engine, from proprietary to ope...
Lessons learned building a big data analytics engine, from proprietary to ope...Lessons learned building a big data analytics engine, from proprietary to ope...
Lessons learned building a big data analytics engine, from proprietary to ope...
 
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
Big Data Overview for Chinese University of Hong Kong Centre for Innovation a...
 
MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...
MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...
MongoDB IoT City Tour STUTTGART: Why your Dad's database won't work for IoT. ...
 
Oracle Data Science Platform
Oracle Data Science PlatformOracle Data Science Platform
Oracle Data Science Platform
 
Hadoop
HadoopHadoop
Hadoop
 

Viewers also liked

Encendre leds amb fruita segona part
Encendre leds amb fruita segona partEncendre leds amb fruita segona part
Encendre leds amb fruita segona part
lestel2015
 
2014 2015 ieee java projects titles list globalsoft technologies
2014 2015 ieee java projects titles list globalsoft technologies2014 2015 ieee java projects titles list globalsoft technologies
2014 2015 ieee java projects titles list globalsoft technologies
IEEEMATLABPROJECTS
 
多媒體10314142
多媒體10314142多媒體10314142
多媒體10314142
竣鋒 王
 
Media Sebagai Aktor Politik
Media Sebagai Aktor PolitikMedia Sebagai Aktor Politik
Media Sebagai Aktor Politik
Pindai Media
 
manual de instalacion y configuracion del servidor DHCP y manual de configura...
manual de instalacion y configuracion del servidor DHCP y manual de configura...manual de instalacion y configuracion del servidor DHCP y manual de configura...
manual de instalacion y configuracion del servidor DHCP y manual de configura...
Gris Perez
 
Español Uno
Español UnoEspañol Uno
Español Uno
Julián Guzman
 
2014 2015 ieee matlab power electronics projects titles list globalsoft techn...
2014 2015 ieee matlab power electronics projects titles list globalsoft techn...2014 2015 ieee matlab power electronics projects titles list globalsoft techn...
2014 2015 ieee matlab power electronics projects titles list globalsoft techn...
IEEEJAVAPROJECTS
 
14 tips untuk awet muda buat cewek juga
14 tips untuk awet muda buat cewek juga14 tips untuk awet muda buat cewek juga
14 tips untuk awet muda buat cewek juga
uzaenudin38
 
Designed slide - Định giá BĐS
Designed slide - Định giá BĐSDesigned slide - Định giá BĐS
Designed slide - Định giá BĐS
Vu Tuyen Hoang
 
ТРИЗ для коучей. 14 противоречий коучинга.
ТРИЗ для коучей. 14 противоречий коучинга. ТРИЗ для коучей. 14 противоречий коучинга.
ТРИЗ для коучей. 14 противоречий коучинга.
Tatiana Novoselova
 
Meteor Day Gothenburg
Meteor Day GothenburgMeteor Day Gothenburg
Meteor Day Gothenburg
Oscar Nilsson
 
How to create a new Master Page in SharePoint 2013?
How to create a new Master Page in SharePoint 2013?How to create a new Master Page in SharePoint 2013?
How to create a new Master Page in SharePoint 2013?
Velocity Software
 
"BENVENUTI AD ECOLANDIA"
"BENVENUTI AD ECOLANDIA""BENVENUTI AD ECOLANDIA"
"BENVENUTI AD ECOLANDIA"
Consorzio LaMMA - Corso UdC
 
Supply Chain Management: Technology's Role in Achieving Optimum Performance
Supply Chain Management: Technology's Role in Achieving Optimum PerformanceSupply Chain Management: Technology's Role in Achieving Optimum Performance
Supply Chain Management: Technology's Role in Achieving Optimum Performance
Jump Technologies, Inc.
 
VPS Hosting Providers Mumbai | VPS Hosting
VPS Hosting Providers Mumbai | VPS HostingVPS Hosting Providers Mumbai | VPS Hosting
VPS Hosting Providers Mumbai | VPS Hosting
Swapnil Dighe
 
Spike Jonze Case Study
Spike Jonze Case StudySpike Jonze Case Study
Spike Jonze Case Study
El_Lester
 
Existing Record Label Company Research
Existing Record Label Company Research Existing Record Label Company Research
Existing Record Label Company Research
Matta10
 
File types pro forma
File types pro formaFile types pro forma
File types pro forma
JessStanton_
 

Viewers also liked (18)

Encendre leds amb fruita segona part
Encendre leds amb fruita segona partEncendre leds amb fruita segona part
Encendre leds amb fruita segona part
 
2014 2015 ieee java projects titles list globalsoft technologies
2014 2015 ieee java projects titles list globalsoft technologies2014 2015 ieee java projects titles list globalsoft technologies
2014 2015 ieee java projects titles list globalsoft technologies
 
多媒體10314142
多媒體10314142多媒體10314142
多媒體10314142
 
Media Sebagai Aktor Politik
Media Sebagai Aktor PolitikMedia Sebagai Aktor Politik
Media Sebagai Aktor Politik
 
manual de instalacion y configuracion del servidor DHCP y manual de configura...
manual de instalacion y configuracion del servidor DHCP y manual de configura...manual de instalacion y configuracion del servidor DHCP y manual de configura...
manual de instalacion y configuracion del servidor DHCP y manual de configura...
 
Español Uno
Español UnoEspañol Uno
Español Uno
 
2014 2015 ieee matlab power electronics projects titles list globalsoft techn...
2014 2015 ieee matlab power electronics projects titles list globalsoft techn...2014 2015 ieee matlab power electronics projects titles list globalsoft techn...
2014 2015 ieee matlab power electronics projects titles list globalsoft techn...
 
14 tips untuk awet muda buat cewek juga
14 tips untuk awet muda buat cewek juga14 tips untuk awet muda buat cewek juga
14 tips untuk awet muda buat cewek juga
 
Designed slide - Định giá BĐS
Designed slide - Định giá BĐSDesigned slide - Định giá BĐS
Designed slide - Định giá BĐS
 
ТРИЗ для коучей. 14 противоречий коучинга.
ТРИЗ для коучей. 14 противоречий коучинга. ТРИЗ для коучей. 14 противоречий коучинга.
ТРИЗ для коучей. 14 противоречий коучинга.
 
Meteor Day Gothenburg
Meteor Day GothenburgMeteor Day Gothenburg
Meteor Day Gothenburg
 
How to create a new Master Page in SharePoint 2013?
How to create a new Master Page in SharePoint 2013?How to create a new Master Page in SharePoint 2013?
How to create a new Master Page in SharePoint 2013?
 
"BENVENUTI AD ECOLANDIA"
"BENVENUTI AD ECOLANDIA""BENVENUTI AD ECOLANDIA"
"BENVENUTI AD ECOLANDIA"
 
Supply Chain Management: Technology's Role in Achieving Optimum Performance
Supply Chain Management: Technology's Role in Achieving Optimum PerformanceSupply Chain Management: Technology's Role in Achieving Optimum Performance
Supply Chain Management: Technology's Role in Achieving Optimum Performance
 
VPS Hosting Providers Mumbai | VPS Hosting
VPS Hosting Providers Mumbai | VPS HostingVPS Hosting Providers Mumbai | VPS Hosting
VPS Hosting Providers Mumbai | VPS Hosting
 
Spike Jonze Case Study
Spike Jonze Case StudySpike Jonze Case Study
Spike Jonze Case Study
 
Existing Record Label Company Research
Existing Record Label Company Research Existing Record Label Company Research
Existing Record Label Company Research
 
File types pro forma
File types pro formaFile types pro forma
File types pro forma
 

Recently uploaded

Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
Miro Wengner
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Precisely
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
panagenda
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
shyamraj55
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
Intelisync
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
Edge AI and Vision Alliance
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 

Recently uploaded (20)

Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
JavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green MasterplanJavaLand 2024: Application Development Green Masterplan
JavaLand 2024: Application Development Green Masterplan
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their MainframeDigital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
Digital Banking in the Cloud: How Citizens Bank Unlocked Their Mainframe
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUHCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAU
 
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with SlackLet's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slack
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024A Comprehensive Guide to DeFi Development Services in 2024
A Comprehensive Guide to DeFi Development Services in 2024
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
“Temporal Event Neural Networks: A More Efficient Alternative to the Transfor...
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 

Big data and hadoop lightining talk

  • 1. BIG DATA AND HADOOP Big Data History,Hadoop, and Industry Trends E S T H E R K U N D I N B L O O M B E R G L P
  • 2. BIG DATA – WHAT IS IT?
  • 3. WHAT IS BIG DATA?
  • 4. BIG DATA ORIGINS Indexing the web requires lots of storage - Petabytes of data! Economic problem – reliable servers expensive! Solution:  Cram in as many cheap machines as possible  Replace them when they fail  Solve reliability via software! Google publishes papers about:  GFS (2000)  MapReduce (2004)  BigTable (2006)
  • 5. HADOOP – OPEN SOURCE BIG DATA SOLUTION Hadoop, originally developed at Yahoo, accepted as Apache top-level project in 2008