SlideShare a Scribd company logo

H is for_hadoop

•Download as PPT, PDF•
•
Steve Loughran
Steve Loughran

Introductory talk from 2008

TechnologyEducation
1 of 18
September 2008 H is for Hadoop Steve Loughran [email_address] Julio Guijarro [email_address]
What is Hadoop? September 2008
A yellow elephant September 2008
A use for a datacentre September 2008
Hadoop is behind Yahoo! ,[object Object],[object Object],[object Object],[object Object],source: Eric Baldeschwieler, OSCON, July 25 2007
Java Cloud Computing Edition ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],September 2008
MapReduce ,[object Object],[object Object],[object Object],[object Object],[object Object],September 2008
Example problem: Bluetooth phones September 2008 ,[object Object],[object Object],[object Object],[object Object],lost,"00:0F:B3:92:05:D3","2008-04-17T22:11:15",1124313075 found,"00:0F:B3:92:05:D3","2008-04-17T22:11:29",1124313089 lost,"00:0F:B3:92:05:D3","2008-04-17T22:24:45",1124313885 found,"00:0F:B3:92:05:D3","2008-04-17T22:25:00",1124313900 found,"00:60:57:70:25:0F","2008-04-17T22:29:00",1124314140
Datacentre View September 2008
old world: September 2008 Java EE
September 2008
Layers on Top September 2008 Pig ( from Pig Latin) MapReduce query language Hive SQL against the data (facebook) HBase non-relational database Mahout Machine Learning Distributed Lucene Search over HDFS Hama Mathematics
Limitations of Hadoop ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],September 2008
What to do? ,[object Object],[object Object],[object Object],[object Object],[object Object],September 2008 http://hadoop.apache.org
what are we up to? September 2008
September 2008 ,[object Object],[object Object]
Around Hadoop ... with SmartFrog now Hardware Hadoop Vertical applications Management, Monitoring, Virtualization
 

Recommended

Metadata For Humans and Machines
Metadata For Humans and MachinesMetadata For Humans and Machines
Metadata For Humans and MachinesTed Habermann
 
ESDIS Metadata Archive
ESDIS Metadata ArchiveESDIS Metadata Archive
ESDIS Metadata ArchiveTed Habermann
 
Guided Tour of Pythonian Museum
Guided Tour of Pythonian MuseumGuided Tour of Pythonian Museum
Guided Tour of Pythonian MuseumThe HDF-EOS Tools and Information Center
 
Overview of slider project
Overview of slider projectOverview of slider project
Overview of slider projectSteve Loughran
 
Battle At Goliad
Battle At GoliadBattle At Goliad
Battle At Goliadcompd
 
Economic Scheduling of Hadoop Jobs
Economic Scheduling of Hadoop JobsEconomic Scheduling of Hadoop Jobs
Economic Scheduling of Hadoop JobsSteve Loughran
 
2013 11-19-hoya-status
2013 11-19-hoya-status2013 11-19-hoya-status
2013 11-19-hoya-statusSteve Loughran
 
Extended essay overview
Extended essay overviewExtended essay overview
Extended essay overviewMary Alice Osborne
 

Recommended

Metadata For Humans and Machines
Metadata For Humans and MachinesMetadata For Humans and Machines
Metadata For Humans and MachinesTed Habermann
 
ESDIS Metadata Archive
ESDIS Metadata ArchiveESDIS Metadata Archive
ESDIS Metadata ArchiveTed Habermann
 
Guided Tour of Pythonian Museum
Guided Tour of Pythonian MuseumGuided Tour of Pythonian Museum
Guided Tour of Pythonian MuseumThe HDF-EOS Tools and Information Center
 
Overview of slider project
Overview of slider projectOverview of slider project
Overview of slider projectSteve Loughran
 
Battle At Goliad
Battle At GoliadBattle At Goliad
Battle At Goliadcompd
 
Economic Scheduling of Hadoop Jobs
Economic Scheduling of Hadoop JobsEconomic Scheduling of Hadoop Jobs
Economic Scheduling of Hadoop JobsSteve Loughran
 
2013 11-19-hoya-status
2013 11-19-hoya-status2013 11-19-hoya-status
2013 11-19-hoya-statusSteve Loughran
 
Extended essay overview
Extended essay overviewExtended essay overview
Extended essay overviewMary Alice Osborne
 
HDP-1 introduction for HUG France
HDP-1 introduction for HUG FranceHDP-1 introduction for HUG France
HDP-1 introduction for HUG FranceSteve Loughran
 
Taming Deployment With Smart Frog
Taming Deployment With Smart FrogTaming Deployment With Smart Frog
Taming Deployment With Smart FrogSteve Loughran
 
My other computer_is_a_datacentre
My other computer_is_a_datacentreMy other computer_is_a_datacentre
My other computer_is_a_datacentreSteve Loughran
 
Farms, Fabrics and Clouds
Farms, Fabrics and CloudsFarms, Fabrics and Clouds
Farms, Fabrics and CloudsSteve Loughran
 
A New Approach To Organization
A New Approach To OrganizationA New Approach To Organization
A New Approach To Organizationcompd
 
Graphs
GraphsGraphs
GraphsSteve Loughran
 
Scholarly articles
Scholarly articlesScholarly articles
Scholarly articlesMary Alice Osborne
 
Echolocation
EcholocationEcholocation
EcholocationMary Alice Osborne
 
Did you really want that data?
Did you really want that data?Did you really want that data?
Did you really want that data?Steve Loughran
 
RESTFul IDEAS
RESTFul IDEASRESTFul IDEAS
RESTFul IDEASJoel Amoussou
 
BBC Programmes and Music on the Linking Open Data Cloud
BBC Programmes and Music on the Linking Open Data CloudBBC Programmes and Music on the Linking Open Data Cloud
BBC Programmes and Music on the Linking Open Data CloudPatrick Sinclair
 
Is There Room For Another Elephant In Tucson
Is There Room For Another Elephant In TucsonIs There Room For Another Elephant In Tucson
Is There Room For Another Elephant In TucsonAndy Lenards
 
GTAC: AtomPub, testing your server implementation
GTAC: AtomPub, testing your server implementationGTAC: AtomPub, testing your server implementation
GTAC: AtomPub, testing your server implementationDavid Calavera
 
The (near) future of personal computers
The (near) future of personal computersThe (near) future of personal computers
The (near) future of personal computersKevin Goldsmith
 
FOWA 09 - Open Strategy Applied
FOWA 09 - Open Strategy AppliedFOWA 09 - Open Strategy Applied
FOWA 09 - Open Strategy AppliedDan Theurer
 
Yahoo - Open Applied
Yahoo - Open AppliedYahoo - Open Applied
Yahoo - Open AppliedCarsonified Team
 
Web::Scraper
Web::ScraperWeb::Scraper
Web::ScraperTatsuhiko Miyagawa
 
Lessons Learned - Building YDN
Lessons Learned - Building YDNLessons Learned - Building YDN
Lessons Learned - Building YDNDan Theurer
 
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas NApache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas NYahoo Developer Network
 
07 Collada Overview
07 Collada Overview07 Collada Overview
07 Collada Overviewjohny2008
 
HTML5 - What h#@$ is it?
HTML5 - What h#@$ is it?HTML5 - What h#@$ is it?
HTML5 - What h#@$ is it?Carlos Ramon
 
Beyond the Node: Arkestration with Noah
Beyond the Node: Arkestration with NoahBeyond the Node: Arkestration with Noah
Beyond the Node: Arkestration with Noahlusis
 

More Related Content

Viewers also liked

HDP-1 introduction for HUG France
HDP-1 introduction for HUG FranceHDP-1 introduction for HUG France
HDP-1 introduction for HUG FranceSteve Loughran
 
Taming Deployment With Smart Frog
Taming Deployment With Smart FrogTaming Deployment With Smart Frog
Taming Deployment With Smart FrogSteve Loughran
 
My other computer_is_a_datacentre
My other computer_is_a_datacentreMy other computer_is_a_datacentre
My other computer_is_a_datacentreSteve Loughran
 
Farms, Fabrics and Clouds
Farms, Fabrics and CloudsFarms, Fabrics and Clouds
Farms, Fabrics and CloudsSteve Loughran
 
A New Approach To Organization
A New Approach To OrganizationA New Approach To Organization
A New Approach To Organizationcompd
 
Did you really want that data?
Did you really want that data?Did you really want that data?
Did you really want that data?Steve Loughran
 

Viewers also liked (9)

HDP-1 introduction for HUG France
HDP-1 introduction for HUG FranceHDP-1 introduction for HUG France
HDP-1 introduction for HUG France
 
Taming Deployment With Smart Frog
Taming Deployment With Smart FrogTaming Deployment With Smart Frog
Taming Deployment With Smart Frog
 
My other computer_is_a_datacentre
My other computer_is_a_datacentreMy other computer_is_a_datacentre
My other computer_is_a_datacentre
 
Farms, Fabrics and Clouds
Farms, Fabrics and CloudsFarms, Fabrics and Clouds
Farms, Fabrics and Clouds
 
A New Approach To Organization
A New Approach To OrganizationA New Approach To Organization
A New Approach To Organization
 
Graphs
GraphsGraphs
Graphs
 
Scholarly articles
Scholarly articlesScholarly articles
Scholarly articles
 
Echolocation
EcholocationEcholocation
Echolocation
 
Did you really want that data?
Did you really want that data?Did you really want that data?
Did you really want that data?
 

Similar to H is for_hadoop

BBC Programmes and Music on the Linking Open Data Cloud
BBC Programmes and Music on the Linking Open Data CloudBBC Programmes and Music on the Linking Open Data Cloud
BBC Programmes and Music on the Linking Open Data CloudPatrick Sinclair
 
Is There Room For Another Elephant In Tucson
Is There Room For Another Elephant In TucsonIs There Room For Another Elephant In Tucson
Is There Room For Another Elephant In TucsonAndy Lenards
 
GTAC: AtomPub, testing your server implementation
GTAC: AtomPub, testing your server implementationGTAC: AtomPub, testing your server implementation
GTAC: AtomPub, testing your server implementationDavid Calavera
 
The (near) future of personal computers
The (near) future of personal computersThe (near) future of personal computers
The (near) future of personal computersKevin Goldsmith
 
FOWA 09 - Open Strategy Applied
FOWA 09 - Open Strategy AppliedFOWA 09 - Open Strategy Applied
FOWA 09 - Open Strategy AppliedDan Theurer
 
Lessons Learned - Building YDN
Lessons Learned - Building YDNLessons Learned - Building YDN
Lessons Learned - Building YDNDan Theurer
 
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas NApache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas NYahoo Developer Network
 
07 Collada Overview
07 Collada Overview07 Collada Overview
07 Collada Overviewjohny2008
 
HTML5 - What h#@$ is it?
HTML5 - What h#@$ is it?HTML5 - What h#@$ is it?
HTML5 - What h#@$ is it?Carlos Ramon
 
Beyond the Node: Arkestration with Noah
Beyond the Node: Arkestration with NoahBeyond the Node: Arkestration with Noah
Beyond the Node: Arkestration with Noahlusis
 
Evolution of API With Blogging
Evolution of API With BloggingEvolution of API With Blogging
Evolution of API With BloggingTakatsugu Shigeta
 
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office HoursIVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office HoursAmazon Web Services Japan
 
Cassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at Ooyala
Cassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at OoyalaCassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at Ooyala
Cassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at OoyalaDataStax Academy
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud
 
Butter Web Browsing with Margarine
Butter Web Browsing with MargarineButter Web Browsing with Margarine
Butter Web Browsing with MargarineWayne Walls
 

Similar to H is for_hadoop (20)

RESTFul IDEAS
RESTFul IDEASRESTFul IDEAS
RESTFul IDEAS
 
BBC Programmes and Music on the Linking Open Data Cloud
BBC Programmes and Music on the Linking Open Data CloudBBC Programmes and Music on the Linking Open Data Cloud
BBC Programmes and Music on the Linking Open Data Cloud
 
Is There Room For Another Elephant In Tucson
Is There Room For Another Elephant In TucsonIs There Room For Another Elephant In Tucson
Is There Room For Another Elephant In Tucson
 
GTAC: AtomPub, testing your server implementation
GTAC: AtomPub, testing your server implementationGTAC: AtomPub, testing your server implementation
GTAC: AtomPub, testing your server implementation
 
The (near) future of personal computers
The (near) future of personal computersThe (near) future of personal computers
The (near) future of personal computers
 
FOWA 09 - Open Strategy Applied
FOWA 09 - Open Strategy AppliedFOWA 09 - Open Strategy Applied
FOWA 09 - Open Strategy Applied
 
Yahoo - Open Applied
Yahoo - Open AppliedYahoo - Open Applied
Yahoo - Open Applied
 
Web::Scraper
Web::ScraperWeb::Scraper
Web::Scraper
 
Lessons Learned - Building YDN
Lessons Learned - Building YDNLessons Learned - Building YDN
Lessons Learned - Building YDN
 
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas NApache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
Apache Hadoop India Summit 2011 talk "Oozie - Workflow for Hadoop" by Andreas N
 
07 Collada Overview
07 Collada Overview07 Collada Overview
07 Collada Overview
 
HTML5 - What h#@$ is it?
HTML5 - What h#@$ is it?HTML5 - What h#@$ is it?
HTML5 - What h#@$ is it?
 
Beyond the Node: Arkestration with Noah
Beyond the Node: Arkestration with NoahBeyond the Node: Arkestration with Noah
Beyond the Node: Arkestration with Noah
 
Evolution of API With Blogging
Evolution of API With BloggingEvolution of API With Blogging
Evolution of API With Blogging
 
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office HoursIVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
IVS CTO Night And Day 2018 Winter - AWS Startup Tech Office Hours
 
Cassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at Ooyala
Cassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at OoyalaCassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at Ooyala
Cassandra Meetup: Real-time Analytics using Cassandra, Spark and Shark at Ooyala
 
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
BigDataCloud meetup - July 8th - Cost effective big-data processing using Ama...
 
How we build Vox
How we build VoxHow we build Vox
How we build Vox
 
20090422 Www
20090422 Www20090422 Www
20090422 Www
 
Butter Web Browsing with Margarine
Butter Web Browsing with MargarineButter Web Browsing with Margarine
Butter Web Browsing with Margarine
 

More from Steve Loughran

The age of rename() is over
The age of rename() is overThe age of rename() is over
The age of rename() is overSteve Loughran
 
What does Rename Do: (detailed version)
What does Rename Do: (detailed version)What does Rename Do: (detailed version)
What does Rename Do: (detailed version)Steve Loughran
 
Put is the new rename: San Jose Summit Edition
Put is the new rename: San Jose Summit EditionPut is the new rename: San Jose Summit Edition
Put is the new rename: San Jose Summit EditionSteve Loughran
 
@Dissidentbot: dissent will be automated!
@Dissidentbot: dissent will be automated!@Dissidentbot: dissent will be automated!
@Dissidentbot: dissent will be automated!Steve Loughran
 
PUT is the new rename()
PUT is the new rename()PUT is the new rename()
PUT is the new rename()Steve Loughran
 
Extreme Programming Deployed
Extreme Programming DeployedExtreme Programming Deployed
Extreme Programming DeployedSteve Loughran
 
What does rename() do?
What does rename() do?What does rename() do?
What does rename() do?Steve Loughran
 
Dancing Elephants: Working with Object Storage in Apache Spark and Hive
Dancing Elephants: Working with Object Storage in Apache Spark and HiveDancing Elephants: Working with Object Storage in Apache Spark and Hive
Dancing Elephants: Working with Object Storage in Apache Spark and HiveSteve Loughran
 
Apache Spark and Object Stores —for London Spark User Group
Apache Spark and Object Stores —for London Spark User GroupApache Spark and Object Stores —for London Spark User Group
Apache Spark and Object Stores —for London Spark User GroupSteve Loughran
 
Spark Summit East 2017: Apache spark and object stores
Spark Summit East 2017: Apache spark and object storesSpark Summit East 2017: Apache spark and object stores
Spark Summit East 2017: Apache spark and object storesSteve Loughran
 
Hadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object StoresHadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object StoresSteve Loughran
 
Apache Spark and Object Stores
Apache Spark and Object StoresApache Spark and Object Stores
Apache Spark and Object StoresSteve Loughran
 
Household INFOSEC in a Post-Sony Era
Household INFOSEC in a Post-Sony EraHousehold INFOSEC in a Post-Sony Era
Household INFOSEC in a Post-Sony EraSteve Loughran
 
Hadoop and Kerberos: the Madness Beyond the Gate: January 2016 edition
Hadoop and Kerberos: the Madness Beyond the Gate: January 2016 editionHadoop and Kerberos: the Madness Beyond the Gate: January 2016 edition
Hadoop and Kerberos: the Madness Beyond the Gate: January 2016 editionSteve Loughran
 
Hadoop and Kerberos: the Madness Beyond the Gate
Hadoop and Kerberos: the Madness Beyond the GateHadoop and Kerberos: the Madness Beyond the Gate
Hadoop and Kerberos: the Madness Beyond the GateSteve Loughran
 
Slider: Applications on YARN
Slider: Applications on YARNSlider: Applications on YARN
Slider: Applications on YARNSteve Loughran
 

More from Steve Loughran (20)

Hadoop Vectored IO
Hadoop Vectored IOHadoop Vectored IO
Hadoop Vectored IO
 
The age of rename() is over
The age of rename() is overThe age of rename() is over
The age of rename() is over
 
What does Rename Do: (detailed version)
What does Rename Do: (detailed version)What does Rename Do: (detailed version)
What does Rename Do: (detailed version)
 
Put is the new rename: San Jose Summit Edition
Put is the new rename: San Jose Summit EditionPut is the new rename: San Jose Summit Edition
Put is the new rename: San Jose Summit Edition
 
@Dissidentbot: dissent will be automated!
@Dissidentbot: dissent will be automated!@Dissidentbot: dissent will be automated!
@Dissidentbot: dissent will be automated!
 
PUT is the new rename()
PUT is the new rename()PUT is the new rename()
PUT is the new rename()
 
Extreme Programming Deployed
Extreme Programming DeployedExtreme Programming Deployed
Extreme Programming Deployed
 
Testing
TestingTesting
Testing
 
I hate mocking
I hate mockingI hate mocking
I hate mocking
 
What does rename() do?
What does rename() do?What does rename() do?
What does rename() do?
 
Dancing Elephants: Working with Object Storage in Apache Spark and Hive
Dancing Elephants: Working with Object Storage in Apache Spark and HiveDancing Elephants: Working with Object Storage in Apache Spark and Hive
Dancing Elephants: Working with Object Storage in Apache Spark and Hive
 
Apache Spark and Object Stores —for London Spark User Group
Apache Spark and Object Stores —for London Spark User GroupApache Spark and Object Stores —for London Spark User Group
Apache Spark and Object Stores —for London Spark User Group
 
Spark Summit East 2017: Apache spark and object stores
Spark Summit East 2017: Apache spark and object storesSpark Summit East 2017: Apache spark and object stores
Spark Summit East 2017: Apache spark and object stores
 
Hadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object StoresHadoop, Hive, Spark and Object Stores
Hadoop, Hive, Spark and Object Stores
 
Apache Spark and Object Stores
Apache Spark and Object StoresApache Spark and Object Stores
Apache Spark and Object Stores
 
Household INFOSEC in a Post-Sony Era
Household INFOSEC in a Post-Sony EraHousehold INFOSEC in a Post-Sony Era
Household INFOSEC in a Post-Sony Era
 
Hadoop and Kerberos: the Madness Beyond the Gate: January 2016 edition
Hadoop and Kerberos: the Madness Beyond the Gate: January 2016 editionHadoop and Kerberos: the Madness Beyond the Gate: January 2016 edition
Hadoop and Kerberos: the Madness Beyond the Gate: January 2016 edition
 
Hadoop and Kerberos: the Madness Beyond the Gate
Hadoop and Kerberos: the Madness Beyond the GateHadoop and Kerberos: the Madness Beyond the Gate
Hadoop and Kerberos: the Madness Beyond the Gate
 
Slider: Applications on YARN
Slider: Applications on YARNSlider: Applications on YARN
Slider: Applications on YARN
 
YARN Services
YARN ServicesYARN Services
YARN Services
 

Recently uploaded

How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?XfilesPro
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024BookNet Canada
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticscarlostorres15106
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsMemoori
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024Scott Keck-Warren
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxnull - The Open Security Community
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphNeo4j
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAndikSusilo4
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraDeakin University
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Key Features Of Token Development (1).pptx
Key Features Of Token Development (1).pptxKey Features Of Token Development (1).pptx
Key Features Of Token Development (1).pptxLBM Solutions
 

Recently uploaded (20)

How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?How to Remove Document Management Hurdles with X-Docs?
How to Remove Document Management Hurdles with X-Docs?
 
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
Transcript: #StandardsGoals for 2024: What’s new for BISAC - Tech Forum 2024
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmaticsKotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
Kotlin Multiplatform & Compose Multiplatform - Starter kit for pragmatics
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
AI as an Interface for Commercial Buildings
AI as an Interface for Commercial BuildingsAI as an Interface for Commercial Buildings
AI as an Interface for Commercial Buildings
 
SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024SQL Database Design For Developers at php[tek] 2024
SQL Database Design For Developers at php[tek] 2024
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptxMaking_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
Making_way_through_DLL_hollowing_inspite_of_CFG_by_Debjeet Banerjee.pptx
 
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptxVulnerability_Management_GRC_by Sohang Sengupta.pptx
Vulnerability_Management_GRC_by Sohang Sengupta.pptx
 
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge GraphSIEMENS: RAPUNZEL – A Tale About Knowledge Graph
SIEMENS: RAPUNZEL – A Tale About Knowledge Graph
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
Azure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & ApplicationAzure Monitor & Application Insight to monitor Infrastructure & Application
Azure Monitor & Application Insight to monitor Infrastructure & Application
 
Pigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food ManufacturingPigging Solutions in Pet Food Manufacturing
Pigging Solutions in Pet Food Manufacturing
 
Artificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning eraArtificial intelligence in the post-deep learning era
Artificial intelligence in the post-deep learning era
 
Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Key Features Of Token Development (1).pptx
Key Features Of Token Development (1).pptxKey Features Of Token Development (1).pptx
Key Features Of Token Development (1).pptx
 

H is for_hadoop

Editor's Notes

  1. September 2008
  2. This is one of Doug Cutting's kid's toys. It is what Hadoop is named after.
  3. Lots of people have those google "our other computer is a datacentre" stickers. We just know where ours is and what runs on it. Soon: our machines will have hadoop stickers on them.
  4. search for the google mapreduce paper
  5. Everyone talks about word counting and click logs, here's something more fun. mapping out all devices with the same ID, debouncing, building up stats. Or by time: which are peak times; are there special groups (school, college students) who can be identified by timings and dates of travel?
  6. The namenode really matters -lose that and the cluster is gone Lose the job tracker and all ongoing work is lost, currently the entire job chain needs to be restarted. But the rest of the cluster says live.
  7. This is how applications used to be written. An App server driving a cluster, a database (Oracle, IBM DB/2 or something else) behind it all. An O/R mapping infrastructure, either entity beans or spring/hibernate to make Java classes persistent, and JSP front ends. On the side: Message beans for queued operations, Corba IIOP or Java WS-* to talk to other parts of the enterprise.
  8. Here are where things get interesting
  9. We may have our own cluster, but it isnt a stable one. We need to adapt to what gets provided to us.