SlideShare a Scribd company logo
https://buildAR.com
WebRTC opens the door to
The Augmented Web
Who am I?
Rob Manson @nambor
CEO of MOB Labs the creators of buildAR.com
Chair of the W3C's Augmented Web Community Group
Invited Expert with the ISO, W3C & the Khronos Group
Co-Founder of ARStandards.org
Author of “Getting started with WebRTC” (available July)
https://buildAR.com
ARStandards Workshop in Seoul 2010
Patterns of Interest Proposal – Rob Manson
https://buildAR.com
Test your device at theAWEsomeWEB.com
https://buildAR.com
https://buildAR.com
https://buildAR.com
Current local image processing pipeline
using the Media Capture & Streams API
1. Setup <video> element in the DOM
a. declaratively then via getElementById or similar
b. createElement(“video”) then appendChild()
2. Access the camera
a. getUserMedia()
NOTE: Currently can only select the default camera
3. Pipe camera stream into <video>
a. video.src = stream
4. Setup <canvas> element in the DOM
a. declaratively then via getElementById or similar
b. createElement(“canvas”) then appendChild()
5. Get 2D drawing context
a. canvas.getContext('2d');
6. Draw <video> frame onto <canvas>
a. canvas.drawImage(video, top, left, width, height);
7. Get RGBA Uint8ClampedArray of the pixels
a. context.getImageData(top, left, width, height).data;
8. Burn CPU (not GPU) cycles
a. for (blah) { for (blah) { … } … }
NOTE: May also integrate other sensor data here
9. Render results
a. using HTML/JS/CSS
b. using another <canvas> and drawImage()
c. using WebGL
d. a combination of all
https://buildAR.com
JSARToolkit demo
https://buildAR.com
FAST feature detection demo
https://buildAR.com
What's in the near future?
Integrating WebRTC and Visual Search
Using WebGL/GLSL to utilise GPU parallelism
Khronos Group's OpenVX
Khronos Group's Camera Working Group
Lots more demos to share! 8)
https://buildAR.com
You should join the W3C's
Augmented Web Community Group

More Related Content

More from Rob Manson

Immersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeekImmersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeek
Rob Manson
 
The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17
Rob Manson
 
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the webISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
Rob Manson
 
Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017
Rob Manson
 
Immersive - Sydney July 2017
Immersive - Sydney July 2017Immersive - Sydney July 2017
Immersive - Sydney July 2017
Rob Manson
 
Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!
Rob Manson
 
Build your own reality
Build your own realityBuild your own reality
Build your own reality
Rob Manson
 
Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13
Rob Manson
 
The Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the massesThe Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the masses
Rob Manson
 
Augmented Reality is just a feature!
Augmented Reality is just a feature!Augmented Reality is just a feature!
Augmented Reality is just a feature!
Rob Manson
 
Introduction to AR
Introduction to ARIntroduction to AR
Introduction to AR
Rob Manson
 
Web Standards adoption in the AR market
Web Standards adoption in the AR marketWeb Standards adoption in the AR market
Web Standards adoption in the AR market
Rob Manson
 
Now here is everywhere
Now here is everywhereNow here is everywhere
Now here is everywhere
Rob Manson
 
AR is the UI for Pervasive Computing
AR is the UI for Pervasive ComputingAR is the UI for Pervasive Computing
AR is the UI for Pervasive Computing
Rob Manson
 
Your browser can see and hear and ...
Your browser can see and hear and ...Your browser can see and hear and ...
Your browser can see and hear and ...
Rob Manson
 
Web Standards based Augmented Reality
Web Standards based Augmented RealityWeb Standards based Augmented Reality
Web Standards based Augmented Reality
Rob Manson
 
Is that sense-able?
Is that sense-able?Is that sense-able?
Is that sense-able?
Rob Manson
 
What is web3?
What is web3?What is web3?
What is web3?
Rob Manson
 
streetARt case study for ARE2011
streetARt case study for ARE2011streetARt case study for ARE2011
streetARt case study for ARE2011
Rob Manson
 
ARSyd March 2011
ARSyd March 2011ARSyd March 2011
ARSyd March 2011
Rob Manson
 

More from Rob Manson (20)

Immersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeekImmersive Sydney - #WebXRWeek
Immersive Sydney - #WebXRWeek
 
The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17The Extended Reality Landscape #Reality17
The Extended Reality Landscape #Reality17
 
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the webISMAR17 - Augmenting, Mixing & Extending Reality on the web
ISMAR17 - Augmenting, Mixing & Extending Reality on the web
 
Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017Immersive Computing @ #YOWConnected 2017
Immersive Computing @ #YOWConnected 2017
 
Immersive - Sydney July 2017
Immersive - Sydney July 2017Immersive - Sydney July 2017
Immersive - Sydney July 2017
 
Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!Computer Vision - now working
 in over 2 Billion Web Browsers!
Computer Vision - now working
 in over 2 Billion Web Browsers!
 
Build your own reality
Build your own realityBuild your own reality
Build your own reality
 
Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13Web Standards for AR workshop at ISMAR13
Web Standards for AR workshop at ISMAR13
 
The Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the massesThe Augmented Web delivers AR to the masses
The Augmented Web delivers AR to the masses
 
Augmented Reality is just a feature!
Augmented Reality is just a feature!Augmented Reality is just a feature!
Augmented Reality is just a feature!
 
Introduction to AR
Introduction to ARIntroduction to AR
Introduction to AR
 
Web Standards adoption in the AR market
Web Standards adoption in the AR marketWeb Standards adoption in the AR market
Web Standards adoption in the AR market
 
Now here is everywhere
Now here is everywhereNow here is everywhere
Now here is everywhere
 
AR is the UI for Pervasive Computing
AR is the UI for Pervasive ComputingAR is the UI for Pervasive Computing
AR is the UI for Pervasive Computing
 
Your browser can see and hear and ...
Your browser can see and hear and ...Your browser can see and hear and ...
Your browser can see and hear and ...
 
Web Standards based Augmented Reality
Web Standards based Augmented RealityWeb Standards based Augmented Reality
Web Standards based Augmented Reality
 
Is that sense-able?
Is that sense-able?Is that sense-able?
Is that sense-able?
 
What is web3?
What is web3?What is web3?
What is web3?
 
streetARt case study for ARE2011
streetARt case study for ARE2011streetARt case study for ARE2011
streetARt case study for ARE2011
 
ARSyd March 2011
ARSyd March 2011ARSyd March 2011
ARSyd March 2011
 

Recently uploaded

GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
Tomaz Bratanic
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Matthew Sinclair
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
innovationoecd
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
Pixlogix Infotech
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
DianaGray10
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
Neo4j
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
Zilliz
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
Zilliz
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Neo4j
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
SOFTTECHHUB
 

Recently uploaded (20)

GraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracyGraphRAG for Life Science to increase LLM accuracy
GraphRAG for Life Science to increase LLM accuracy
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 202420240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
 
Presentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of GermanyPresentation of the OECD Artificial Intelligence Review of Germany
Presentation of the OECD Artificial Intelligence Review of Germany
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Best 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERPBest 20 SEO Techniques To Improve Website Visibility In SERP
Best 20 SEO Techniques To Improve Website Visibility In SERP
 
UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5UiPath Test Automation using UiPath Test Suite series, part 5
UiPath Test Automation using UiPath Test Suite series, part 5
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 
Building Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and MilvusBuilding Production Ready Search Pipelines with Spark and Milvus
Building Production Ready Search Pipelines with Spark and Milvus
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
Infrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI modelsInfrastructure Challenges in Scaling RAG with Custom AI models
Infrastructure Challenges in Scaling RAG with Custom AI models
 
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
 
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
 

WebRTC opens the door to the Augmented Web

  • 1. https://buildAR.com WebRTC opens the door to The Augmented Web
  • 2. Who am I? Rob Manson @nambor CEO of MOB Labs the creators of buildAR.com Chair of the W3C's Augmented Web Community Group Invited Expert with the ISO, W3C & the Khronos Group Co-Founder of ARStandards.org Author of “Getting started with WebRTC” (available July) https://buildAR.com
  • 3. ARStandards Workshop in Seoul 2010 Patterns of Interest Proposal – Rob Manson https://buildAR.com
  • 4. Test your device at theAWEsomeWEB.com https://buildAR.com
  • 6. https://buildAR.com Current local image processing pipeline using the Media Capture & Streams API 1. Setup <video> element in the DOM a. declaratively then via getElementById or similar b. createElement(“video”) then appendChild() 2. Access the camera a. getUserMedia() NOTE: Currently can only select the default camera 3. Pipe camera stream into <video> a. video.src = stream 4. Setup <canvas> element in the DOM a. declaratively then via getElementById or similar b. createElement(“canvas”) then appendChild() 5. Get 2D drawing context a. canvas.getContext('2d'); 6. Draw <video> frame onto <canvas> a. canvas.drawImage(video, top, left, width, height); 7. Get RGBA Uint8ClampedArray of the pixels a. context.getImageData(top, left, width, height).data; 8. Burn CPU (not GPU) cycles a. for (blah) { for (blah) { … } … } NOTE: May also integrate other sensor data here 9. Render results a. using HTML/JS/CSS b. using another <canvas> and drawImage() c. using WebGL d. a combination of all
  • 9. https://buildAR.com What's in the near future? Integrating WebRTC and Visual Search Using WebGL/GLSL to utilise GPU parallelism Khronos Group's OpenVX Khronos Group's Camera Working Group Lots more demos to share! 8)
  • 10. https://buildAR.com You should join the W3C's Augmented Web Community Group