SlideShare a Scribd company logo
1 of 29
Download to read offline
Teaching a Robot
to Play Angry Birds




 Jason Huggins, Sauce Labs Inc
Jason Huggins
Co-creator, The Selenium Project
Co-founder, CTO, Sauce Labs Inc
         twitter: @hugs
  email: hugs@saucelabs.com
@hugs
bitbeam.org
Motorized Pin Art
pinthing.com
Maker Faire
Gridbeam
The “Clicker”
LEGO Technic
         Compatibility
•   Beam width: 5/16 inches

•   Distance between holes: 8mm

•   Hole diameter: 4.8 mm
Hypothesis:
• Selenium is a software-based robot.
• Selenium’s mission is to mimic and
  automate how users interact with an
  application.
• For mobile, this means handling real
  devices.
• To test mobile, could I take Selenium out of
  the screen and into the real world?
My unscientific method:
•   Electronics: Arduino
• Mechanics: Bitbeam
    - Lego Technic compatible building toy
    - Made from Basswood
    - Designed with 3D tool OpenSCAD
    - Lasercut at TechShop San Francisco

•   Software: Python, OpenCV
BitbeamBot I:
The Angry-Birds Playing
Functional Testing Robot
“Too Slow!”
“Delta!”
BitbeamBot II:
The Faster Angry-Birds Playing
   Functional Testing Robot
Moar Demos!
Links
http://github.com/hugs/bitbeam
http://bitbeam.org
http://pinthing.com
http://gridbeam.biz/
http://saucelabs.com

More Related Content

Similar to Selenium Conf - Robots

ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...
ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...
ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...ITCamp
 
Rapid Prototyping 2015: Its a Mad Mad World
Rapid Prototyping 2015: Its a Mad Mad WorldRapid Prototyping 2015: Its a Mad Mad World
Rapid Prototyping 2015: Its a Mad Mad WorldMarti Gold
 
Notice Me, Senpai! Get Discovered with Creative Technical Content
Notice Me, Senpai! Get Discovered with Creative Technical ContentNotice Me, Senpai! Get Discovered with Creative Technical Content
Notice Me, Senpai! Get Discovered with Creative Technical ContentTomomi Imura
 
CubexMotion - Psychomotor Game Learning Using Leap Motion Technology
CubexMotion - Psychomotor Game Learning Using Leap Motion TechnologyCubexMotion - Psychomotor Game Learning Using Leap Motion Technology
CubexMotion - Psychomotor Game Learning Using Leap Motion TechnologyRizal Asrul Pambudi
 
Hardware Hacking in schools (ACEC2014)
Hardware Hacking in schools (ACEC2014)Hardware Hacking in schools (ACEC2014)
Hardware Hacking in schools (ACEC2014)Dan Bowen
 
Patrick Curry Meet Up
Patrick Curry Meet UpPatrick Curry Meet Up
Patrick Curry Meet UpTsugi
 
Hacker Games & DevSecOps
Hacker Games & DevSecOpsHacker Games & DevSecOps
Hacker Games & DevSecOpslokori
 
The Making of Melody Jams (CAMPFest 2017)
The Making of Melody Jams (CAMPFest 2017)The Making of Melody Jams (CAMPFest 2017)
The Making of Melody Jams (CAMPFest 2017)Jamie Kosoy
 
The future of User Interface
The future of User InterfaceThe future of User Interface
The future of User InterfaceJerome Lacote
 
Iproduct presentation
Iproduct presentationIproduct presentation
Iproduct presentation11009373
 
Iproduct presentation(final)
Iproduct presentation(final)Iproduct presentation(final)
Iproduct presentation(final)11009373
 
Interaction modalities, technologies and tools for interactive art
Interaction modalities, technologies and tools for interactive artInteraction modalities, technologies and tools for interactive art
Interaction modalities, technologies and tools for interactive artJorge Cardoso
 
What you can Learn from a 4 Year Old about Ponies and Mobile UX
What you can Learn from a 4 Year Old about Ponies and Mobile UXWhat you can Learn from a 4 Year Old about Ponies and Mobile UX
What you can Learn from a 4 Year Old about Ponies and Mobile UXAndrew Smyk
 
Bridge between Objective C and MIT Scratch
Bridge between Objective C and MIT ScratchBridge between Objective C and MIT Scratch
Bridge between Objective C and MIT ScratchJunya Ishihara
 
STEM Programming Ideas at the Library.pdf
STEM Programming Ideas at the Library.pdfSTEM Programming Ideas at the Library.pdf
STEM Programming Ideas at the Library.pdfBrian Pichman
 
What I Learned about Mobile UX... The Hard Way
What I Learned about Mobile UX... The Hard WayWhat I Learned about Mobile UX... The Hard Way
What I Learned about Mobile UX... The Hard WayJosh Jeffryes
 
Social Zombies II: Your Friends Need More Brains
Social Zombies II: Your Friends Need More BrainsSocial Zombies II: Your Friends Need More Brains
Social Zombies II: Your Friends Need More BrainsTom Eston
 
SDLC in Blockchain - Ethereum edition
SDLC in Blockchain - Ethereum editionSDLC in Blockchain - Ethereum edition
SDLC in Blockchain - Ethereum editionBeier Cai
 

Similar to Selenium Conf - Robots (20)

ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...
ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...
ITCamp 2017 - Laurent Ellerbach - Bot. You said bot? Let's build a bot then...
 
Mind the Gap: Designing the Space Between Devices - Josh Clark at UXI Studio
Mind the Gap: Designing the Space Between Devices - Josh Clark at UXI StudioMind the Gap: Designing the Space Between Devices - Josh Clark at UXI Studio
Mind the Gap: Designing the Space Between Devices - Josh Clark at UXI Studio
 
Rapid Prototyping 2015: Its a Mad Mad World
Rapid Prototyping 2015: Its a Mad Mad WorldRapid Prototyping 2015: Its a Mad Mad World
Rapid Prototyping 2015: Its a Mad Mad World
 
Notice Me, Senpai! Get Discovered with Creative Technical Content
Notice Me, Senpai! Get Discovered with Creative Technical ContentNotice Me, Senpai! Get Discovered with Creative Technical Content
Notice Me, Senpai! Get Discovered with Creative Technical Content
 
CubexMotion - Psychomotor Game Learning Using Leap Motion Technology
CubexMotion - Psychomotor Game Learning Using Leap Motion TechnologyCubexMotion - Psychomotor Game Learning Using Leap Motion Technology
CubexMotion - Psychomotor Game Learning Using Leap Motion Technology
 
Hardware Hacking in schools (ACEC2014)
Hardware Hacking in schools (ACEC2014)Hardware Hacking in schools (ACEC2014)
Hardware Hacking in schools (ACEC2014)
 
Patrick Curry Meet Up
Patrick Curry Meet UpPatrick Curry Meet Up
Patrick Curry Meet Up
 
Hacker Games & DevSecOps
Hacker Games & DevSecOpsHacker Games & DevSecOps
Hacker Games & DevSecOps
 
The Making of Melody Jams (CAMPFest 2017)
The Making of Melody Jams (CAMPFest 2017)The Making of Melody Jams (CAMPFest 2017)
The Making of Melody Jams (CAMPFest 2017)
 
The future of User Interface
The future of User InterfaceThe future of User Interface
The future of User Interface
 
Iproduct presentation
Iproduct presentationIproduct presentation
Iproduct presentation
 
Iproduct presentation(final)
Iproduct presentation(final)Iproduct presentation(final)
Iproduct presentation(final)
 
Interaction modalities, technologies and tools for interactive art
Interaction modalities, technologies and tools for interactive artInteraction modalities, technologies and tools for interactive art
Interaction modalities, technologies and tools for interactive art
 
What you can Learn from a 4 Year Old about Ponies and Mobile UX
What you can Learn from a 4 Year Old about Ponies and Mobile UXWhat you can Learn from a 4 Year Old about Ponies and Mobile UX
What you can Learn from a 4 Year Old about Ponies and Mobile UX
 
Bridge between Objective C and MIT Scratch
Bridge between Objective C and MIT ScratchBridge between Objective C and MIT Scratch
Bridge between Objective C and MIT Scratch
 
Maker Boot Camp
Maker Boot CampMaker Boot Camp
Maker Boot Camp
 
STEM Programming Ideas at the Library.pdf
STEM Programming Ideas at the Library.pdfSTEM Programming Ideas at the Library.pdf
STEM Programming Ideas at the Library.pdf
 
What I Learned about Mobile UX... The Hard Way
What I Learned about Mobile UX... The Hard WayWhat I Learned about Mobile UX... The Hard Way
What I Learned about Mobile UX... The Hard Way
 
Social Zombies II: Your Friends Need More Brains
Social Zombies II: Your Friends Need More BrainsSocial Zombies II: Your Friends Need More Brains
Social Zombies II: Your Friends Need More Brains
 
SDLC in Blockchain - Ethereum edition
SDLC in Blockchain - Ethereum editionSDLC in Blockchain - Ethereum edition
SDLC in Blockchain - Ethereum edition
 

More from hugs

Selenium 2 - PyCon 2011
Selenium 2 - PyCon 2011Selenium 2 - PyCon 2011
Selenium 2 - PyCon 2011hugs
 
The Secret Sauce in the Open Cloud
The Secret Sauce in the Open CloudThe Secret Sauce in the Open Cloud
The Secret Sauce in the Open Cloudhugs
 
Selenium and Sauce Labs
Selenium and Sauce LabsSelenium and Sauce Labs
Selenium and Sauce Labshugs
 
Orbison Orb - Pycon 2010
Orbison Orb - Pycon 2010Orbison Orb - Pycon 2010
Orbison Orb - Pycon 2010hugs
 
Node.js - JavaScript Chicago Meetup
Node.js - JavaScript Chicago MeetupNode.js - JavaScript Chicago Meetup
Node.js - JavaScript Chicago Meetuphugs
 
WTFV is the new RTFM
WTFV is the new RTFMWTFV is the new RTFM
WTFV is the new RTFMhugs
 
The Future Of Web App Testing and How To Stop It
The Future Of Web App Testing and How To Stop ItThe Future Of Web App Testing and How To Stop It
The Future Of Web App Testing and How To Stop Ithugs
 
Hacking Selenium @ JSConf
Hacking Selenium @ JSConfHacking Selenium @ JSConf
Hacking Selenium @ JSConfhugs
 

More from hugs (8)

Selenium 2 - PyCon 2011
Selenium 2 - PyCon 2011Selenium 2 - PyCon 2011
Selenium 2 - PyCon 2011
 
The Secret Sauce in the Open Cloud
The Secret Sauce in the Open CloudThe Secret Sauce in the Open Cloud
The Secret Sauce in the Open Cloud
 
Selenium and Sauce Labs
Selenium and Sauce LabsSelenium and Sauce Labs
Selenium and Sauce Labs
 
Orbison Orb - Pycon 2010
Orbison Orb - Pycon 2010Orbison Orb - Pycon 2010
Orbison Orb - Pycon 2010
 
Node.js - JavaScript Chicago Meetup
Node.js - JavaScript Chicago MeetupNode.js - JavaScript Chicago Meetup
Node.js - JavaScript Chicago Meetup
 
WTFV is the new RTFM
WTFV is the new RTFMWTFV is the new RTFM
WTFV is the new RTFM
 
The Future Of Web App Testing and How To Stop It
The Future Of Web App Testing and How To Stop ItThe Future Of Web App Testing and How To Stop It
The Future Of Web App Testing and How To Stop It
 
Hacking Selenium @ JSConf
Hacking Selenium @ JSConfHacking Selenium @ JSConf
Hacking Selenium @ JSConf
 

Recently uploaded

Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAndrey Devyatkin
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesBoston Institute of Analytics
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century educationjfdjdjcjdnsjd
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 

Recently uploaded (20)

Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
HTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation StrategiesHTML Injection Attacks: Impact and Mitigation Strategies
HTML Injection Attacks: Impact and Mitigation Strategies
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 

Selenium Conf - Robots