SlideShare a Scribd company logo

A TurtleBot Configurations Measurement Harness to Build a Sensitivity Model

•
•
M
Miguel Velez

Miguel Velez proposes building a sensitivity model to analyze how different configurations of a TurtleBot's sensors and localization algorithms affect its performance. He plans to systematically test combinations of over 25 numeric parameters across different environments and measure their impact on localization error, CPU usage, and time. His infrastructure will distribute experiments across multiple machines to handle the large configuration space. The expected results are a sensitivity model that identifies configurations that balance localization and efficiency to help self-adaptive systems automatically optimize performance.

Software
1 of 32
Download to read offline
A TurtleBot Configurations Measurement Harness to Build a Sensitivity Model Miguel Velez Software Engineering Ph.D. student 1
Personal Background Quito, Ecuador University of St. Thomas Computer Science and Physics Sportradar US CMU advisor: Christian Kästner 2
Broad Picture 3 X X Localization Power Localization Power
TurtleBot Sensor Sensor 4 Sensitivity model Self- adaptation
Sensitivity model Self- adaptation TurtleBot Sensor Sensor 5 Ideas?
Variables Independent AMCL Environment ~25 numeric configurations (i.e. min_particles, max_particles, update_min_a) Kinect noise Kinect miscalibration Odometry noise Odometry miscalibration Dependent Time CPU Utilization Localization Error 6
Expected Results 7
Analyzing Configurations 8
Sensitivity Model Difficult! Exponential configuration space 25 configurations with 10 options each 1025 total points * 2 min = ~109 * universe age 9
Dealing with Large Space 10
Dealing with Large Space 11
Dealing with Large Space 12 Default
Dealing with Large Space 13 Default
Dealing with Large Space 14 Default
Dealing with Large Space 15 No difference Difference
Experiments 16
Simulation TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc LocalizationTime 17 CPU Simulator Process
Simulation Wean 4th floor 18 (0,0) (7,-10.5)
Challenges 19
Data Synchronization Problem: Different topic publish rate 20 Solution: Get data every second
Simulator uses most CPU CPU Utilization 21 Run on separate machines CPU Simulator TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp LocalizationTime CPU ( ( X Process Computer Simulator ( ( TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc CPU
Infrastructure feature4.andrew.cmu.edu TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc feature3.andrew.cmu.edu ROS_MASTER_URI db job data job data job data job data job data Master run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh feature.isri.cmu.edu Fetch Save Execute 22 Process Computer Simulator ROS_MASTER_URI
Infrastructure featureY.andrew.cmu.edu TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc featureX.andrew.cmu.edu db job data job data job data job data job data Master run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh feature.isri.cmu.edu 23 Process Computer Simulator featureW.andrew.cmu.edu TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc featureZ.andrew.cmu.edu Simulator
Infrastructure 24 ~2 min/job 1 server 360 jobs in 12h 36 individual options 144 options with 4 servers ~14 configurations with 10 options each Can measure other metrics
Results 25
Results 26
Results 27
Lessons Learned Few configurations affect CPU and localization Best configurations around default values AMCL highly adaptive and robust Some options can balance CPU and localization 28
Lessons Learned Some options can balance CPU and localization 29
Future Work Explore environmental configurations Kinect and Odometry Remap? White-box analysis of AMCL Combine configurations Build sensitivity model Measure other metrics? 30
Summary • Sensitivity model needed in self-adaptive systems • Infrastructure for automatic measurement • Suggestions for problems and ideas for infrastructure 31
Questions? 32

Recommended

Low Overhead System Tracing with eBPF
Low Overhead System Tracing with eBPFLow Overhead System Tracing with eBPF
Low Overhead System Tracing with eBPFAkshay Kapoor
 
PaaS祭り - pagoda box
PaaS祭り - pagoda boxPaaS祭り - pagoda box
PaaS祭り - pagoda boxTsutomu Kawamura
 
异步io框架的实现
异步io框架的实现异步io框架的实现
异步io框架的实现rfyiamcool
 
深層学習フレームワークにおけるIntel CPU/富岳向け最適化法
深層学習フレームワークにおけるIntel CPU/富岳向け最適化法深層学習フレームワークにおけるIntel CPU/富岳向け最適化法
深層学習フレームワークにおけるIntel CPU/富岳向け最適化法MITSUNARI Shigeo
 
How to make the Fastest C# Serializer, In the case of ZeroFormatter
How to make the Fastest C# Serializer, In the case of ZeroFormatterHow to make the Fastest C# Serializer, In the case of ZeroFormatter
How to make the Fastest C# Serializer, In the case of ZeroFormatterYoshifumi Kawai
 
機械学習によるデータ分析 実践編
機械学習によるデータ分析 実践編機械学習によるデータ分析 実践編
機械学習によるデータ分析 実践編Ryota Kamoshida
 
PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8Henry Schreiner
 
Accelerating microbiome research with OpenACC
Accelerating microbiome research with OpenACCAccelerating microbiome research with OpenACC
Accelerating microbiome research with OpenACCIgor Sfiligoi
 

Recommended

Low Overhead System Tracing with eBPF
Low Overhead System Tracing with eBPFLow Overhead System Tracing with eBPF
Low Overhead System Tracing with eBPFAkshay Kapoor
 
PaaS祭り - pagoda box
PaaS祭り - pagoda boxPaaS祭り - pagoda box
PaaS祭り - pagoda boxTsutomu Kawamura
 
异步io框架的实现
异步io框架的实现异步io框架的实现
异步io框架的实现rfyiamcool
 
深層学習フレームワークにおけるIntel CPU/富岳向け最適化法
深層学習フレームワークにおけるIntel CPU/富岳向け最適化法深層学習フレームワークにおけるIntel CPU/富岳向け最適化法
深層学習フレームワークにおけるIntel CPU/富岳向け最適化法MITSUNARI Shigeo
 
How to make the Fastest C# Serializer, In the case of ZeroFormatter
How to make the Fastest C# Serializer, In the case of ZeroFormatterHow to make the Fastest C# Serializer, In the case of ZeroFormatter
How to make the Fastest C# Serializer, In the case of ZeroFormatterYoshifumi Kawai
 
機械学習によるデータ分析 実践編
機械学習によるデータ分析 実践編機械学習によるデータ分析 実践編
機械学習によるデータ分析 実践編Ryota Kamoshida
 
PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8PyHEP 2019: Python 3.8
PyHEP 2019: Python 3.8Henry Schreiner
 
Accelerating microbiome research with OpenACC
Accelerating microbiome research with OpenACCAccelerating microbiome research with OpenACC
Accelerating microbiome research with OpenACCIgor Sfiligoi
 
Some Rough Fibrous Material
Some Rough Fibrous MaterialSome Rough Fibrous Material
Some Rough Fibrous MaterialMurray Steele
 
Ceph Day Shanghai - Ceph Performance Tools
Ceph Day Shanghai - Ceph Performance Tools Ceph Day Shanghai - Ceph Performance Tools
Ceph Day Shanghai - Ceph Performance Tools Ceph Community
 
Deploying Prometheus stacks with Juju
Deploying Prometheus stacks with JujuDeploying Prometheus stacks with Juju
Deploying Prometheus stacks with JujuJ.J. Ciarlante
 
Streams for the Web
Streams for the WebStreams for the Web
Streams for the WebDomenic Denicola
 
Практический опыт профайлинга и оптимизации производительности Ruby-приложений
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийПрактический опыт профайлинга и оптимизации производительности Ruby-приложений
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийOlga Lavrentieva
 
PenTest using Python By Purna Chander
PenTest using Python By Purna ChanderPenTest using Python By Purna Chander
PenTest using Python By Purna Chandernforceit
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for productionMark Rees
 
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.PVS-Studio
 
15 map reduce on azure
15 map reduce on azure15 map reduce on azure
15 map reduce on azurePatrick Bury
 
Scientific visualization with_gr
Scientific visualization with_grScientific visualization with_gr
Scientific visualization with_grJosef Heinen
 
讓 Python Script 擁有圖形化介面的簡單方法
讓 Python Script 擁有圖形化介面的簡單方法讓 Python Script 擁有圖形化介面的簡單方法
讓 Python Script 擁有圖形化介面的簡單方法pycontw
 
Templating with your {{mustache}}js
Templating with your {{mustache}}jsTemplating with your {{mustache}}js
Templating with your {{mustache}}jsStefano Ceschi Berrini
 
Building a DSL with GraalVM (VoxxedDays Luxembourg)
Building a DSL with GraalVM (VoxxedDays Luxembourg)Building a DSL with GraalVM (VoxxedDays Luxembourg)
Building a DSL with GraalVM (VoxxedDays Luxembourg)Maarten Mulders
 
WebAssembly向け多倍長演算の実装
WebAssembly向け多倍長演算の実装WebAssembly向け多倍長演算の実装
WebAssembly向け多倍長演算の実装MITSUNARI Shigeo
 
Pycon2017
Pycon2017Pycon2017
Pycon2017Yuta Kashino
 
Go on!
Go on!Go on!
Go on!Vadim Petrov
 
The Quantum Physics of Java
The Quantum Physics of JavaThe Quantum Physics of Java
The Quantum Physics of JavaMichael Heinrichs
 
Docker Online Meetup #27: Raspberry Pi DockerCon Challenge
Docker Online Meetup #27: Raspberry Pi DockerCon ChallengeDocker Online Meetup #27: Raspberry Pi DockerCon Challenge
Docker Online Meetup #27: Raspberry Pi DockerCon ChallengeDocker, Inc.
 
Device-specific Clang Tooling for Embedded Systems
Device-specific Clang Tooling for Embedded SystemsDevice-specific Clang Tooling for Embedded Systems
Device-specific Clang Tooling for Embedded SystemsemBO_Conference
 
淺入淺出 GDB
淺入淺出 GDB淺入淺出 GDB
淺入淺出 GDBJim Chang
 
Concurrency and Python - PyCon MY 2015
Concurrency and Python - PyCon MY 2015Concurrency and Python - PyCon MY 2015
Concurrency and Python - PyCon MY 2015Boey Pak Cheong
 
Skiron - Experiments in CPU Design in D
Skiron - Experiments in CPU Design in DSkiron - Experiments in CPU Design in D
Skiron - Experiments in CPU Design in DMithun Hunsur
 

More Related Content

What's hot

Some Rough Fibrous Material
Some Rough Fibrous MaterialSome Rough Fibrous Material
Some Rough Fibrous MaterialMurray Steele
 
Ceph Day Shanghai - Ceph Performance Tools
Ceph Day Shanghai - Ceph Performance Tools Ceph Day Shanghai - Ceph Performance Tools
Ceph Day Shanghai - Ceph Performance Tools Ceph Community
 
Deploying Prometheus stacks with Juju
Deploying Prometheus stacks with JujuDeploying Prometheus stacks with Juju
Deploying Prometheus stacks with JujuJ.J. Ciarlante
 
Практический опыт профайлинга и оптимизации производительности Ruby-приложений
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийПрактический опыт профайлинга и оптимизации производительности Ruby-приложений
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийOlga Lavrentieva
 
PenTest using Python By Purna Chander
PenTest using Python By Purna ChanderPenTest using Python By Purna Chander
PenTest using Python By Purna Chandernforceit
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for productionMark Rees
 
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.PVS-Studio
 
15 map reduce on azure
15 map reduce on azure15 map reduce on azure
15 map reduce on azurePatrick Bury
 
Scientific visualization with_gr
Scientific visualization with_grScientific visualization with_gr
Scientific visualization with_grJosef Heinen
 
讓 Python Script 擁有圖形化介面的簡單方法
讓 Python Script 擁有圖形化介面的簡單方法讓 Python Script 擁有圖形化介面的簡單方法
讓 Python Script 擁有圖形化介面的簡單方法pycontw
 
Building a DSL with GraalVM (VoxxedDays Luxembourg)
Building a DSL with GraalVM (VoxxedDays Luxembourg)Building a DSL with GraalVM (VoxxedDays Luxembourg)
Building a DSL with GraalVM (VoxxedDays Luxembourg)Maarten Mulders
 
WebAssembly向け多倍長演算の実装
WebAssembly向け多倍長演算の実装WebAssembly向け多倍長演算の実装
WebAssembly向け多倍長演算の実装MITSUNARI Shigeo
 
Docker Online Meetup #27: Raspberry Pi DockerCon Challenge
Docker Online Meetup #27: Raspberry Pi DockerCon ChallengeDocker Online Meetup #27: Raspberry Pi DockerCon Challenge
Docker Online Meetup #27: Raspberry Pi DockerCon ChallengeDocker, Inc.
 
Device-specific Clang Tooling for Embedded Systems
Device-specific Clang Tooling for Embedded SystemsDevice-specific Clang Tooling for Embedded Systems
Device-specific Clang Tooling for Embedded SystemsemBO_Conference
 
淺入淺出 GDB
淺入淺出 GDB淺入淺出 GDB
淺入淺出 GDBJim Chang
 

What's hot (20)

Some Rough Fibrous Material
Some Rough Fibrous MaterialSome Rough Fibrous Material
Some Rough Fibrous Material
 
Ceph Day Shanghai - Ceph Performance Tools
Ceph Day Shanghai - Ceph Performance Tools Ceph Day Shanghai - Ceph Performance Tools
Ceph Day Shanghai - Ceph Performance Tools
 
Deploying Prometheus stacks with Juju
Deploying Prometheus stacks with JujuDeploying Prometheus stacks with Juju
Deploying Prometheus stacks with Juju
 
Streams for the Web
Streams for the WebStreams for the Web
Streams for the Web
 
Практический опыт профайлинга и оптимизации производительности Ruby-приложений
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийПрактический опыт профайлинга и оптимизации производительности Ruby-приложений
Практический опыт профайлинга и оптимизации производительности Ruby-приложений
 
PenTest using Python By Purna Chander
PenTest using Python By Purna ChanderPenTest using Python By Purna Chander
PenTest using Python By Purna Chander
 
PyPy - is it ready for production
PyPy - is it ready for productionPyPy - is it ready for production
PyPy - is it ready for production
 
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
Celebrating 30-th anniversary of the first C++ compiler: let's find bugs in it.
 
15 map reduce on azure
15 map reduce on azure15 map reduce on azure
15 map reduce on azure
 
Scientific visualization with_gr
Scientific visualization with_grScientific visualization with_gr
Scientific visualization with_gr
 
讓 Python Script 擁有圖形化介面的簡單方法
讓 Python Script 擁有圖形化介面的簡單方法讓 Python Script 擁有圖形化介面的簡單方法
讓 Python Script 擁有圖形化介面的簡單方法
 
Templating with your {{mustache}}js
Templating with your {{mustache}}jsTemplating with your {{mustache}}js
Templating with your {{mustache}}js
 
Building a DSL with GraalVM (VoxxedDays Luxembourg)
Building a DSL with GraalVM (VoxxedDays Luxembourg)Building a DSL with GraalVM (VoxxedDays Luxembourg)
Building a DSL with GraalVM (VoxxedDays Luxembourg)
 
WebAssembly向け多倍長演算の実装
WebAssembly向け多倍長演算の実装WebAssembly向け多倍長演算の実装
WebAssembly向け多倍長演算の実装
 
Pycon2017
Pycon2017Pycon2017
Pycon2017
 
Go on!
Go on!Go on!
Go on!
 
The Quantum Physics of Java
The Quantum Physics of JavaThe Quantum Physics of Java
The Quantum Physics of Java
 
Docker Online Meetup #27: Raspberry Pi DockerCon Challenge
Docker Online Meetup #27: Raspberry Pi DockerCon ChallengeDocker Online Meetup #27: Raspberry Pi DockerCon Challenge
Docker Online Meetup #27: Raspberry Pi DockerCon Challenge
 
Device-specific Clang Tooling for Embedded Systems
Device-specific Clang Tooling for Embedded SystemsDevice-specific Clang Tooling for Embedded Systems
Device-specific Clang Tooling for Embedded Systems
 
淺入淺出 GDB
淺入淺出 GDB淺入淺出 GDB
淺入淺出 GDB
 

Similar to A TurtleBot Configurations Measurement Harness to Build a Sensitivity Model

Concurrency and Python - PyCon MY 2015
Concurrency and Python - PyCon MY 2015Concurrency and Python - PyCon MY 2015
Concurrency and Python - PyCon MY 2015Boey Pak Cheong
 
Skiron - Experiments in CPU Design in D
Skiron - Experiments in CPU Design in DSkiron - Experiments in CPU Design in D
Skiron - Experiments in CPU Design in DMithun Hunsur
 
maXbox Starter 45 Robotics
maXbox Starter 45 RoboticsmaXbox Starter 45 Robotics
maXbox Starter 45 RoboticsMax Kleiner
 
Machine Learning on Code - SF meetup
Machine Learning on Code - SF meetupMachine Learning on Code - SF meetup
Machine Learning on Code - SF meetupsource{d}
 
Streaming 101: Hello World
Streaming 101: Hello WorldStreaming 101: Hello World
Streaming 101: Hello WorldJosh Fischer
 
20141219 workshop methylation sequencing analysis
20141219 workshop methylation sequencing analysis20141219 workshop methylation sequencing analysis
20141219 workshop methylation sequencing analysisYi-Feng Chang
 
PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.
PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.
PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.Puppet
 
streamparse and pystorm: simple reliable parallel processing with storm
streamparse and pystorm: simple reliable parallel processing with stormstreamparse and pystorm: simple reliable parallel processing with storm
streamparse and pystorm: simple reliable parallel processing with stormDaniel Blanchard
 
[Kiwicon 2011] Post Memory Corruption Memory Analysis
[Kiwicon 2011] Post Memory Corruption Memory Analysis[Kiwicon 2011] Post Memory Corruption Memory Analysis
[Kiwicon 2011] Post Memory Corruption Memory AnalysisMoabi.com
 
[HITB Malaysia 2011] Exploit Automation
[HITB Malaysia 2011] Exploit Automation[HITB Malaysia 2011] Exploit Automation
[HITB Malaysia 2011] Exploit AutomationMoabi.com
 
[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory Analysis[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory AnalysisMoabi.com
 
WRENCH: Workflow Management System Simulation Workbench
WRENCH: Workflow Management System Simulation WorkbenchWRENCH: Workflow Management System Simulation Workbench
WRENCH: Workflow Management System Simulation WorkbenchRafael Ferreira da Silva
 
Distributed computing and hyper-parameter tuning with Ray
Distributed computing and hyper-parameter tuning with RayDistributed computing and hyper-parameter tuning with Ray
Distributed computing and hyper-parameter tuning with RayJan Margeta
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyAerospike
 
Parallelism in a NumPy-based program
Parallelism in a NumPy-based programParallelism in a NumPy-based program
Parallelism in a NumPy-based programRalf Gommers
 
Erlang/OTP
Erlang/OTPErlang/OTP
Erlang/OTPvoluntas
 
Euro python2011 High Performance Python
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance PythonIan Ozsvald
 
OpenCon2014 - Sumatra as an Open Science tool
OpenCon2014 - Sumatra as an Open Science toolOpenCon2014 - Sumatra as an Open Science tool
OpenCon2014 - Sumatra as an Open Science toolFelix Z. Hoffmann
 

Similar to A TurtleBot Configurations Measurement Harness to Build a Sensitivity Model (20)

Concurrency and Python - PyCon MY 2015
Concurrency and Python - PyCon MY 2015Concurrency and Python - PyCon MY 2015
Concurrency and Python - PyCon MY 2015
 
Skiron - Experiments in CPU Design in D
Skiron - Experiments in CPU Design in DSkiron - Experiments in CPU Design in D
Skiron - Experiments in CPU Design in D
 
Elasticwulf Pycon Talk
Elasticwulf Pycon TalkElasticwulf Pycon Talk
Elasticwulf Pycon Talk
 
maXbox Starter 45 Robotics
maXbox Starter 45 RoboticsmaXbox Starter 45 Robotics
maXbox Starter 45 Robotics
 
Machine Learning on Code - SF meetup
Machine Learning on Code - SF meetupMachine Learning on Code - SF meetup
Machine Learning on Code - SF meetup
 
Streaming 101: Hello World
Streaming 101: Hello WorldStreaming 101: Hello World
Streaming 101: Hello World
 
20141219 workshop methylation sequencing analysis
20141219 workshop methylation sequencing analysis20141219 workshop methylation sequencing analysis
20141219 workshop methylation sequencing analysis
 
Storm Anatomy
Storm AnatomyStorm Anatomy
Storm Anatomy
 
PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.
PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.
PuppetConf 2016: Multi-Tenant Puppet at Scale – John Jawed, eBay, Inc.
 
streamparse and pystorm: simple reliable parallel processing with storm
streamparse and pystorm: simple reliable parallel processing with stormstreamparse and pystorm: simple reliable parallel processing with storm
streamparse and pystorm: simple reliable parallel processing with storm
 
[Kiwicon 2011] Post Memory Corruption Memory Analysis
[Kiwicon 2011] Post Memory Corruption Memory Analysis[Kiwicon 2011] Post Memory Corruption Memory Analysis
[Kiwicon 2011] Post Memory Corruption Memory Analysis
 
[HITB Malaysia 2011] Exploit Automation
[HITB Malaysia 2011] Exploit Automation[HITB Malaysia 2011] Exploit Automation
[HITB Malaysia 2011] Exploit Automation
 
[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory Analysis[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory Analysis
 
WRENCH: Workflow Management System Simulation Workbench
WRENCH: Workflow Management System Simulation WorkbenchWRENCH: Workflow Management System Simulation Workbench
WRENCH: Workflow Management System Simulation Workbench
 
Distributed computing and hyper-parameter tuning with Ray
Distributed computing and hyper-parameter tuning with RayDistributed computing and hyper-parameter tuning with Ray
Distributed computing and hyper-parameter tuning with Ray
 
Golang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war storyGolang Performance : microbenchmarks, profilers, and a war story
Golang Performance : microbenchmarks, profilers, and a war story
 
Parallelism in a NumPy-based program
Parallelism in a NumPy-based programParallelism in a NumPy-based program
Parallelism in a NumPy-based program
 
Erlang/OTP
Erlang/OTPErlang/OTP
Erlang/OTP
 
Euro python2011 High Performance Python
Euro python2011 High Performance PythonEuro python2011 High Performance Python
Euro python2011 High Performance Python
 
OpenCon2014 - Sumatra as an Open Science tool
OpenCon2014 - Sumatra as an Open Science toolOpenCon2014 - Sumatra as an Open Science tool
OpenCon2014 - Sumatra as an Open Science tool
 

Recently uploaded

AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAlluxio, Inc.
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfOrtus Solutions, Corp
 
GlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobus
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptxGeorgi Kodinov
 
Breaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfBreaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfMeon Technology
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of ProgrammingMatt Welsh
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Krakówbim.edu.pl
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024Ortus Solutions, Corp
 
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...rajkumar669520
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesKrzysztofKkol1
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
 
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion Clinic
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandIES VE
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
 
GraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisGraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisNeo4j
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsGlobus
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILNatan Silnitsky
 
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Shahin Sheidaei
 

Recently uploaded (20)

AI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning FrameworkAI/ML Infra Meetup | Perspective on Deep Learning Framework
AI/ML Infra Meetup | Perspective on Deep Learning Framework
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
 
GlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote sessionGlobusWorld 2024 Opening Keynote session
GlobusWorld 2024 Opening Keynote session
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
Breaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdfBreaking the Code : A Guide to WhatsApp Business API.pdf
Breaking the Code : A Guide to WhatsApp Business API.pdf
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of Programming
 
Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...Developing Distributed High-performance Computing Capabilities of an Open Sci...
Developing Distributed High-performance Computing Capabilities of an Open Sci...
 
Agnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in KrakówAgnieszka Andrzejewska - BIM School Course in Kraków
Agnieszka Andrzejewska - BIM School Course in Kraków
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
 
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
Facemoji Keyboard released its 2023 State of Emoji report, outlining the most...
 
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERROR
 
Designing for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web ServicesDesigning for Privacy in Amazon Web Services
Designing for Privacy in Amazon Web Services
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
Abortion ^Clinic ^%[+971588192166''] Abortion Pill Al Ain (?@?) Abortion Pill...
 
Using IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New ZealandUsing IESVE for Room Loads Analysis - Australia & New Zealand
Using IESVE for Room Loads Analysis - Australia & New Zealand
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
GraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysisGraphAware - Transforming policing with graph-based intelligence analysis
GraphAware - Transforming policing with graph-based intelligence analysis
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
Gamify Your Mind; The Secret Sauce to Delivering Success, Continuously Improv...
 

A TurtleBot Configurations Measurement Harness to Build a Sensitivity Model

  • 1. A TurtleBot Configurations Measurement Harness to Build a Sensitivity Model Miguel Velez Software Engineering Ph.D. student 1
  • 2. Personal Background Quito, Ecuador University of St. Thomas Computer Science and Physics Sportradar US CMU advisor: Christian Kästner 2
  • 6. Variables Independent AMCL Environment ~25 numeric configurations (i.e. min_particles, max_particles, update_min_a) Kinect noise Kinect miscalibration Odometry noise Odometry miscalibration Dependent Time CPU Utilization Localization Error 6
  • 9. Sensitivity Model Difficult! Exponential configuration space 25 configurations with 10 options each 1025 total points * 2 min = ~109 * universe age 9
  • 10. Dealing with Large Space 10
  • 11. Dealing with Large Space 11
  • 12. Dealing with Large Space 12 Default
  • 13. Dealing with Large Space 13 Default
  • 14. Dealing with Large Space 14 Default
  • 15. Dealing with Large Space 15 No difference Difference
  • 17. Simulation TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc LocalizationTime 17 CPU Simulator Process
  • 20. Data Synchronization Problem: Different topic publish rate 20 Solution: Get data every second
  • 21. Simulator uses most CPU CPU Utilization 21 Run on separate machines CPU Simulator TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp LocalizationTime CPU ( ( X Process Computer Simulator ( ( TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc CPU
  • 22. Infrastructure feature4.andrew.cmu.edu TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc feature3.andrew.cmu.edu ROS_MASTER_URI db job data job data job data job data job data Master run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh feature.isri.cmu.edu Fetch Save Execute 22 Process Computer Simulator ROS_MASTER_URI
  • 23. Infrastructure featureY.andrew.cmu.edu TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc featureX.andrew.cmu.edu db job data job data job data job data job data Master run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh run.sh feature.isri.cmu.edu 23 Process Computer Simulator featureW.andrew.cmu.edu TurtleBot foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc foo bar moo cpp py pyc featureZ.andrew.cmu.edu Simulator
  • 24. Infrastructure 24 ~2 min/job 1 server 360 jobs in 12h 36 individual options 144 options with 4 servers ~14 configurations with 10 options each Can measure other metrics
  • 28. Lessons Learned Few configurations affect CPU and localization Best configurations around default values AMCL highly adaptive and robust Some options can balance CPU and localization 28
  • 29. Lessons Learned Some options can balance CPU and localization 29
  • 30. Future Work Explore environmental configurations Kinect and Odometry Remap? White-box analysis of AMCL Combine configurations Build sensitivity model Measure other metrics? 30
  • 31. Summary • Sensitivity model needed in self-adaptive systems • Infrastructure for automatic measurement • Suggestions for problems and ideas for infrastructure 31