Deploying OSGi on an Army of CubieTrucksSendrato powerpoint

Frank Lyaruu
Frank LyaruuSenior Technology Architect at Sendrato
Wearables at Scale 
Deploying OSGi on an army of Cubie Trucks 
Frank Lyaruu
About me 
Frank Lyaruu 
CTO at 
Senior Technology Architect at 
@lyaruu 
11/17/14 2
About Dexels 
• Est. 2001 based in Amsterdam 
• Enterprise software for large 
companies 
• Heavily invested in OSGi 
• Focused on sports and entertainment 
11/17/14 
3
About Dexels 
• Est. 2013 
• Joint venture with Dexels 
• Custom hardware for large events 
11/17/14 
4
• Case study 
• Hardware stack 
• Software stack 
• Development chain 
• DEMO! 
5
Tomorrowland
Tomorrowland
Tomorrowland 2014 
Electronic music festival near Antwerp, 
Belgium 
• 2 weekends 
• 8 stages 
• ~100 hectares 
• ~100k visitors a weekend 
• ~ € 250 per ticket 
11/17/14 
8
Requirements
Requirements 
Design a wristband to do: 
• Access control 
• Location 
• Profiling / feedback 
• Crowd monitoring 
• Integrate with 3rd party applications 
• Something that adds to the UX 
11/17/14 
10
Access control 
• Should always work 
• Should be secure 
…simplicity is essential: Passive RFID 
11/17/14 
11
Location 
• Ability to locate individuals 
• Ability to locate security personnel 
11/17/14 
12
Profiling / feedback 
• How do the visitors behave? 
• What acts are popular? 
11/17/14 
13
Crowd monitoring 
• Find hotspots in the flow of people 
• Dashboard for security personnel 
11/17/14 
14
UX 
• UX is key 
• Comfortable 
• Match the festival 
• Suitable for non-nerds 
11/17/14 
15
The plan
The setup 
• Set up base stations around the 
terrain 
• Wristbands will report status 
• The base stations triangulate and 
consolidate the data 
• Visualize and push the data to other 
applications 
• Use regular passive RFID for access 
11/17c/14ontrol 
17
Something cool 
• LED lights on the wristbands, 
integrated with the light show 
• A button to send Facebook friend 
requests 
11/17/14 
18
The protocol 
• Time slotted protocol 
• All base stations synchronise with 
each other 
• Visitor bracelets send every few 
minutes 
• Staff bracelets send every few 
seconds 
11/17/14 
19
Hardware
Hardware 
• One 3V battery for 3 days 
• Leather 
• Branded for the festival 
• < $10 per bracelet 
11/17/14 
21
Base station hardware 
22
Base station 
23
§ 
24
Hardware challenges
11/17/14 
26
Base stations 
• Heat 
• Rain 
• Unreliable power 
• Unreliable network 
• Unstable mount points 
• Electromagnetic interference 
11/17/14 
27
Base station deployment 
28
Base stations 
11/17/14 
29
Sendrato Infrastructure 
LLooccaal l“ “ddaattaacceennttrree”” 
HHaazzeelclcaasstt c clulusstteerr 
11/17/14 
33rrdd p paarrttyy 
30 
CCuubbieieTTrruucckk CCuubbieieTTrruucckk CCuubbieieTTrruucckk 
CClolouudd 
SSaaltlt M Maasstteerr 
LLoogg s seerrvveerr 
MMoonnggooDDbb MMoonnggooDDbb 
SSuuppeerr c coooorrddininaattoorr 
MMaannaaggeemmeenntt U UII
Base station software stack 
11/17/14 
31 
AAppaacchhee F Feelilxix 
JJaavvaa 
FFeeddoorraa 2 200 
SSaalltt aaggeenntt 
SSeennddrraattoo A Apppplilcicaattioionn
Basestation software 
BBaassee s sttaattioionn 
FFilitlteerr e evveenntt PPuubblilsishh i nin c clulusstteerr 
32 
WWrrisisttbbaannddss 
o 
Radi 
RRaaddioio D Drriviveerr OOSSGGi ie evveenntt 
LLoogg l oloccaalllyly?? 
Radi 
o
Logging 
• Custom Logback appender to Redis 
queue 
• ElasticSearch analytics 
• Kibana web interface 
33
Salt Stack 
• Configuration manager 
• Remote execution 
• Agent based 
• Versatile & Fast 
• Master running in the cloud 
34
“Testing in production”
Development cycle 
36 
• Pretty crude but effective: 
EEcclilpipssee MMaavveenn b buuilidld GGitit SSaaltlt FFilieleInInssttaallll
Future work 
• Distributed OSGi 
• More mature deployment model 
• Apache Karaf + Cellar 
• Pluggable network / clustering 
• Proper 3rd party API 
37
Demo
• http://youtu.be/eDZaea7OCd0 39
Thank you! 
@Sendrato 
sendrato.com 
info@sendrato.com 
40
1 of 40

Recommended

المناطق الحارة by
المناطق الحارةالمناطق الحارة
المناطق الحارةCharles El Khoury
5.3K views34 slides
Cataract and refractive surgery by
Cataract and refractive surgeryCataract and refractive surgery
Cataract and refractive surgeryOphthalmicDocs Chiong
2.9K views28 slides
Scripting Languages in OSGi by
Scripting Languages in OSGiScripting Languages in OSGi
Scripting Languages in OSGiFrank Lyaruu
863 views35 slides
Service Discovery in OSGi: Beyond the JVM using Docker and Consul by
Service Discovery in OSGi: Beyond the JVM using Docker and ConsulService Discovery in OSGi: Beyond the JVM using Docker and Consul
Service Discovery in OSGi: Beyond the JVM using Docker and ConsulFrank Lyaruu
1.8K views30 slides
It's beautiful enRoute - Paul Fraser by
It's beautiful enRoute - Paul FraserIt's beautiful enRoute - Paul Fraser
It's beautiful enRoute - Paul Frasermfrancis
1K views34 slides
Eclipse + Maven + OSGi has never been so easy - Atllia Kiss by
Eclipse + Maven + OSGi has never been so easy - Atllia KissEclipse + Maven + OSGi has never been so easy - Atllia Kiss
Eclipse + Maven + OSGi has never been so easy - Atllia Kissmfrancis
1.1K views22 slides

More Related Content

Similar to Deploying OSGi on an Army of CubieTrucksSendrato powerpoint

After the hackday by
After the hackdayAfter the hackday
After the hackdaySomeRandomBloke
400 views26 slides
Silent sound technology by
Silent sound technologySilent sound technology
Silent sound technologyJeet Das
19.2K views19 slides
OSGi in the consumer entertainment environment - P Bristow & L Rozendaal by
OSGi in the consumer entertainment environment  - P Bristow & L RozendaalOSGi in the consumer entertainment environment  - P Bristow & L Rozendaal
OSGi in the consumer entertainment environment - P Bristow & L Rozendaalmfrancis
412 views29 slides
BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications" by
BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"
BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"Daniel Bryant
600 views42 slides
Sensor Based Blind Stick by
Sensor Based Blind StickSensor Based Blind Stick
Sensor Based Blind StickGagandeep Singh
48K views24 slides
computer_IntroductionComputer introduction by
computer_IntroductionComputer introductioncomputer_IntroductionComputer introduction
computer_IntroductionComputer introductionEhsan Ullah Kakar
444 views84 slides

Similar to Deploying OSGi on an Army of CubieTrucksSendrato powerpoint(20)

Silent sound technology by Jeet Das
Silent sound technologySilent sound technology
Silent sound technology
Jeet Das19.2K views
OSGi in the consumer entertainment environment - P Bristow & L Rozendaal by mfrancis
OSGi in the consumer entertainment environment  - P Bristow & L RozendaalOSGi in the consumer entertainment environment  - P Bristow & L Rozendaal
OSGi in the consumer entertainment environment - P Bristow & L Rozendaal
mfrancis412 views
BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications" by Daniel Bryant
BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"
BETA: "Cloud Developers DHARMA: Redefining 'done' for Cloud applications"
Daniel Bryant600 views
computer_IntroductionComputer introduction by Ehsan Ullah Kakar
computer_IntroductionComputer introductioncomputer_IntroductionComputer introduction
computer_IntroductionComputer introduction
Ehsan Ullah Kakar444 views
Ostinato - Craft Packets, Generate Traffic [SharkFest '20] by pstavirs
Ostinato - Craft Packets, Generate Traffic [SharkFest '20]Ostinato - Craft Packets, Generate Traffic [SharkFest '20]
Ostinato - Craft Packets, Generate Traffic [SharkFest '20]
pstavirs1.8K views
Fukuoka University Public NTP Service Deployment Use case by APNIC
Fukuoka University Public NTP Service Deployment Use caseFukuoka University Public NTP Service Deployment Use case
Fukuoka University Public NTP Service Deployment Use case
APNIC5K views
silentsoundtechnology-131124090312-phpapp01.pptx by 20269vinay
silentsoundtechnology-131124090312-phpapp01.pptxsilentsoundtechnology-131124090312-phpapp01.pptx
silentsoundtechnology-131124090312-phpapp01.pptx
20269vinay3 views
Mobile Phone Instruments, the Possibilities of Networks, and OSC by NathanBowen8
Mobile Phone Instruments, the Possibilities of Networks, and OSCMobile Phone Instruments, the Possibilities of Networks, and OSC
Mobile Phone Instruments, the Possibilities of Networks, and OSC
NathanBowen8306 views
FFMEET: running a non-profit conference system by Annika Wickert
FFMEET: running a non-profit conference systemFFMEET: running a non-profit conference system
FFMEET: running a non-profit conference system
Annika Wickert2K views
DC4420 2014 - NFC - The Non-Radio Bits by Tom Keetch
DC4420 2014 - NFC - The Non-Radio BitsDC4420 2014 - NFC - The Non-Radio Bits
DC4420 2014 - NFC - The Non-Radio Bits
Tom Keetch2.8K views
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B... by Larry Smarr
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Larry Smarr7 views
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B... by Larry Smarr
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Larry Smarr193 views
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B... by Larry Smarr
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Panel: Open Infrastructure for an Open Society: OSG, Commercial Clouds, and B...
Larry Smarr6 views
How i built my own irrigation controller by Shree Kumar
How i built my own irrigation controllerHow i built my own irrigation controller
How i built my own irrigation controller
Shree Kumar2.5K views
Turku Random Tour presentation by Juho Loukonen
Turku Random Tour presentationTurku Random Tour presentation
Turku Random Tour presentation
Juho Loukonen265 views

More from Frank Lyaruu

Too young to quit, too old to change by
Too young to quit, too old to changeToo young to quit, too old to change
Too young to quit, too old to changeFrank Lyaruu
92 views33 slides
Embracing Database Diversity with Kafka and Debezium by
Embracing Database Diversity with Kafka and DebeziumEmbracing Database Diversity with Kafka and Debezium
Embracing Database Diversity with Kafka and DebeziumFrank Lyaruu
1.1K views58 slides
Non Blocking I/O for Everyone with RxJava by
Non Blocking I/O for Everyone with RxJavaNon Blocking I/O for Everyone with RxJava
Non Blocking I/O for Everyone with RxJavaFrank Lyaruu
2K views71 slides
The Road To Reactive with RxJava JEEConf 2016 by
The Road To Reactive with RxJava JEEConf 2016The Road To Reactive with RxJava JEEConf 2016
The Road To Reactive with RxJava JEEConf 2016Frank Lyaruu
749 views80 slides
ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co... by
ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co...ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co...
ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co...Frank Lyaruu
724 views44 slides
Developing Like There's No Tomorrow by
Developing Like There's No TomorrowDeveloping Like There's No Tomorrow
Developing Like There's No TomorrowFrank Lyaruu
782 views33 slides

More from Frank Lyaruu(6)

Too young to quit, too old to change by Frank Lyaruu
Too young to quit, too old to changeToo young to quit, too old to change
Too young to quit, too old to change
Frank Lyaruu92 views
Embracing Database Diversity with Kafka and Debezium by Frank Lyaruu
Embracing Database Diversity with Kafka and DebeziumEmbracing Database Diversity with Kafka and Debezium
Embracing Database Diversity with Kafka and Debezium
Frank Lyaruu1.1K views
Non Blocking I/O for Everyone with RxJava by Frank Lyaruu
Non Blocking I/O for Everyone with RxJavaNon Blocking I/O for Everyone with RxJava
Non Blocking I/O for Everyone with RxJava
Frank Lyaruu2K views
The Road To Reactive with RxJava JEEConf 2016 by Frank Lyaruu
The Road To Reactive with RxJava JEEConf 2016The Road To Reactive with RxJava JEEConf 2016
The Road To Reactive with RxJava JEEConf 2016
Frank Lyaruu749 views
ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co... by Frank Lyaruu
ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co...ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co...
ApacheCon Core: Service Discovery in OSGi: Beyond the JVM using Docker and Co...
Frank Lyaruu724 views
Developing Like There's No Tomorrow by Frank Lyaruu
Developing Like There's No TomorrowDeveloping Like There's No Tomorrow
Developing Like There's No Tomorrow
Frank Lyaruu782 views

Recently uploaded

Case Study Copenhagen Energy and Business Central.pdf by
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdfAitana
12 views3 slides
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N... by
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...James Anderson
33 views32 slides
Lilypad @ Labweek, Istanbul, 2023.pdf by
Lilypad @ Labweek, Istanbul, 2023.pdfLilypad @ Labweek, Istanbul, 2023.pdf
Lilypad @ Labweek, Istanbul, 2023.pdfAlly339821
9 views45 slides
virtual reality.pptx by
virtual reality.pptxvirtual reality.pptx
virtual reality.pptxG036GaikwadSnehal
11 views15 slides
Uni Systems for Power Platform.pptx by
Uni Systems for Power Platform.pptxUni Systems for Power Platform.pptx
Uni Systems for Power Platform.pptxUni Systems S.M.S.A.
50 views21 slides
AMAZON PRODUCT RESEARCH.pdf by
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdfJerikkLaureta
15 views13 slides

Recently uploaded(20)

Case Study Copenhagen Energy and Business Central.pdf by Aitana
Case Study Copenhagen Energy and Business Central.pdfCase Study Copenhagen Energy and Business Central.pdf
Case Study Copenhagen Energy and Business Central.pdf
Aitana12 views
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N... by James Anderson
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
GDG Cloud Southlake 28 Brad Taylor and Shawn Augenstein Old Problems in the N...
James Anderson33 views
Lilypad @ Labweek, Istanbul, 2023.pdf by Ally339821
Lilypad @ Labweek, Istanbul, 2023.pdfLilypad @ Labweek, Istanbul, 2023.pdf
Lilypad @ Labweek, Istanbul, 2023.pdf
Ally3398219 views
AMAZON PRODUCT RESEARCH.pdf by JerikkLaureta
AMAZON PRODUCT RESEARCH.pdfAMAZON PRODUCT RESEARCH.pdf
AMAZON PRODUCT RESEARCH.pdf
JerikkLaureta15 views
6g - REPORT.pdf by Liveplex
6g - REPORT.pdf6g - REPORT.pdf
6g - REPORT.pdf
Liveplex9 views
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas... by Bernd Ruecker
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
iSAQB Software Architecture Gathering 2023: How Process Orchestration Increas...
Bernd Ruecker26 views
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院 by IttrainingIttraining
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
【USB韌體設計課程】精選講義節錄-USB的列舉過程_艾鍗學院
From chaos to control: Managing migrations and Microsoft 365 with ShareGate! by sammart93
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
From chaos to control: Managing migrations and Microsoft 365 with ShareGate!
sammart939 views
DALI Basics Course 2023 by Ivory Egg
DALI Basics Course  2023DALI Basics Course  2023
DALI Basics Course 2023
Ivory Egg14 views
Unit 1_Lecture 2_Physical Design of IoT.pdf by StephenTec
Unit 1_Lecture 2_Physical Design of IoT.pdfUnit 1_Lecture 2_Physical Design of IoT.pdf
Unit 1_Lecture 2_Physical Design of IoT.pdf
StephenTec11 views
HTTP headers that make your website go faster - devs.gent November 2023 by Thijs Feryn
HTTP headers that make your website go faster - devs.gent November 2023HTTP headers that make your website go faster - devs.gent November 2023
HTTP headers that make your website go faster - devs.gent November 2023
Thijs Feryn19 views
handbook for web 3 adoption.pdf by Liveplex
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdf
Liveplex19 views
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2216 views

Deploying OSGi on an Army of CubieTrucksSendrato powerpoint

  • 1. Wearables at Scale Deploying OSGi on an army of Cubie Trucks Frank Lyaruu
  • 2. About me Frank Lyaruu CTO at Senior Technology Architect at @lyaruu 11/17/14 2
  • 3. About Dexels • Est. 2001 based in Amsterdam • Enterprise software for large companies • Heavily invested in OSGi • Focused on sports and entertainment 11/17/14 3
  • 4. About Dexels • Est. 2013 • Joint venture with Dexels • Custom hardware for large events 11/17/14 4
  • 5. • Case study • Hardware stack • Software stack • Development chain • DEMO! 5
  • 8. Tomorrowland 2014 Electronic music festival near Antwerp, Belgium • 2 weekends • 8 stages • ~100 hectares • ~100k visitors a weekend • ~ € 250 per ticket 11/17/14 8
  • 10. Requirements Design a wristband to do: • Access control • Location • Profiling / feedback • Crowd monitoring • Integrate with 3rd party applications • Something that adds to the UX 11/17/14 10
  • 11. Access control • Should always work • Should be secure …simplicity is essential: Passive RFID 11/17/14 11
  • 12. Location • Ability to locate individuals • Ability to locate security personnel 11/17/14 12
  • 13. Profiling / feedback • How do the visitors behave? • What acts are popular? 11/17/14 13
  • 14. Crowd monitoring • Find hotspots in the flow of people • Dashboard for security personnel 11/17/14 14
  • 15. UX • UX is key • Comfortable • Match the festival • Suitable for non-nerds 11/17/14 15
  • 17. The setup • Set up base stations around the terrain • Wristbands will report status • The base stations triangulate and consolidate the data • Visualize and push the data to other applications • Use regular passive RFID for access 11/17c/14ontrol 17
  • 18. Something cool • LED lights on the wristbands, integrated with the light show • A button to send Facebook friend requests 11/17/14 18
  • 19. The protocol • Time slotted protocol • All base stations synchronise with each other • Visitor bracelets send every few minutes • Staff bracelets send every few seconds 11/17/14 19
  • 21. Hardware • One 3V battery for 3 days • Leather • Branded for the festival • < $10 per bracelet 11/17/14 21
  • 24. § 24
  • 27. Base stations • Heat • Rain • Unreliable power • Unreliable network • Unstable mount points • Electromagnetic interference 11/17/14 27
  • 30. Sendrato Infrastructure LLooccaal l“ “ddaattaacceennttrree”” HHaazzeelclcaasstt c clulusstteerr 11/17/14 33rrdd p paarrttyy 30 CCuubbieieTTrruucckk CCuubbieieTTrruucckk CCuubbieieTTrruucckk CClolouudd SSaaltlt M Maasstteerr LLoogg s seerrvveerr MMoonnggooDDbb MMoonnggooDDbb SSuuppeerr c coooorrddininaattoorr MMaannaaggeemmeenntt U UII
  • 31. Base station software stack 11/17/14 31 AAppaacchhee F Feelilxix JJaavvaa FFeeddoorraa 2 200 SSaalltt aaggeenntt SSeennddrraattoo A Apppplilcicaattioionn
  • 32. Basestation software BBaassee s sttaattioionn FFilitlteerr e evveenntt PPuubblilsishh i nin c clulusstteerr 32 WWrrisisttbbaannddss o Radi RRaaddioio D Drriviveerr OOSSGGi ie evveenntt LLoogg l oloccaalllyly?? Radi o
  • 33. Logging • Custom Logback appender to Redis queue • ElasticSearch analytics • Kibana web interface 33
  • 34. Salt Stack • Configuration manager • Remote execution • Agent based • Versatile & Fast • Master running in the cloud 34
  • 36. Development cycle 36 • Pretty crude but effective: EEcclilpipssee MMaavveenn b buuilidld GGitit SSaaltlt FFilieleInInssttaallll
  • 37. Future work • Distributed OSGi • More mature deployment model • Apache Karaf + Cellar • Pluggable network / clustering • Proper 3rd party API 37
  • 38. Demo
  • 40. Thank you! @Sendrato sendrato.com info@sendrato.com 40

Editor's Notes

  1. Last name for twitter handle! 1m
  2. Means ‘wireless’ in Esperanto. (We think) 3m
  3. Ask questions when you have them
  4. 5m
  5. Anyone heard of Tomorrowland? Big deal
  6. … pretty scary launch customer 6m
  7. Not the typical enterprise requirements
  8. People can get hurt when access control fails 9m
  9. … typical big data questions
  10. 13m
  11. Cubie = power pi, with onboard storage ARM architecture
  12. 17m
  13. +5
  14. +8 Also dmx 25 min
  15. 27m
  16. 30m
  17. Server in the cloud because of possible network config problems
  18. 34m master in the cloud because of possible routing problems agent based faster and more resilient to network probe (Ansible)
  19. We knew we’d be hacking all weekend
  20. +3 37 min 41 min
  21. 42 min.