SlideShare a Scribd company logo
1 of 54
Download to read offline
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Abstract 
● An entertainment system at 10,000 metres 
– Embedded computer in each plane 
– Synchronized after each flight 
– And a Platform with lots of integrations 
● This is how we do it 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Hi! 
● I'm David Arcos 
● Python/Django developer since 2008 
● Interested in distributed systems, 
databases, scalability, security 
● Backend engineer at Immfly 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Immfly: In-Flight Entertainment 
Immfly is a new Entertainment, Retail and 
Communication platform for the in-flight 
experience. 
Focused on the European domestic flights 
market, Immfly offers wireless content to 
passengers via their Personal Electronic 
Devices. 
Private & Confidential 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
The three challenges 
1) On board the aircraft: 
● Provide a Backend with lots of features 
● Will work off-line 
2) Synchronize the aircrafts 
● Keep the replicas consistent 
● Update the contents: I don't want yesterday's newspaper! 
3) Integrate with 3rd parties 
● Content providers, payments, mailing, weather, flight info... 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
1) On board the aircraft 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Some requirements 
● Users have different needs 
● Common authentication 
● It's offline 
● Show some flight info 
● Deliver static content 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Users have different needs 
● Apps: web, android, iOS 
● Devices: laptop, smartphone, tablet 
● Return different results based on language, 
destination, airline, schedule... 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Users have different needs 
● Apps: web, android, iOS 
– Same REST API for every frontend 
● Devices: laptop, smartphone, tablet 
– Generate thumbnails with different sizes to allow 
a responsive design 
● Return different results based on language, 
destination, airline, schedule... 
– API calls allow filtering by many parameters 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Common authentication 
● Same user, different apps 
– Example: web app opens mobile app 
– Example: mobile app embeds a webview 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Common authentication 
● Same user, different apps 
– Example: web app opens mobile app 
– Example: mobile app embeds a webview 
● Django Rest Framework has TokenAuthentication 
– Just a HTTP header, example: 
Authorization: Token 9944b09199c62bcf9418ad846dd0e4bbdfc6ee4b 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
It's off-line! 
● Can't use any external service: 
– No analytics, online error logging, google maps 
– No fancy SaaS integrations 
– Can't just use CDN, email or DNS servers 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
It's off-line! 
● Can't use any external service: 
– No analytics, online error logging, google maps 
– No fancy SaaS integrations 
– Can't just use CDN, email or DNS servers 
● We deployed our own solutions 
– Very time-consuming 
– Can't hotfix bugs. Need to do lots of Q&A. 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Show some flight info 
● Estimated Time of Arrival? 
● Weather at destination 
● Show a nice map 
● Where is the plane? 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Show THIS flight info 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Lots of flight info 
● Estimated Time of Arrival? 
– Pre-load Flightstats API 
● Weather at destination 
– Pre-load OpenWeatherMap (pyowm) 
● Show a nice map 
– OpenStreetMap 
● Where is the plane? 
– The plane tells us :) 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Avionics data bus 
We get data in real-time: 
● altitude 
● flight_id 
● ground_speed 
● heading 
● latitude 
● longitude 
● mach_speed 
● outside_temperature 
● pitch 
● roll 
● wind_speed 
● yaw 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Discrete-time signals 
● DCFAILSIG 
● ACFAILSIG 
● OVERTEMPSIG 
● GSM_POWER_STATUS 
● ENB2SIG 
● ENB1SIG 
● ENB0SIG 
● GSMSIG_STATUS 
● CPLD_REV0 
● CPLD_REV1 
● SYSENSIG 
● ENB3SIG 
● ENB4SIG 
● ENB5SIG 
● ALERT 
● CONFIGSIG0 
● CONFIGSIG1 
● CONFIGSIG2 
● INTTEST_OUT 
● INTTEST_IN 
● ISO_OUT0 
● ISO_OUT1 
● ISO_OUT2 
● ISO_OUT3 
● GPIO_DCFAILSIG 
● GPIO_ACFAILSIG 
● GPIO_OVERTEMPSIG 
● GPIO_SYSENSIG 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Deliver static content 
● Provide the user with different contents: 
– TV Shows, Videos (format? size?) 
– Newspapers, Magazines 
– Images 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Deliver static content 
● Provide the user with different contents: 
– TV Shows, Videos 
– Newspapers, Magazines 
– Images 
● Just use nginx, but: 
– Transcode video to HLS (bitrate, chunks) 
– Pre-process PDFs 
– Load test to verify assumptions 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Other components 
Our django apps: 
– common 
– content 
– flightinfo 
– images 
– stats 
– surveys 
– users 
Some external libraries: 
– Django 
– Django Rest Framework 
– django-uuidfield 
– django-redis 
– django-celery 
– django-extensions 
– django-imagekit 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Aircraft Infrastructure 
HTTP server: 
+ 
+ 
Databases: 
& 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Embedded computer (lab) 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Access point + dashboard (lab) 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Hardware schema 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
The three challenges 
2) Synchronize the aircrafts 
● Keep the replicas consistent 
● Update the contents: I don't want yesterday's newspaper! 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
2) Synchronize the aircrafts 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Context 
● An Aircraft needs to update contents and 
usage 
● During the flight, the Aircraft is off-line 
● Once landed, it gets connectivity 
● Then it tries to synchronize 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Connectivity 
● Internal 3G data card 
– Enabled when grounded 
– Disabled when about to flight 
● VPN (“control tower”) 
– SSH to the Hangar 
– Hangar can SSH back 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
The Hangar 
● Central point of the platform 
● Special instance, in the cloud 
– Same software (db schema) 
– Different apps and settings 
● Used as a CMS, to manage all resources 
– django.contrib.admin 
● All other instances just replicate some data 
from the Hangar 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
The Heartbeat 
● An Aircraft sends Heartbeats when it's online 
– It's just a “ping” to the Hangar 
– A simple POST, with some details 
● The Hangar stores the heartbeats 
– Knows what planes are online, grounded, at a 
given moment 
– Will send commands to those planes 
● In example: synchronize 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Aircraft synchronization 
● An Aircraft: 
– Sends usage data 
● New/updated users, surveys, payments, stats, etc 
– Gets usage data 
● New/updated users 
– Gets updated contents 
● New/updated resources, contents, offers, destinations, 
misc data... 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Trouble? 
● For some models (like Users), the Primary Keys 
will collide 
● Can get conflicts 
● May need to merge resources, edited in 
several Aircrafts at the same time 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Prevent the trouble 
● For some models (like Users), the Primary Keys 
will collide 
– Those PKs must be UUIDs 
● Can get conflicts 
– Each model should be synchronized only in one 
direction: A->H or H->A 
● May need to merge resources, edited in 
several Aircrafts at the same time 
– Make sure our logic doesn't allow that 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
fabric 
Python library and command-line tool for 
streamlining the use of SSH for application 
deployment or systems administration tasks 
http://fabfile.org 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
fabtools.require 
fabtools includes useful functions to help you write 
your Fabric files. 
Using fabtools.require allows you to use a more 
declarative style, similar to Chef or Puppet. 
http://fabtools.readthedocs.org 
Example: 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Other tools 
● Python 
– boto: upload/download from S3 
– requests: API calls 
– django.db.migrations 
● Unix 
– openssh 
– rsync 
– bash script 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
The three challenges 
3) Integrate with 3rd parties 
● Providers, payments, mailing, weather, flight info... 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
3) Integrate with 3rd parties 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Highlights 
● Resource Ingest 
● API diversity 
● AWS 
● Celery 
● Other APIs 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Resource ingest 
● Entertainment 
– Newspapers 
– Magazines 
– Videos 
– TV Shows 
● Deals 
– Airport 
– Transport 
– Destination 
● Lots of resources with lots of providers 
– Per language, country, etc 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
API diversity... 
● Best case 
– Python API :) 
● Good enough 
– REST API, lots of metadata, well documented 
● Acceptable 
– S3 bucket or FTP, import all, but no metadata 
● Worst case: 
– ? 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Beware of the Intern-API! 
https://www.flickr.com/photos/reidrac/2387432357/ 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Is this even a API? 
● Word document 
– (why?) 
● Excel document 
– (no format, of course) 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
AWS, using boto 
● ElasticTranscoder 
– Media transcoding in the cloud 
● Simple Notification Service (SNS) 
– A fast, flexible, fully managed push messaging 
service 
● Simple Storage Service (S3) 
– Secure, durable, highly-scalable object storage. 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Celery task queue 
● Hangar tasks use Celery: 
– Celery is an asynchronous task queue/job 
queue based on distributed message passing 
– Async 
– I/O intensive tasks: crawling APIs 
– CPU intensive tasks: thumbnails, billing pdfs 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Flower: Celery monitoring tool 
● Flower is a web based tool for monitoring 
and administrating Celery clusters 
– http://flower.readthedocs.org 
● Real time 
● Tip: a queue per task type 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Other APIs 
● Bookings 
● Payments 
● Billing 
● Emails 
● Weather 
● Flight info 
● Statistics 
● Slack 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Other libraries 
● Images: 
– Pillow 
– django-imagekit 
– pilkit 
– PyPDF2 
– Wand 
● PDFs: 
– django-easy-pdf 
– xhtml2pdf 
● Other: 
– django-yubin 
– libsaas 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Recap 
● Solved the three challenges! 
– In-flight API with tons of features 
– When grounded, they get synchronized 
– The Hangar manages the full platform 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Conclusions 
● Python made it possible! 
● Very versatile, covers all our use cases 
● “We stand on the shoulders of giants” 
● Developed in a short time 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Thanks for attending! 
Get the slides at http://slideshare.net/DZPM 
Any questions? 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Questions? 
Python on a Plane – PyConES 2014 David Arcos - @DZPM
Thanks for attending! 
Get the slides at http://slideshare.net/DZPM 
Enjoy your flight! http://immfly.com 
Python on a Plane – PyConES 2014 David Arcos - @DZPM

More Related Content

Viewers also liked

Authentication & Authorization in ASPdotNet MVC
Authentication & Authorization in ASPdotNet MVCAuthentication & Authorization in ASPdotNet MVC
Authentication & Authorization in ASPdotNet MVCMindfire Solutions
 
The Django Book, Chapter 16: django.contrib
The Django Book, Chapter 16: django.contribThe Django Book, Chapter 16: django.contrib
The Django Book, Chapter 16: django.contribTzu-ping Chung
 
Overview of Testing Talks at Pycon
Overview of Testing Talks at PyconOverview of Testing Talks at Pycon
Overview of Testing Talks at PyconJacqueline Kazil
 
Django - The Web framework for perfectionists with deadlines
Django - The Web framework for perfectionists with deadlinesDjango - The Web framework for perfectionists with deadlines
Django - The Web framework for perfectionists with deadlinesMarkus Zapke-Gründemann
 
2007 - 应用系统脆弱性概论
2007 - 应用系统脆弱性概论 2007 - 应用系统脆弱性概论
2007 - 应用系统脆弱性概论 Na Lee
 
The Django Book Chapter 9 - Django Workshop - Taipei.py
The Django Book Chapter 9 - Django Workshop - Taipei.pyThe Django Book Chapter 9 - Django Workshop - Taipei.py
The Django Book Chapter 9 - Django Workshop - Taipei.pyTzu-ping Chung
 
Super Advanced Python –act1
Super Advanced Python –act1Super Advanced Python –act1
Super Advanced Python –act1Ke Wei Louis
 
Django e il Rap Elia Contini
Django e il Rap Elia ContiniDjango e il Rap Elia Contini
Django e il Rap Elia ContiniWEBdeBS
 
NoSql Day - Apertura
NoSql Day - AperturaNoSql Day - Apertura
NoSql Day - AperturaWEBdeBS
 
Django mongodb -djangoday_
Django mongodb -djangoday_Django mongodb -djangoday_
Django mongodb -djangoday_WEBdeBS
 

Viewers also liked (18)

Authentication & Authorization in ASPdotNet MVC
Authentication & Authorization in ASPdotNet MVCAuthentication & Authorization in ASPdotNet MVC
Authentication & Authorization in ASPdotNet MVC
 
The Django Book, Chapter 16: django.contrib
The Django Book, Chapter 16: django.contribThe Django Book, Chapter 16: django.contrib
The Django Book, Chapter 16: django.contrib
 
User-centered open source
User-centered open sourceUser-centered open source
User-centered open source
 
Overview of Testing Talks at Pycon
Overview of Testing Talks at PyconOverview of Testing Talks at Pycon
Overview of Testing Talks at Pycon
 
Digesting jQuery
Digesting jQueryDigesting jQuery
Digesting jQuery
 
EuroDjangoCon 2009 - Ein Rückblick
EuroDjangoCon 2009 - Ein RückblickEuroDjangoCon 2009 - Ein Rückblick
EuroDjangoCon 2009 - Ein Rückblick
 
Vim for Mere Mortals
Vim for Mere MortalsVim for Mere Mortals
Vim for Mere Mortals
 
PythonBrasil[8] closing
PythonBrasil[8] closingPythonBrasil[8] closing
PythonBrasil[8] closing
 
Html5 History-API
Html5 History-APIHtml5 History-API
Html5 History-API
 
Django - The Web framework for perfectionists with deadlines
Django - The Web framework for perfectionists with deadlinesDjango - The Web framework for perfectionists with deadlines
Django - The Web framework for perfectionists with deadlines
 
2007 - 应用系统脆弱性概论
2007 - 应用系统脆弱性概论 2007 - 应用系统脆弱性概论
2007 - 应用系统脆弱性概论
 
The Django Book Chapter 9 - Django Workshop - Taipei.py
The Django Book Chapter 9 - Django Workshop - Taipei.pyThe Django Book Chapter 9 - Django Workshop - Taipei.py
The Django Book Chapter 9 - Django Workshop - Taipei.py
 
Super Advanced Python –act1
Super Advanced Python –act1Super Advanced Python –act1
Super Advanced Python –act1
 
Django e il Rap Elia Contini
Django e il Rap Elia ContiniDjango e il Rap Elia Contini
Django e il Rap Elia Contini
 
NoSql Day - Apertura
NoSql Day - AperturaNoSql Day - Apertura
NoSql Day - Apertura
 
Website optimization
Website optimizationWebsite optimization
Website optimization
 
Django mongodb -djangoday_
Django mongodb -djangoday_Django mongodb -djangoday_
Django mongodb -djangoday_
 
2 × 3 = 6
2 × 3 = 62 × 3 = 6
2 × 3 = 6
 

More from David Arcos

Definiendo Estrategia: Mapas de Wardley
Definiendo Estrategia: Mapas de WardleyDefiniendo Estrategia: Mapas de Wardley
Definiendo Estrategia: Mapas de WardleyDavid Arcos
 
Does Django scale? An introduction to Scalability
Does Django scale? An introduction to ScalabilityDoes Django scale? An introduction to Scalability
Does Django scale? An introduction to ScalabilityDavid Arcos
 
Security for Data Scientists
Security for Data ScientistsSecurity for Data Scientists
Security for Data ScientistsDavid Arcos
 
Efficient Django
Efficient DjangoEfficient Django
Efficient DjangoDavid Arcos
 
The Zen of Python
The Zen of PythonThe Zen of Python
The Zen of PythonDavid Arcos
 
12 tips on Django Best Practices
12 tips on Django Best Practices12 tips on Django Best Practices
12 tips on Django Best PracticesDavid Arcos
 

More from David Arcos (6)

Definiendo Estrategia: Mapas de Wardley
Definiendo Estrategia: Mapas de WardleyDefiniendo Estrategia: Mapas de Wardley
Definiendo Estrategia: Mapas de Wardley
 
Does Django scale? An introduction to Scalability
Does Django scale? An introduction to ScalabilityDoes Django scale? An introduction to Scalability
Does Django scale? An introduction to Scalability
 
Security for Data Scientists
Security for Data ScientistsSecurity for Data Scientists
Security for Data Scientists
 
Efficient Django
Efficient DjangoEfficient Django
Efficient Django
 
The Zen of Python
The Zen of PythonThe Zen of Python
The Zen of Python
 
12 tips on Django Best Practices
12 tips on Django Best Practices12 tips on Django Best Practices
12 tips on Django Best Practices
 

Recently uploaded

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESmohitsingh558521
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024BookNet Canada
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxBkGupta21
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 

Recently uploaded (20)

Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICESSALESFORCE EDUCATION CLOUD | FEXLE SERVICES
SALESFORCE EDUCATION CLOUD | FEXLE SERVICES
 
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
Transcript: New from BookNet Canada for 2024: Loan Stars - Tech Forum 2024
 
unit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptxunit 4 immunoblotting technique complete.pptx
unit 4 immunoblotting technique complete.pptx
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 

Python on a Plane

  • 1. Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 2. Abstract ● An entertainment system at 10,000 metres – Embedded computer in each plane – Synchronized after each flight – And a Platform with lots of integrations ● This is how we do it Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 3. Hi! ● I'm David Arcos ● Python/Django developer since 2008 ● Interested in distributed systems, databases, scalability, security ● Backend engineer at Immfly Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 4. Immfly: In-Flight Entertainment Immfly is a new Entertainment, Retail and Communication platform for the in-flight experience. Focused on the European domestic flights market, Immfly offers wireless content to passengers via their Personal Electronic Devices. Private & Confidential Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 5. The three challenges 1) On board the aircraft: ● Provide a Backend with lots of features ● Will work off-line 2) Synchronize the aircrafts ● Keep the replicas consistent ● Update the contents: I don't want yesterday's newspaper! 3) Integrate with 3rd parties ● Content providers, payments, mailing, weather, flight info... Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 6. 1) On board the aircraft Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 7. Some requirements ● Users have different needs ● Common authentication ● It's offline ● Show some flight info ● Deliver static content Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 8. Users have different needs ● Apps: web, android, iOS ● Devices: laptop, smartphone, tablet ● Return different results based on language, destination, airline, schedule... Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 9. Users have different needs ● Apps: web, android, iOS – Same REST API for every frontend ● Devices: laptop, smartphone, tablet – Generate thumbnails with different sizes to allow a responsive design ● Return different results based on language, destination, airline, schedule... – API calls allow filtering by many parameters Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 10. Common authentication ● Same user, different apps – Example: web app opens mobile app – Example: mobile app embeds a webview Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 11. Common authentication ● Same user, different apps – Example: web app opens mobile app – Example: mobile app embeds a webview ● Django Rest Framework has TokenAuthentication – Just a HTTP header, example: Authorization: Token 9944b09199c62bcf9418ad846dd0e4bbdfc6ee4b Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 12. It's off-line! ● Can't use any external service: – No analytics, online error logging, google maps – No fancy SaaS integrations – Can't just use CDN, email or DNS servers Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 13. It's off-line! ● Can't use any external service: – No analytics, online error logging, google maps – No fancy SaaS integrations – Can't just use CDN, email or DNS servers ● We deployed our own solutions – Very time-consuming – Can't hotfix bugs. Need to do lots of Q&A. Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 14. Show some flight info ● Estimated Time of Arrival? ● Weather at destination ● Show a nice map ● Where is the plane? Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 15. Show THIS flight info Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 16. Lots of flight info ● Estimated Time of Arrival? – Pre-load Flightstats API ● Weather at destination – Pre-load OpenWeatherMap (pyowm) ● Show a nice map – OpenStreetMap ● Where is the plane? – The plane tells us :) Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 17. Avionics data bus We get data in real-time: ● altitude ● flight_id ● ground_speed ● heading ● latitude ● longitude ● mach_speed ● outside_temperature ● pitch ● roll ● wind_speed ● yaw Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 18. Discrete-time signals ● DCFAILSIG ● ACFAILSIG ● OVERTEMPSIG ● GSM_POWER_STATUS ● ENB2SIG ● ENB1SIG ● ENB0SIG ● GSMSIG_STATUS ● CPLD_REV0 ● CPLD_REV1 ● SYSENSIG ● ENB3SIG ● ENB4SIG ● ENB5SIG ● ALERT ● CONFIGSIG0 ● CONFIGSIG1 ● CONFIGSIG2 ● INTTEST_OUT ● INTTEST_IN ● ISO_OUT0 ● ISO_OUT1 ● ISO_OUT2 ● ISO_OUT3 ● GPIO_DCFAILSIG ● GPIO_ACFAILSIG ● GPIO_OVERTEMPSIG ● GPIO_SYSENSIG Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 19. Deliver static content ● Provide the user with different contents: – TV Shows, Videos (format? size?) – Newspapers, Magazines – Images Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 20. Deliver static content ● Provide the user with different contents: – TV Shows, Videos – Newspapers, Magazines – Images ● Just use nginx, but: – Transcode video to HLS (bitrate, chunks) – Pre-process PDFs – Load test to verify assumptions Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 21. Other components Our django apps: – common – content – flightinfo – images – stats – surveys – users Some external libraries: – Django – Django Rest Framework – django-uuidfield – django-redis – django-celery – django-extensions – django-imagekit Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 22. Aircraft Infrastructure HTTP server: + + Databases: & Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 23. Embedded computer (lab) Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 24. Access point + dashboard (lab) Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 25. Hardware schema Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 26. The three challenges 2) Synchronize the aircrafts ● Keep the replicas consistent ● Update the contents: I don't want yesterday's newspaper! Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 27. 2) Synchronize the aircrafts Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 28. Context ● An Aircraft needs to update contents and usage ● During the flight, the Aircraft is off-line ● Once landed, it gets connectivity ● Then it tries to synchronize Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 29. Connectivity ● Internal 3G data card – Enabled when grounded – Disabled when about to flight ● VPN (“control tower”) – SSH to the Hangar – Hangar can SSH back Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 30. The Hangar ● Central point of the platform ● Special instance, in the cloud – Same software (db schema) – Different apps and settings ● Used as a CMS, to manage all resources – django.contrib.admin ● All other instances just replicate some data from the Hangar Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 31. The Heartbeat ● An Aircraft sends Heartbeats when it's online – It's just a “ping” to the Hangar – A simple POST, with some details ● The Hangar stores the heartbeats – Knows what planes are online, grounded, at a given moment – Will send commands to those planes ● In example: synchronize Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 32. Aircraft synchronization ● An Aircraft: – Sends usage data ● New/updated users, surveys, payments, stats, etc – Gets usage data ● New/updated users – Gets updated contents ● New/updated resources, contents, offers, destinations, misc data... Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 33. Trouble? ● For some models (like Users), the Primary Keys will collide ● Can get conflicts ● May need to merge resources, edited in several Aircrafts at the same time Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 34. Prevent the trouble ● For some models (like Users), the Primary Keys will collide – Those PKs must be UUIDs ● Can get conflicts – Each model should be synchronized only in one direction: A->H or H->A ● May need to merge resources, edited in several Aircrafts at the same time – Make sure our logic doesn't allow that Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 35. fabric Python library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks http://fabfile.org Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 36. fabtools.require fabtools includes useful functions to help you write your Fabric files. Using fabtools.require allows you to use a more declarative style, similar to Chef or Puppet. http://fabtools.readthedocs.org Example: Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 37. Other tools ● Python – boto: upload/download from S3 – requests: API calls – django.db.migrations ● Unix – openssh – rsync – bash script Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 38. The three challenges 3) Integrate with 3rd parties ● Providers, payments, mailing, weather, flight info... Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 39. 3) Integrate with 3rd parties Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 40. Highlights ● Resource Ingest ● API diversity ● AWS ● Celery ● Other APIs Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 41. Resource ingest ● Entertainment – Newspapers – Magazines – Videos – TV Shows ● Deals – Airport – Transport – Destination ● Lots of resources with lots of providers – Per language, country, etc Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 42. API diversity... ● Best case – Python API :) ● Good enough – REST API, lots of metadata, well documented ● Acceptable – S3 bucket or FTP, import all, but no metadata ● Worst case: – ? Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 43. Beware of the Intern-API! https://www.flickr.com/photos/reidrac/2387432357/ Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 44. Is this even a API? ● Word document – (why?) ● Excel document – (no format, of course) Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 45. AWS, using boto ● ElasticTranscoder – Media transcoding in the cloud ● Simple Notification Service (SNS) – A fast, flexible, fully managed push messaging service ● Simple Storage Service (S3) – Secure, durable, highly-scalable object storage. Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 46. Celery task queue ● Hangar tasks use Celery: – Celery is an asynchronous task queue/job queue based on distributed message passing – Async – I/O intensive tasks: crawling APIs – CPU intensive tasks: thumbnails, billing pdfs Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 47. Flower: Celery monitoring tool ● Flower is a web based tool for monitoring and administrating Celery clusters – http://flower.readthedocs.org ● Real time ● Tip: a queue per task type Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 48. Other APIs ● Bookings ● Payments ● Billing ● Emails ● Weather ● Flight info ● Statistics ● Slack Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 49. Other libraries ● Images: – Pillow – django-imagekit – pilkit – PyPDF2 – Wand ● PDFs: – django-easy-pdf – xhtml2pdf ● Other: – django-yubin – libsaas Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 50. Recap ● Solved the three challenges! – In-flight API with tons of features – When grounded, they get synchronized – The Hangar manages the full platform Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 51. Conclusions ● Python made it possible! ● Very versatile, covers all our use cases ● “We stand on the shoulders of giants” ● Developed in a short time Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 52. Thanks for attending! Get the slides at http://slideshare.net/DZPM Any questions? Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 53. Questions? Python on a Plane – PyConES 2014 David Arcos - @DZPM
  • 54. Thanks for attending! Get the slides at http://slideshare.net/DZPM Enjoy your flight! http://immfly.com Python on a Plane – PyConES 2014 David Arcos - @DZPM