Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Django for IoT: From hackathon to production (DjangoCon US)

1,372 views

Published on

Talk given at DjangoCon US 2016
Also check out the code: https://github.com/aschn/cookiecutter-django-iot

Published in: Engineering
  • Be the first to comment

Django for IoT: From hackathon to production (DjangoCon US)

  1. 1. Django for Internet of Things: from hackathon to production Anna Schneider DjangoCon US, 18 July 2016 @windupanna
  2. 2. I’m Anna :) Co-founder and CTO at WattTime WattTime.org @wattTime
  3. 3. UnconsciousBiasProject.org @UBP_STEM
  4. 4. what?? Django for Internet of Things: from hackathon to production
  5. 5. everyone’s favorite framework 🎸🐍 Django
  6. 6. Internet of Things (IoT) when a thing you don’t normally think of as a computer can transmit data and respond to controls in real time 🏡🚗 📡🎮⏰
  7. 7. some people like to write the code on the thing, I like to talk to things through their APIs Internet of Things (IoT) ☁️ 🚗☁️you them the thing
  8. 8. a really fun way to write really fragile code really quickly hackathon 💻☕️ 👏⚠️
  9. 9. when you can ship your code and trust it while you’re having fun at DjangoCon production 💻 🍎 🚢 😎
  10. 10. design patterns (and anti-patterns) for writing and deploying Django projects to monitor and control an IoT device using its API so you can get started fast then build well what you’ll learn ☕️ 🚢 😱
  11. 11. python manage.py startproject awesome_iot_hackathon 💻☕️ 👏⚠️
  12. 12. models!
  13. 13. the books app Book title author year Book title author year Author name hometown Book title author year Book title author year Author name hometown Author name hometown
  14. 14. the books IoT app Observation value device timestamp Device name location Device name location Device name location Observation value device timestamp Observation value device timestamp Observation value device timestamp
  15. 15. the IoT app Device name location vendor ID Device name location vendor ID Device name location vendor ID ☕️ 🚢 vendor’s unique ID(s) whatever data you need to send to their API Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp
  16. 16. Observation value device timestamp the IoT app plan for big(gish) time series data eg use db_index 🚢 Device name location vendor ID Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Device name location vendor ID Device name location vendor ID
  17. 17. the IoT app Attribute vs Status numerical vs str/bool values Observation value device timestamp Device name location vendor ID Observation value device timestamp Observation value device timestamp Attribute value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Attribute value device timestamp Observation value device timestamp Observation value device timestamp Observation value device timestamp Attribute value device timestamp Device name location vendor ID Device name location vendor ID Observation value device timestamp Observation value device timestamp Observation value device timestamp Status is_on device timestamp ☕️ 🚢
  18. 18. views? models
  19. 19. views? tasks!
  20. 20. ☁️ 🚗☁️you them the thing request response ☁️ 💻you them request response normal views IoT tasks
  21. 21. what tasks do we need to do? pull new attributes pull new on/off statuses set attributes set on/off statuses monitor control
  22. 22. tasks.py def do_something_awesome(device): # decide to turn on or off is_on = run_decision_criterion() # set status using device API set_status(device, is_on) # create status in db status = device.status_set.create( is_on=is_on, valid_at=timezone.now(), ) # return return status ☕️ the awesome part
  23. 23. tasks.py def do_something_awesome(device): # decide to turn on or off is_on = run_decision_criterion() # set status using device API set_status(device, is_on) # create status in db status = device.status_set.create( is_on=is_on, valid_at=timezone.now(), ) # return return status 🚢 bad for asynchronous 😱
  24. 24. tasks.py def do_something_awesome(device_id): # get device from pk device = Device.objects.get( pk=device_id ) # decide to turn on or off is_on = run_decision_criterion() # set status using device API set_status(device, is_on) # create status in db status = device.status_set.create( is_on=is_on, valid_at=timezone.now(), ) # return return [status.pk] pass pks pro: don’t rely on database state con: may be extra queries 🚢
  25. 25. put it together! models tasks
  26. 26. the hackathon app myapp models.py Device Attribute Status tasks.py pull_status set_status pull_attributes set_attributes client.py admin.py views.py tests.py ☕️
  27. 27. the production apps devices models.py admin.py views.py tests.py interactions tasks.py views.py tests.py vendor client.py tests.py observations models.py admin.py views.py tests.py views for adding/removing devices analytics, DRF for dashboards models for logging, views for clickable tasks swappable vendors 🚢
  28. 28. deploy! models tasks apps
  29. 29. deploy tasks? goals: • run any task (control or monitor) • at frequent, deterministic times • outside of request/response cycle cron in the cloud
  30. 30. 1) the hackathon way two ways to automate management commands + Heroku Scheduler ☕️ python manage.py do_something_awesome --device_id=1
  31. 31. Heroku Scheduler pros: • easy to set up cons: • limited frequencies (daily, hourly, 10 min) • “best effort” service (may skip some) ☕️
  32. 32. 2) the production way task functions + celery periodic task scheduler 🚢 two ways to automate
  33. 33. Celery distributed message queuing system for asynchronous stuff slow event-driven tasks send the user sign-up email scheduled periodic tasks run the daily report 🚢
  34. 34. web servers Celery architecture (how) tasks tasks worker servers messages results result store message broker transport queue messages tasks scheduler 🚢
  35. 35. Celery architecture (how) scheduler do_awesome do_awesome worker status.pk ☁️ turn on on! Status(is_on=True) 🚢
  36. 36. from celery import shared_task @shared_task def set_status(device_id): ... @shared_task def do_something_awesome(device_id): ... just add decorator :) tasks.py 🚢 (what)
  37. 37. from celery.schedules import crontab SCHEDULE = { 'run_task': { # use the actual path 'task': ‘do_something_awesome’, # every 5 minutes 'schedule': crontab(minute='*/5'), # args and kwargs 'args': [], 'kwargs': {}, }, } very close to crontab once per minute - once per year schedule.py 🚢 (when)
  38. 38. from celery.schedules import crontab SCHEDULE = { 'run_task': { # use the actual path 'task': ‘do_something_awesome’, # every 5 minutes 'schedule': crontab(minute='*/5'), # args and kwargs 'args': [], 'kwargs': {'device_id': d.pk}, } for d in Device.objects.all() } schedule.py 🚢 static arguments only code is only evaluated once at run time, better to have one task spawn daughters (when) 😱
  39. 39. from celery.schedules import crontab SCHEDULE = { 'run_task': { # use the actual path 'task': ‘run_all', # every 5 minutes 'schedule': crontab(minute='*/5'), # args and kwargs 'args': [], 'kwargs': {}, } } @shared_task def run_all(): for d in Device.objects.all(): do_something_awesome(d.pk) schedule.py 🚢 static arguments only code is only evaluated once at run time, better to have one task spawn daughters (when)
  40. 40. production, Celerified devices models.py admin.py views.py tests.py interactions tasks.py schedule.py views.py tests.py brand client.py tests.py observations models.py admin.py views.py tests.py 🚢 django_iot __init__.py celery.py settings.py wsgi.py urls.py django_iot Procfile requirements.txt manage.py
  41. 41. cookiecutter https://github.com/aschn/cookiecutter-django-iot.git ☕️ 🚢
  42. 42. • think time series data: mind your models • async not request/response: tasks not views • ☕️ easy but rigid: Heroku Scheduler • 🚢 flexible but complex: celery periodic tasks • hack better+faster: cookiecutter-django-iot • IoT can be for good, not just for fun and profit what have we learned?
  43. 43. Thanks!! Anna Schneider @windupanna @wattTime cookiecutter https://github.com/aschn/cookiecutter-django-iot.git

×