Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PyParis2017 / Incremental computation in python, by Philip Schanely

121 views

Published on

PyParis 2017
http://pyparis.org

Published in: Technology
  • Be the first to comment

  • Be the first to like this

PyParis2017 / Incremental computation in python, by Philip Schanely

  1. 1. Incremental Computation in Python Phillip Schanely pschanely @medium pschanely @twitter
  2. 2. The New Messaging App
  3. 3. You have no new messages Joe IDK, let’s go to Sal’s Sarah LOLZ Mike Right? I’ve never …
  4. 4. You have no new messages Joe IDK, let’s go to Sal’s Sarah LOLZ Mike Right? I’ve never …
  5. 5. You have no new messages Joe IDK, let’s go to Sal’s Sarah LOLZ Mike Right? I’ve never …
  6. 6. The Model-View Problem Incremental MapReduce in Python Contents:
  7. 7. The Model-View Problem
  8. 8. Model, M View, V
  9. 9. Model, M State Change Model, M’ View, V
  10. 10. Model, M State Change Model, M’ View, V View, V’ Decision Time!
  11. 11. Model, M State Change Recompute Model, M’ View, V View, V’
  12. 12. Model, M State Change Infer Model, M’ View, V View, V’
  13. 13. Model, M State Change Invalidate Model, M’ View, V View, V’
  14. 14. Recompute Invalidate Infer
  15. 15. Recompute Invalidate Infer Incremental MapReduce
  16. 16. It’s Everywhere
  17. 17. Javascript Objects DOM Javascript Objects DOM Web Apps Infer Recompute
  18. 18. DOM Pixels DOM Pixels Browsers Invalidate
  19. 19. Application Pixels Display Application Pixels Display OS Display Infer
  20. 20. Database State API Responses Database State API Responses Backend Logic Recompute
  21. 21. Normalized Data Computed Data Normalized Data Computed Data Application Databases Infer Recompute
  22. 22. Database Row Database Index Database Row Database Index Databases Infer
  23. 23. RAM Processor Cache RAM Processor Cache Multiprocessor L1/L2 caches Invalidate
  24. 24. Code Build Code Build Build Systems Recompute
  25. 25. Code Test Results Code Test Results Test Runners Recompute
  26. 26. Incremental MapReduce
  27. 27. Model, M State Change Model, M’ View, V View, V’ Incremental MapReduce
  28. 28. 4 6 10 1 5 6 16
  29. 29. 4 6 10 1 5 6 16 4 6 1 5 3 Insert “3”
  30. 30. 4 6 10 1 5 6 16 4 6 1 5 3 Insert “3” 10 9 19 4
  31. 31. ScenicOverlook $ pip install ScenicOverlook
  32. 32. 4 6 1 5 >>> l = viewablelist([4,6,1,5])
  33. 33. 4 6 1 5 >>> l = viewablelist([4,6,1,5]) >>> add = lambda x, y: x + y >>> l.reduce(add, initializer=0) 16
  34. 34. 4 6 1 5 4 6 1 5 3 Insert “3” >>> l = viewablelist([4,6,1,5]) >>> add = lambda x, y: x + y >>> l.reduce(add, initializer=0) 16 >>> l = l[:2] + [3] + l[2:]
  35. 35. 4 6 1 5 4 6 1 5 3 Insert “3” >>> l = viewablelist([4,6,1,5]) >>> add = lambda x, y: x + y >>> l.reduce(add, initializer=0) 16 >>> l = l[:2] + [3] + l[2:] >>> l.reduce(add, initializer=0) 19
  36. 36. >>> import heapq >>> def sort(l): ... l = l.map(lambda x:[x]) ... return l.reduce(heapq.merge, initializer=()) >>> l = viewablelist([9, 3, 5, 7]) >>> sort(l) [3, 5, 7, 9] >>> sort(l + [ 4 ]) [3, 4, 5, 7, 9] 3 5 79 9 5 73 5 7 93
  37. 37. Graphics Demo
  38. 38. $ python demo_graphics.py
  39. 39. Compiler Demo
  40. 40. $ python demo_graphics.py
  41. 41. Normalize
  42. 42. Generate Call Tree Normalize
  43. 43. Type Check Generate Call Tree Normalize
  44. 44. Type Check Generate Call Tree Normalize LLVM Generation
  45. 45. Type Check Generate Call Tree Normalize LLVM Generation link
  46. 46. Umut Acar (CMU): “self-adjusting computation” Matthew A. Hammer: “Adapton” Interesting Related Work
  47. 47. github.com/pschanely/ScenicOverlook pschanely @medium pschanely @twitter Questions?

×