Download to read offline
![Project StreamBox
stream analytics at the memory speed
24
• RDMA
• Co‐designed with
mm/scheduling
• RDMA
• Co‐designed with
mm/scheduling
Stream pipeline Threads
Ingestion
Scheduler
Mem
• Squeeze parallelism for
multi/manycore
• Manage NUMA domains
• Squeeze parallelism for
multi/manycore
• Manage NUMA domains
Exploit high‐bandwidth memoryExploit high‐bandwidth memory
[ASPLOS'19] "StreamBox‐HBM: Stream Analytics on High Bandwidth Hybrid Memory," Hongyu Miao, Myeongjae Jeon, Gennady Pekhimenko,
Kathryn S. McKinley, and Felix Xiaozhu Lin
[USENIX ATC'17] "StreamBox: Modern Stream Processing on a Multicore Machine," Hongyu Miao, Heejin Park, Myeongjae Jeon, Gennady
Pekhimenko, Kathryn S. McKinley, and Felix Xiaozhu Lin, in Proc. USENIX Annual Technical Conference, 2017.
[ASPLOS'16] "memif: Towards Programming Heterogeneous Memory Asynchronously," Felix Xiaozhu Lin and Xu Liu, in Proc. ACM Int. Conf.
Architectural Support for Programming Languages and Operating Systems, 2016.](https://image-private.slidesharecdn.com/2020-06osfrontiersintheaiera-200608192116/85/slide-18-320.jpg?hdnea=acl=/2020-06osfrontiersintheaiera-200608192116/85/slide-18-320.jpg*~exp=1771110351~hmac=71af1196066c5ccdc8300195c08cc45dfd0ec5e79cdea57bc251b3d7309ab8cf&cb=1591644116)
![Case 2: Autonomous AI on
Cameras
[MobiSys'20] "Approximate Query Processing on Autonomous Cameras," Mengwei Xu, Xiwen Zhang, Yunxin Liu,
Xuanzhe Liu, and Felix Xiaozhu Lin 29](https://image-private.slidesharecdn.com/2020-06osfrontiersintheaiera-200608192116/85/slide-23-320.jpg?hdnea=acl=/2020-06osfrontiersintheaiera-200608192116/85/slide-23-320.jpg*~exp=1771110351~hmac=dc82f09d6aa2d82e9de7c7b0eceaaf01c5b044a8b0f5d5e4b155135b576e715b&cb=1591644116)
![Query: (car, 30 mins)
Install
7:00AM-7:30AM [500 + 100] Cars
7:30AM-8:00AM [700 + 140] Cars
8:00AM-8:30AM [800 + 180] Cars
8:30AM-9:00AM [400 + 100] Cars
9:30AM-10:00AM [200 + 80] Cars
Sample &
capture
200‐80 +80
Elf: Query model
38](https://image-private.slidesharecdn.com/2020-06osfrontiersintheaiera-200608192116/85/slide-32-320.jpg?hdnea=acl=/2020-06osfrontiersintheaiera-200608192116/85/slide-32-320.jpg*~exp=1771110351~hmac=cb3f9f165f5813a99c5c5f35f6297eadbb731d51d933b1b8e8a9287ebbfb8055&cb=1591644116)
![7:00AM-7:30AM [500 + 100] Cars
7:30AM-8:00AM [700 + 140] Cars
8:00AM-8:30AM [800 + 180] Cars
8:30AM-9:00AM [400 + 100] Cars
9:30AM-10:00AM [200 + 80] Cars
Ground
Truth
Error:
11%
Confidence interval width: 17%
Auburn, AL
Hampton, NY Jackson, WY
Taipei Taipei
~1000 hours
41](https://image-private.slidesharecdn.com/2020-06osfrontiersintheaiera-200608192116/85/slide-35-320.jpg?hdnea=acl=/2020-06osfrontiersintheaiera-200608192116/85/slide-35-320.jpg*~exp=1771110351~hmac=d4eb6111fe7580a187f9acffbfe1843c934c96dd2e69d5f235ff2fc99f3b0ff6&cb=1591644116)
![49
CPU Co
Proc
DRAM IO
Co‐processor translates unmodified kernel binary
Dynamic
Binary
Translation
Linux Kernel
IO
tasks
[USENIX ATC 19] "Transkernel: An Executor for Commodity Kernels on Peripheral Cores,"
Liwei Guo, Shuang Zhai, Yi Qiao, and Felix Xiaozhu Lin](https://image-private.slidesharecdn.com/2020-06osfrontiersintheaiera-200608192116/85/slide-43-320.jpg?hdnea=acl=/2020-06osfrontiersintheaiera-200608192116/85/slide-43-320.jpg*~exp=1771110351~hmac=416c23c929ca58de179ed6dd8ec8d8eabed842d9b19dac704dbcc7234026b003&cb=1591644116)
OS frontiers in the AI era
- 1.
- 2. Who Am I • Fall 2020. Assoc. Prof, UVA CS – Looking forward to it! •2014 – 2020. Asst. Prof, Purdue ECE • 2014 PhD in CS. Rice • 2008 MS + BS. Tsinghua 2
- 3.
- 4.
- 5. The 3rd boom of AI 1960s 1970s1980s 1990s 2000s 2010s 2020s Classical AI Expert systems Machine learning Ups Downs 6
- 6. Two Pendulums 1960s 1970s 1980s1990s 2000s 2010s 2020s Expert systems Machine learning Ups Downs Centralized Decentralized Dedicated hw & net PC + Internet Datacenters Cloud computing 5G, edge, embedded AI… 7 Classical AI
- 7. Two Pendulums 1960s 1970s 1980s1990s 2000s 2010s 2020s Expert systems Machine learning Generalized Specialized Microcomputers, x86, DBMS, PC Mainframes, Apollo11 Ups Downs Wearables/IoT, NoSQL, GPU/FPGA, RISCV Centralized Decentralized Dedicated hw & net PC + Internet Datacenters Cloud computing 5G, edge, embedded AI… 8 Classical AI
- 8. Two Pendulums 1960s 1970s 1980s1990s 2000s 2010s 2020s Expert systems Machine learning Generalized Specialized Microcomputers, x86, DBMS, PC Mainframes, Apollo11 Ups Downs Wearables/IoT, NoSQL, GPU/FPGA, RISCV Centralized Decentralized Dedicated hw & net PC + Internet Datacenters Cloud computing 5G, edge, embedded AI… 9 Classical AI
- 9.
- 10. What you have learnt from undergrad OS class 1960s 1970s 1980s1990s 2000s 2010s 2020s GOFAI Expert systems Machine learning Generalized Specialized Microcomputers, x86, DBMS, PC Mainframes, Apollo11 Ups Downs Wearables/IoT, NoSQL, GPU/FPGA, RISCV Centralized Decentraliz ed Dedicated hw & net PC + Internet Datacenters Cloud computing 5G, edge, embedded AI… 14
- 11. What OS people are working on 1960s 1970s 1980s1990s 2000s 2010s 2020s GOFAI Expert systems Machine learning Generalized Specialized Microcomputers, x86, DBMS, PC Mainframes, Apollo11 Ups Downs Wearables/IoT, NoSQL, GPU/FPGA, RISCV Centralized Decentraliz ed Dedicated hw & net PC + Internet Datacenters Cloud computing 5G, edge, embedded AI… 15
- 12.
- 13. • Diverse • A few common OSes •Many specialized ones • Serving “things” in addition to humans • Defined by scenarios • Generic OSes firmware • Specialized OSes overlays • Blurry boundaries • arch, runtime, compiler, kernel, hypervisor, trusted exec environment… Cloud Edge Devices 18
- 14.
- 15.
- 16.
- 17. Stream analytics: state of the art • Classic engines? • StreamBase, Aurora, TelegraphCQ, NiagaraST… •Single threaded. Not scaling well. • Modern engines for datacenters? • Apache Flink, Spark Streaming, Beam… • Designed for tens ‐ hundreds of machines. Scaling out. • Assuming okay if individual nodes perform poorly • As analytics moves to the edge bad 23
- 18. Project StreamBox stream analytics at the memory speed 24 • RDMA • Co‐designed with mm/scheduling •RDMA • Co‐designed with mm/scheduling Stream pipeline Threads Ingestion Scheduler Mem • Squeeze parallelism for multi/manycore • Manage NUMA domains • Squeeze parallelism for multi/manycore • Manage NUMA domains Exploit high‐bandwidth memoryExploit high‐bandwidth memory [ASPLOS'19] "StreamBox‐HBM: Stream Analytics on High Bandwidth Hybrid Memory," Hongyu Miao, Myeongjae Jeon, Gennady Pekhimenko, Kathryn S. McKinley, and Felix Xiaozhu Lin [USENIX ATC'17] "StreamBox: Modern Stream Processing on a Multicore Machine," Hongyu Miao, Heejin Park, Myeongjae Jeon, Gennady Pekhimenko, Kathryn S. McKinley, and Felix Xiaozhu Lin, in Proc. USENIX Annual Technical Conference, 2017. [ASPLOS'16] "memif: Towards Programming Heterogeneous Memory Asynchronously," Felix Xiaozhu Lin and Xu Liu, in Proc. ACM Int. Conf. Architectural Support for Programming Languages and Operating Systems, 2016.
- 19. CoresCores High‐bandwidth hybrid memory 25 3D DRAM NormalDRAM Tradeoffs: capacity vs. bandwidth Untraditional memory hierarchy No latency benefit Unlike SRAM+DRAM 16 GB 375 GB/s ~100 GB ~100 GB/s
- 20. 26 Already on off‐the‐shelf machinesIntel Xeon PhiKnights Landing (KNL)
- 21.
- 22.
- 23. Case 2: Autonomous AI on Cameras [MobiSys'20] "Approximate Query Processing on Autonomous Cameras," Mengwei Xu, Xiwen Zhang, YunxinLiu, Xuanzhe Liu, and Felix Xiaozhu Lin 29
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32. Query: (car, 30mins) Install 7:00AM-7:30AM [500 + 100] Cars 7:30AM-8:00AM [700 + 140] Cars 8:00AM-8:30AM [800 + 180] Cars 8:30AM-9:00AM [400 + 100] Cars 9:30AM-10:00AM [200 + 80] Cars Sample & capture 200‐80 +80 Elf: Query model 38
- 33. Camera Operating System Planning Energy via Reinforcement Learning Planning Energy via Reinforcement Learning Sampledframes Aggregator with error Integration Aggregator with error Integration Selected NeuralNet Object counts Elf: System Internals 39
- 34.
- 35. 7:00AM-7:30AM [500 +100] Cars 7:30AM-8:00AM [700 + 140] Cars 8:00AM-8:30AM [800 + 180] Cars 8:30AM-9:00AM [400 + 100] Cars 9:30AM-10:00AM [200 + 80] Cars Ground Truth Error: 11% Confidence interval width: 17% Auburn, AL Hampton, NY Jackson, WY Taipei Taipei ~1000 hours 41
- 36.
- 37.
- 38.
- 39.
- 40. 46 CPU Co Proc 2.5GHz 50MHz DRAMIO A heterogeneous SoC
- 41. 47 CPU Co Proc 2.5GHz 50MHz DRAMIO Weak co‐processors: suits low‐power IO tasks! high efficiency Linux Kernel IO tasks
- 42. 48 CPU Co Proc 2.5GHz 50MHz DRAMIO Kernel execution on weak co‐processors? Linux Kernel IO tasks Diff ISA No MMU …
- 43. 49 CPU Co Proc DRAM IO Co‐processor translates unmodified kernel binary Dynamic Binary Translation LinuxKernel IO tasks [USENIX ATC 19] "Transkernel: An Executor for Commodity Kernels on Peripheral Cores," Liwei Guo, Shuang Zhai, Yi Qiao, and Felix Xiaozhu Lin
- 44.
- 45. Recap • What are OSes in 2020? • Three OS projects: •spanning IoT, mobile, and datacenters • each with different flavors • The builder culture • open blackboxes • break things • build things from the ground up • Started by a small group of hardcore hackers • Now more diverse and inclusive • A brave new world 51 StreamBox Elf
- 46.