Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

SparkWeaver: Full-Stack Solution to Accelerate Real-Time DNN Applications on FPGA-Enabled Spark Streaming

137 views

Published on

In recent years, Deep Neural Networks (DNNs) have rapidly advanced and reached a sufficiently mature state to be adopted in real-world applications. Especially, as DNNs solve difficult problems in computer vision (e.g., image classification and object detection), community has started exploring the use of DNNs in real-time or nearly real-time vision applications. Video content analysis (VCA) is one such application that often utilizes DNNs as its core engine and offers plentiful capabilities for a wide range of domains including safety and security, flame and smoke detection, automotive, health-care, home automation, and retail. Apache Spark Streaming has been the de-facto standard platform where real-time Big data applications such as VCA are run at hyper-scale. While the integration of DNNs and real-time vision applications promise ample opportunities for Spark Streaming community, the massive compute demand to accommodate (1) the ever-increasing DNN model size, and (2) the growing scale of data (e.g., billions of high-resolution video data) significantly limits its practicality. In this work, we seek to address this challenge and provide a solution to meet this gigantic compute demand by leveraging FPGA acceleration. We develop SparkWeaver, a full-stack solution that, from DNN-based real-time vision applications (e.g., VCA), automatically offloads the heavy DNN computations to our FPGA accelerators without the developers' intervention. We use FPGAs as our DNN acceleration platform since they not only offer low inference-latency and high power-efficiency, oftentimes required for real-time vision applications, but also provide a programmable substrate for acceleration of non-DNN components of the applications. To demonstrate the easy use of the solution, we will do a live-demo that shows the SparkWeaver's automated workflow that takes a DNN-based VCA application written using Spark Streaming APIs and runs the VCA application on a Spark cluster, while offloading DNN computations to FPGAs, without imposing additional manual efforts on the developers.

Published in: Data & Analytics
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • DOWNLOAD FULL eBOOK INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... 1.DOWNLOAD FULL. doc eBook here { https://tinyurl.com/y3nhqquc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, CookeBOOK Crime, eeBOOK Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

SparkWeaver: Full-Stack Solution to Accelerate Real-Time DNN Applications on FPGA-Enabled Spark Streaming

  1. 1. WIFI SSID: SparkAISummit | Password: UnifiedAnalytics
  2. 2. Behnam Robatmili, Jongse Park, and Blake Skinner Bigstream Solutions SparkWeaver: Accelerating Real-time DNN Applications with Spark and DNNWEAVER #UnifiedAnalytics #SparkAISummit
  3. 3. A little about Bigstream 3 Cross platform Cross acceleration hardware Intelligent, automatic computation slicing Zero code change Dataflow Adaptation Layer Bigstream Dataflow Bigstream Hypervisor HYPER-ACCELERATION 2X to 30X acceleration BIG DATA PLATFORMS Many-cores GPU FPGA
  4. 4. Ingest Bottleneck in Big Data 4
  5. 5. Applications with Ingest Bottleneck ● Many big data applications ○ Lots of raw data ○ Video surveillance ■ Industrial camera market is projected to increase 2.3x by 2024 [1] ■ For a 4k camera in 60fps, the amount of data per hour is 5.2 TB (1TB for 10fps) ○ Voice recognition ○ Fraud detection 5 1. https://www.gminsights.com/industry-analysis/ip-camera-market
  6. 6. Traditional Architecture does not Scale 6 Kafka DNN HDFS Spark Spark Streaming - Online analytics - Cross camera Raw image frames Batch processing - Offline Training - Offline analytics Image features 1) Ingest stage 2) Data streaming 3) Online analytics 4) Offline analytics
  7. 7. Use Cases ● How many people went from the shoe department to the jewelry department? ● How many people were observed walking around the entire building on a given day? 7 Requires cross-camera online and offline analytics
  8. 8. Traditional Architecture does not Scale 8 Kafka DNN HDFS Spark Spark Streaming Online analytics Raw image frames Batch processing Offline analytics Image features Detection, Tracking, Anomaly detection, cross camera Re-Identification, …
  9. 9. Semantic Compression with DNNs ● DNNs can be used for compression ○ Converting raw data into condensed, semantic data ○ For video analytics, we observed a ~5x compression rate 9 1. https://www.gminsights.com/industry-analysis/ip-camera-market
  10. 10. Large Scale Image Processing ● Deep learning on traditional big data clusters presents many challenges ○ Computationally intensive ■ Adds pressure to the entire ETL toolchain ○ Traditional CPUs are not ideal for evaluating DNN models ○ Doing many levels of DL processing on every input frame requires ■ Storing a lot of raw data ■ Storing and managing all interim data 10
  11. 11. DNN Optimized Ingest 11 Kafka Server HDFS Online analytics Image Features Offline analytics Batch processing Spark Spark Streaming Datacenter Boundary
  12. 12. Challenges with DNNs ● Computationally expensive ● Require a lot of data and energy 12
  13. 13. DNNs with FPGA ● FPGAs are a good candidate ○ Faster than CPUs ○ More power efficient than GPUs ○ More programmable than ASICs ● Programmability ○ Need HDL 13
  14. 14. Solution: DNN+FPGA for Ingest ● Ingest only the data you need ○ Run DNNs on the edge ○ Condensed, meaningful features instead of raw, largely meaningless data ● Accelerate with FPGAs ○ Power efficient ○ Can be deployed with minimal infrastructure ○ Using DNNWEAVER technology for programmability ■ Compiler and full stack for automatic DNN acceleration 14
  15. 15. DNNWEAVER 15
  16. 16. DNNWEAVER ● Ease DNN Deployment to FPGAs ○ Tensorflow and ONNX ○ No code changes ○ No hardware expertise needed ● Open source implementation based on original paper[1] ● Enterprise version under development by Bigstream 16 1: https://github.com/hsharma35/dnnweaver2
  17. 17. End-to-end DNN acceleration 17 Tensorflow / ONNX Translator Macro Dataflow Graph Design Planner Resource Allocation Execution Schedule Design Weaver Hand Optimized Templates Integrator Memory Interface FPGA Binary Inputs Compiler Modules Internal Outputs Final Output
  18. 18. DNNWEAVER Compute Stack 18
  19. 19. SparkWeaver 19
  20. 20. SparkWeaver Architecture 20 Kafka Server HDFS Online analytics Image Features Offline analytics Batch processing FPGA Cluster Spark Spark Streaming
  21. 21. YOLO Detection and Tracking with YOLO[1] and Deep SORT[3] [1] Redmon et al. “You Only Look Once, Unified, Real-Time Object Detection” [2] https://github.com/thtrieu/darkflow [3] Wojke et. al. “Simple Online and Real Time Tracking with a Deep Association Metric” Bounding Boxes Tagged Bounding Boxes [2] [3] 21 Deep SORT Image sources:
  22. 22. SparkWeaver Architecture 22 Kafka HDFS Image Features Spark Detection: YOLOv2 Tracking: Deep_SORT, ...
  23. 23. SparkWeaver Architecture 23 Kafka HDFS Image Features Spark Detection: YOLOv2 Tracking: Deep_SORT, ... ● Multiple cameras stream video to an FPGA cluster ● FPGA clusters implement YOLOv2 with DNNWEAVER ● YOLO’s image features are streamed to a Kafka server ● Features are aggregated by Spark and written to HDFS ● Detection performed on the fog layer, tracking on the cluster
  24. 24. Single Node Max FPS 24 Benchmark FPS Traditional Architecture 7.3 Detection only 10 Tracking only 12.8 YOLO on DNNWEAVER 13.2 SparkWeaver 12.8 *Dependent on the number of people in a frame
  25. 25. Next Release Max FPS (projected) 25 Benchmark FPS SparkWeaver 46.1 *Dependent on the number of people in a frame
  26. 26. Single Node Compression Rate • 5.5x (82%) – Deep_SORT tracking only needs the pixels within the bounding boxes, and their locations 26
  27. 27. Demo 27
  28. 28. Streaming and Batch Analytic Operations 28
  29. 29. Streaming Analysis ● Person Re-Identification[1] ○ Multiple solution (hot area of research) ○ Some solutions use pre-trained DNNs[2] ■ Generate a feature vector and apply similarity check on vectors ● Anomaly Detection[3] ○ Suspicious events/threats 29 1. Zheng et al “Person Re-identification: Past, Present, and Future” 2. Hermans et al “In Defense of the Triplet Loss for Person Re-Identification” 3. https://databricks.com/blog/2018/09/13/identify-suspicious-behavior-in-video-with-databricks-runtime-for-machine-learning.html
  30. 30. Use Case 1 How many people went from department X to department Y? 30 select * from people_present where camera_id == X as peopleX select * from people_present where camera_id == Y as peopleY count(select person_id from peopleX inner join peopleY where WITHIN_THRESHOLD(peopleX.exit, peopleY.enter)) people_present person_id: INT, camera_id: INT, enter: TIMESTAMP, exit: TIMESTAMP
  31. 31. Use Case 2 How many people were observed walking around the entire building on a given day? 31 select id from unique_people where count(select * from people_present where camera_id == CAMERA1) > 1 and count(select * from people_present where camera_id == CAMERA2) > 1 and count(select * from people_present where camera_id == CAMERA3) > 1 and ... unique_people id: INT
  32. 32. Conclusion ● DNN-optimized ingest ○ Smart compression ○ Use network resources on dense, highly meaningful data rather than sparse, raw data 32
  33. 33. Conclusion ● FPGAs are well suited to DNN acceleration edge computing ○ Highly parallel ○ Power efficient ○ Can be deployed with minimal resources ○ DNNWEAVER can compile Tensorflow/ONNX to FPGA ● Not just DNNs, also infrastructure ○ Online and offline analytics ○ Moving data 33
  34. 34. About the Authors 34 behnam@bigstream.co jongse@bigstream.co blake@bigstream.co
  35. 35. DON’T FORGET TO RATE AND REVIEW THE SESSIONS SEARCH SPARK + AI SUMMIT

×