Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Differential Approximation and Sprinting for Multi-Priority Big Data Engines

49 views

Published on

Today's big data clusters based on the MapReduce paradigm are capable of executing analysis jobs with multiple priorities, providing differential latency guarantees. Traces from production systems show that the latency advantage of high-priority jobs comes at the cost of severe latency degradation of low-priority jobs as well as daunting resource waste caused by repetitive eviction and re-execution of low-priority jobs. We advocate a new resource management design that exploits the idea of differential approximation and sprinting. The unique combination of approximation and sprinting avoids the eviction of low-priority jobs and its consequent latency degradation and resource waste. To this end, we designed, implemented and evaluated DiAS, an extension of the Spark processing engine to support deflate jobs by dropping tasks and to sprint jobs. Our experiments on scenarios with two and three priority classes indicate that DiAS achieves up to 90% and 60% latency reduction for low- and high-priority jobs, respectively. DiAS not only eliminates resource waste but also (surprisingly) lowers energy consumption up to 30% at only a marginal accuracy loss for low-priority jobs.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Differential Approximation and Sprinting for Multi-Priority Big Data Engines

  1. 1. Differential Approximation and Sprinting for Multi-Priority Big Data Engines Robert Birke1, Isabelly Rocha2, Juan Perez3, Valerio Schiavoni2, Pascal Felber2, Lydia Y. Chen4 ABB Research, Switzerland1 University of Neuchâtel, Switzerland2 Universidad del Rosario, Colombia3 TU Delft, The Netherlands4 December 13th, 2019
  2. 2. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 2 Big Data Analytics FrameworksBig Data Analytics Applications Context
  3. 3. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 2 Big Data Analytics FrameworksBig Data Analytics Applications Different requirements: • Latency • Accuracy Context
  4. 4. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 2 Big Data Analytics FrameworksBig Data Analytics Applications Different requirements: • Latency • Accuracy Solution: • Priority scheduling Context
  5. 5. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive
  6. 6. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive
  7. 7. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive
  8. 8. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive
  9. 9. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive
  10. 10. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive
  11. 11. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive evict job!
  12. 12. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive evict job!
  13. 13. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive evict job!
  14. 14. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 3 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Preemptive evict job! waste
  15. 15. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 4 machine Dme distribuDon Success execution Failed execution Priority eviction 20 % 35 % 45 % Priority Scheduling: Preemptive • Consequences: • Repetitive eviction of low priority jobs • Significant latency degradation • High amount of resources wasted on subsequent evictions Source: Demystifying Casualties of Evictions in Big Data Priority Scheduling. ACM SIGMETRICS Performance Evaluation Review 42, 4 (2015), 12–21.
  16. 16. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 5 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Non-Preemptive
  17. 17. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 5 Scheduling Queue Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Non-Preemptive
  18. 18. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 5 Scheduling Queue accuracy loss Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Non-Preemptive
  19. 19. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 5 Scheduling Queue accuracy loss extra waiting Priority 2 Priority 1 Big Data Analytics Framework Priority Scheduling: Non-Preemptive
  20. 20. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 6 DiAS Deflator • Dispatching • Monitoring Dropper • Defining ratios Scheduler Sprinter • Frequency scaling Our Approach: DiAS • Spark as big data processing engine • Augmented Spark with the capability of drop tasks • Prototype implemented in Golang • Implemented a workload generator for evaluation purpose
  21. 21. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 7 Deflator DiAS Deflator Dropper Scheduler Sprinter • Decide how much data to process base on • Arrival time • Size of inputs • Current system load • Latency model • Given that certain inputs are dropped • Define how the entire latency distribution will change
  22. 22. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 8 • Distributions: • Phase-Type job processing times to model concurrent task times of jobs • States: • Matrix analytics methods, to track states • Number of jobs in queue/engine • Number of task queue/engine • The job distribution across priorities • Solver: • MMAP[K]/PH[K]/1 priority queue from Horvath [EJOR’15] Deflator DiAS Deflator Dropper Scheduler Sprinter
  23. 23. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 8 • Distributions: • Phase-Type job processing times to model concurrent task times of jobs • States: • Matrix analytics methods, to track states • Number of jobs in queue/engine • Number of task queue/engine • The job distribution across priorities • Solver: • MMAP[K]/PH[K]/1 priority queue from Horvath [EJOR’15] Deflator DiAS Deflator Dropper Scheduler Sprinter
  24. 24. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 9 0 10 20 30 40 50 60 70 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Meanabsolutepercenterror[%] Θm • Trade-off between accuracy and latency • Low dropping ratios -> low accuracy loss Deflator DiAS Deflator Dropper Scheduler Sprinter
  25. 25. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 9 0 10 20 30 40 50 60 70 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Meanabsolutepercenterror[%] Θm • Trade-off between accuracy and latency • Low dropping ratios -> low accuracy loss Deflator DiAS Deflator Dropper Scheduler Sprinter Dropping ratio
  26. 26. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 9 0 10 20 30 40 50 60 70 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Meanabsolutepercenterror[%] Θm • Trade-off between accuracy and latency • Low dropping ratios -> low accuracy loss Deflator DiAS Deflator Dropper Scheduler Sprinter Dropping ratio Dropping ratio: 50% MAE: 42%
  27. 27. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 10 0 0.2 0.4 0.6 0.8 Drop Ratio 0 100 200 300 400 MeanJobResponseTime model - high obs - high model - low obs - low • Validation of response time for 2-priority jobs Deflator DiAS Deflator Dropper Scheduler Sprinter
  28. 28. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 10 0 0.2 0.4 0.6 0.8 Drop Ratio 0 100 200 300 400 MeanJobResponseTime model - high obs - high model - low obs - low • Validation of response time for 2-priority jobs Deflator DiAS Deflator Dropper Scheduler Sprinter Dropping ratio: 20% Response time: 152 seconds
  29. 29. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 11 Deflator DiAS Deflator Dropper Scheduler Sprinter Original MR job
  30. 30. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 11 Deflator DiAS Deflator Dropper Scheduler Sprinter Original MR job Approximate MR job
  31. 31. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 12 Deflator Dropper Scheduler Sprinter DiAS • Implemented in Spark by modifying the function findMissingPartitions() • Modification: return only [n(1 − θk )] partitions out of n • Follows specifications of Dropper Deflator
  32. 32. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 13 Deflator Dropper Scheduler Sprinter DiAS Sprinter • Performs CPU frequency scaling • Handle sprinting timer for each job • Budget defined from different constraints • Thermal • Power • Provisioning
  33. 33. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 14 Our Approach: Overview DiAS Deflator Dropper Scheduler Sprinter
  34. 34. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Objetive • Low waste • Latency/accuracy requirements • Multi-priority 14 Our Approach: Overview DiAS Deflator Dropper Scheduler Sprinter
  35. 35. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Objetive • Low waste • Latency/accuracy requirements • Multi-priority 14 Our Approach: Overview • Challenges • Many parameters • Tail latency • Priority DiAS Deflator Dropper Scheduler Sprinter
  36. 36. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Objetive • Low waste • Latency/accuracy requirements • Multi-priority • Goal • Define: Task dropping ratio and sprinting timeout • Given: Priority class, tolerance to accuracy degradation, available sprinting budget 14 Our Approach: Overview • Challenges • Many parameters • Tail latency • Priority DiAS Deflator Dropper Scheduler Sprinter
  37. 37. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 15 • Spark processing engine • Version 2.1 • 10 workers cluster • 2 CPU cores and 4 GB memory per worker • Intel Xeon E3-1270 v6 CPU, 64 cores and 128 GB memory • Key parameters • Ratio between low- and high priority jobs • Average size • Cluster load • Workload • Text analysis jobs • Graph analysis jobs Evaluation Setup
  38. 38. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 16 • Differential Approximation • Two- and three priority system • Sensitivity analisys • Similar job size for all priorities • Several high- to low-priority job ratio • Several system load • Differential Approximation and Sprinting • Latency gain • Energy gain Evaluation
  39. 39. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Tail and average latency of low priority job decreases • Average latency of high priority job increases 17 1 10 100 1000 P NP DA 0/10 DA 0/20 -80 -60 -40 -20 0 20 40 60 80 Responsetime[s] Difference[%] High Low Evaluation: Differential Approximation
  40. 40. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Tail and average latency of low priority job decreases • Average latency of high priority job increases 17 1 10 100 1000 P NP DA 0/10 DA 0/20 -80 -60 -40 -20 0 20 40 60 80 Responsetime[s] Difference[%] High Low Evaluation: Differential Approximation Preemptive
  41. 41. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Tail and average latency of low priority job decreases • Average latency of high priority job increases 17 1 10 100 1000 P NP DA 0/10 DA 0/20 -80 -60 -40 -20 0 20 40 60 80 Responsetime[s] Difference[%] High Low Evaluation: Differential Approximation Preemptive Non-Preemptive
  42. 42. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Tail and average latency of low priority job decreases • Average latency of high priority job increases 17 1 10 100 1000 P NP DA 0/10 DA 0/20 -80 -60 -40 -20 0 20 40 60 80 Responsetime[s] Difference[%] High Low Evaluation: Differential Approximation Preemptive Non-Preemptive Differential Approximation x/y: drop ratio on high- (x) and low-priority (y)
  43. 43. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 18 1 10 100 1000 10000 P DiAS 0/10 DiAS 0/20 -80 -60 -40 -20 0 20 40 60 80 Responsetime[s] Difference[%] High Low • Tail and average latency of low priority job decreases even more (up to 90%) • Average latency of high priority also decreases Evaluation: Differential Approximation and Sprinting
  44. 44. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 • Reduction of more than 20% on energy consumption 19 1 10 100 1000 10000 P DiAS 0/10 DiAS 0/20 -80 -60 -40 -20 0 20 40 60 80 Energy[kJ] Difference[%] Unimited Limited Evaluation: Differential Approximation and Sprinting
  45. 45. Isabelly Rocha Differential Approximation and Sprinting for Multi-Priority Big Data Engines | Middleware 2019 20 • Main goal: reduce resource waste • Strategy: • Drop job eviction • Deflate low-priority jobs • Sprint high-priority jobs • Additional gains: • Up to 90% latency reduction • More than 20% energy savings • Implemented in Golang on top of Spark Summary and Takeaways

×