Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)

1,646 views

Published on

Presentation at Spark Summit 2015

Published in: Data & Analytics
  • Be the first to comment

Flyby: Improved Dense Matrix Multiplication-(Tom Vacek, Thomson Reuters)

  1. 1. Flyby: Improved Dense Matrix Multiplication(& other tasks) Tom Vacek Thomson Reuters R&D
  2. 2. Outline •  Matching use case •  3 Algorithms •  Evaluation
  3. 3. Matching application users cartesian movies map…combineByKey
  4. 4. cartesian matrix multiply aRows cartesian(bCols) map (a , b) => a*b
  5. 5. shuffleMultiply •  Local reduceByKey •  Optimal efficiency: O(n1.5) workers each do O(n1.5) work O(n1.25) wire i,j (i,1),j (i,n),j j,k (1,k),j (m,k),j flatmap join (i,k),1 (i,k),n i,k map ._1 reduceByKey +
  6. 6. left flyby right
  7. 7. Flyby matrix multiply
  8. 8. Flyby matrix multiply
  9. 9. Flyby implementation (hack) def flyby[L,R,O] (left: RDD[L], right:RDD[R], fold0: (L,R)=>O, foldFun: (L,R,O)=>O))={ var reduction:RDD[O] = null val leftLocal = getPartitionsAsLocalArray(left) for(part <- leftLocal) { val flyer = right.sparkContext broadcast (part) val nextReduction = if (reduction == null) { right.map { case rr => val init = fold0(flyer.value.head, rr) flyer.value.tail.foldLeft(init) { case (acc, nextflyer) => foldFun(nextflyer, rr, acc)} } } else { right.zip(reduction).map { case (rr, red) => flyer.value.foldLeft(red) { case (acc, nextflyer) => foldFun(nextflyer, rr, acc)} } }.persist nextReduction.count flyer.unpersist (false) reduction.unpersist(false) reduction = nextReduction
  10. 10. Matrix Experiments •  Implemented dist dense matrix class using Breeze •  5 Nodes, quad socket Xeon E5-2690 (32 cores), 250GB. Cloudera 5.3 (Spark 1.2.0) •  Evaluate n×n matrices, npartitions •  Code: bitbucket.org/twvacek/spark-flyby
  11. 11. MLLib •  BlockMatrix introduced to mllib at version 1.3.0 (13 Mar 2015) •  Not available at time this work was done. •  Our matrix class very similar. •  Mllib multiply ~= shuffleMultiply
  12. 12. Results 0" 500" 1000" 1500" 2000" 2500" 3000" 3500" 4000" 10k (36)" 25k (100)" 30k (144)" time(s)" n (npartitions)" cart" shuf" flyby"
  13. 13. Discussion •  Flyby: signif improvement over cartesian. Gain for matching applications. •  Improvement over shuffle probably from memory caching. Asymptotics? •  FlybyRDD ? •  Code: bitbucket.org/twvacek/spark-flyby

×