Flink Forward Berlin 2017: Daniel Berecz, Gabor Hermann - Parameter Server on Flink, an approach for model-serving parallel machine learning

Flink Forward Berlin 2017: Daniel Berecz, Gabor Hermann - Parameter Server on Flink, an approach for model-serving parallel machine learning
Parameter Server on Flink an approach for model-parallel machine learning Gábor Hermann mail@gaborhermann.com Dániel Berecz berecz.daniel@sztaki.mta.hu This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement No 688191.
About us • Institute for Computer Science and Control, Hungarian Academy of Sciences (MTA SZTAKI) • Informatics Laboratory • „Big Data – Momemtum” research group • „Data Mining and Search” research group • Research group with strong industry ties • Ericsson, Rovio, Portugal Telekom, etc.
DISCLAIMER • Gábor Hermann works at Bol.com • Largest e-commerce platform in the Netherlands and Belgium
Agenda 1. Model-parallel training 2. Parameter Server on Flink Streaming 3. Batch on streaming?
Model-parallel training
Spam filtering Hey, how are you… Free money for you... One click, free sex... …
Spam filtering Hey, how are you… ü Free money for you... û One click, free sex... û …
Spam filtering Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û …
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û 0 1 0 1 1 … 0 0 0 𝑋 0
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û 0 1 0 1 1 free how … 0 0 0 𝑋 0
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û 0 1 0 1 1 +1 -1 free how +1 … 0 0 0 𝑋 𝑦 0
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 +1 … 0 0 0 𝑋 𝑦 𝑤 0 𝑋 $ 𝑤 ≈ 𝑦
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 01 +1 … 0 0 0 𝑤 0 𝑋 𝑦 𝑋 $ 𝑤 ≈ 𝑦
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ? One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 01 +1 … 0 0 0 -4 𝑤 0 𝑋 𝑦 𝑋 $ 𝑤 ≈ 𝑦
Spam filtering 0 0 0 Hey, how are you… ü Free money for you... û Hi, how was FF2017… ü One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 01 +1 … 0 0 0 -4 𝑤 0 𝑋 𝑦 𝑋 $ 𝑤 ≈ 𝑦
Spam filtering with Passive-Aggressive algorithm ? ? ?
Spam filtering with Passive-Aggressive algorithm 0 0 Free money for you... û 0 1 0 +1 free how 2 7 0 𝑤 0 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û 0 1 0 1 +1 -1 free how 2 7 0 0 𝑤 0 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û 0 1 0 1 +1 -1 free how 2 7 0 0 𝑤 0 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û 0 1 0 1 +1 -1 free how -4 7 0 0 𝑤 0 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û 0 1 0 1 +1 -1 free how -4 7 0 0 𝑤 0 Crammer et al.: Online Passive-Aggressive Algorithms (2006) 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 +1 … 0 0 0 𝑤 0 Crammer et al.: Online Passive-Aggressive Algorithms (2006) 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 +1 … 0 0 0 𝑤 0 Crammer et al.: Online Passive-Aggressive Algorithms (2006) laaarge 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 +1 … 0 0 0 𝑤 0 Crammer et al.: Online Passive-Aggressive Algorithms (2006) laaarge model- parallel 𝑋 𝑦
Spam filtering with Passive-Aggressive algorithm 0 0 0 Hey, how are you… ü Free money for you... û One click, free sex... û 0 1 0 1 1 +1 -1 free how -4 7 +1 … 0 0 0 𝑤 0 Crammer et al.: Online Passive-Aggressive Algorithms (2006) laaarge model- parallel 𝑋 𝑦
𝑅 Recommendation with matrix factorization 5 1 3 5 2 0 0 0 0 Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar min -∗,0∗ 1 𝑟34 − 𝜇 − 𝑏3 − 𝑏4 − 𝑢3 𝑖4 : (3,4)∈@A + +𝜆 1 ( 𝑢3 : 3∈@D + 𝑏3 : ) + 𝜆 1 ( 4∈@E 𝑖4 : + 𝑏4 : ) Dániel rated Iron Man with 5 stars
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor ? 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars Would Gábor like Interstellar?
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor ? 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars Would Gábor like Interstellar?
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor ? 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars Would Gábor like Interstellar? 5 4 -4 3 2 5
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor ? 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars Would Gábor like Interstellar? 5 4 -4 3 2 5 3
𝑅 Recommendation with matrix factorization 𝑈 𝑈 $ 𝐼 ≈ 𝑅 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 2 Level of action Level of drama X factor 3 0 0 0 0 Latent factors Dániel Gábor Iron Man Interstellar Dániel rated Iron Man with 5 stars Would Gábor like Interstellar? 5 4 -4 3 2 5 3
𝑅 Matrix factorization training 𝑈 item vector 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 user vector 5 3 0 0 0 0 Dániel Gábor Iron Man Interstellar 2
𝑅 Matrix factorization training 𝑈 item vector 3 2 6 5 3 2 5 -6 -2 5 4 -4 5 1 3 user vector 5 2 3 0 0 0 0 Dániel Gábor Iron Man Interstellar
𝑅 Matrix factorization training 𝑈 item vector 1 3 5 5 3 2 4 -5 -1 5 4 -4 5 1 3 user vector 5 2 3 0 0 0 0 Dániel Gábor Iron Man Interstellar
𝑅 Matrix factorization training 𝑈 item vector 1 3 5 5 3 2 4 -5 -1 5 4 -4 5 1 3 user vector 5 2 3 0 0 0 0 Dániel Gábor Iron Man Interstellar model-parallel
Parameter Server on Flink
Parameter Server workers server nodes data pull push
Passive-Aggressive with Parameter Server 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦
Passive-Aggressive with Parameter Server workers server nodes 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦
Passive-Aggressive with Parameter Server workers server nodes 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦
Passive-Aggressive with Parameter Server workers server nodes 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦 𝑤
Passive-Aggressive with Parameter Server workers server nodes pull 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦 𝑤
Passive-Aggressive with Parameter Server workers server nodes pull answer 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦 𝑤
Passive-Aggressive with Parameter Server workers server nodes pull answer 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦 𝑤
Passive-Aggressive with Parameter Server workers server nodes push 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦 𝑤
Passive-Aggressive with Parameter Server workers server nodes push 0 0 0 1 0 1 1 +1 -1 -4 7 +10 0 0 𝑤 𝑋 𝑦 𝑋, 𝑦 𝑤
Parameter Server API workers server nodes data pull push
Parameter Server API ps.pull(paramId) ps.push(paramId, paramUpdate) workers server nodes data pull push
Parameter Server API workers server nodes data pull push ps.pull(paramId) ps.push(paramId, paramUpdate) def onRecv(data): Unit
Parameter Server API workers server nodes data pull push ps.pull(paramId) ps.push(paramId, paramUpdate) def onRecv(data): Unit def onPullRecv(paramId, paramValue)
Passive-Aggressive code def onRecv(x: LabeledVector) = { def onPullRecv(paramId: Int, param: Double) = {
Passive-Aggressive code def onRecv(x: LabeledVector) = { cacheLocally(x) def onPullRecv(paramId: Int, param: Double) = {
Passive-Aggressive code def onRecv(x: LabeledVector) = { cacheLocally(x) x.indices.foreach { i => ps.pull(i) } } def onPullRecv(paramId: Int, param: Double) = {
Passive-Aggressive code def onRecv(x: LabeledVector) = { cacheLocally(x) x.indices.foreach { i => ps.pull(i) } } def onPullRecv(paramId: Int, param: Double) = { cacheOrCompute(paramId, param, { (x, w) =>
Passive-Aggressive code def onRecv(x: LabeledVector) = { cacheLocally(x) x.indices.foreach { i => ps.pull(i) } } def onPullRecv(paramId: Int, param: Double) = { cacheOrCompute(paramId, param, { (x, w) => val deltaW = passiveAggressiveUpdate(x, w)
Passive-Aggressive code def onRecv(x: LabeledVector) = { cacheLocally(x) x.indices.foreach { i => ps.pull(i) } } def onPullRecv(paramId: Int, param: Double) = { cacheOrCompute(paramId, param, { (x, w) => val deltaW = passiveAggressiveUpdate(x, w) deltaW.foreach { (pId, pDelta) => ps.push(pId, pDelta) } }) }
Matrix factorization with Parameter Server 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0
Matrix factorization with Parameter Server workers server nodes 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0 𝑅
Matrix factorization with Parameter Server workers server nodes 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0 𝐼 𝑈 𝑅
Matrix factorization with Parameter Server workers server nodes 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0 2 𝐼 𝑈 𝑅
Matrix factorization with Parameter Server workers server nodes pull 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0 2 𝐼 𝑈 𝑅
Matrix factorization with Parameter Server workers server nodes pull answer 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0 2 𝐼 𝑈 𝑅
Matrix factorization with Parameter Server workers server nodes push 𝑅𝑈 3 2 5 5 3 2 5 -6 -1 5 4 -4 5 1 3 5 3 0 0 0 0 2 𝐼 𝑈 𝑅
Matrix Factorization code (SGD) def onRecv(r: Rating) = { def onPullRecv(paramId: Int, param: Vector) = {
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) def onPullRecv(paramId: Int, param: Vector) = {
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = {
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = { val itemId = paramId val item = param
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = { val itemId = paramId val item = param val (r, userId, _) = waitQueues(itemId).pop()
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = { val itemId = paramId val item = param val (r, userId, _) = waitQueues(itemId).pop() val user = users(userId)
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = { val itemId = paramId val item = param val (r, userId, _) = waitQueues(itemId).pop() val user = users(userId) val (userDelta, itemDelta) = updateWithSGD(user, item, r)
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = { val itemId = paramId val item = param val (r, userId, _) = waitQueues(itemId).pop() val user = users(userId) val (userDelta, itemDelta) = updateWithSGD(user, item, r) ps.push(itemId, itemDelta)
Matrix Factorization code (SGD) def onRecv(r: Rating) = { waitQueues(r.itemId).add(r) ps.pull(r.itemId) } def onPullRecv(paramId: Int, param: Vector) = { val itemId = paramId val item = param val (r, userId, _) = waitQueues(itemId).pop() val user = users(userId) val (userDelta, itemDelta) = updateWithSGD(user, item, r) ps.push(itemId, itemDelta) users(userId) += userDelta }
Integration with Flink Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Integration with Flink val input: DataStream[Data] = ... val output: DataStream[Either[PSOutput, WorkerOutput]] = psTransform(input, onRecv, onPullRecv)
Implementation? Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Implementation? Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server ?
Implementation: Loops API Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Implementation: Loops API Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE
Implementation: Loops API Parameter Server API Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE
Implementation: Loops API Parameter Server API Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE busy
Implementation: Loops API Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE busy
Implementation: Loops API Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE busy busy
Implementation: Loops API Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE busy busy deadlock
Implementation: Loops API Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server NOT MATURE WIP: FLIP-15 FLIP-16
Batch on streaming?
Batch vs streaming?
Batch vs streaming? Offline vs online?
Online on streaming Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Online on streaming Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Offline on streaming? Parameter Server API Pull, Push Answer pull DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Offline on streaming? Parameter Server API DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Offline on streaming? Parameter Server API DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Offline on streaming? Parameter Server API Push same param? DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server
Offline on streaming? Parameter Server API Push same param? DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server Write conflict!
Offline on streaming? Parameter Server API Push same param? DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server Write conflict! Synchronization?
Offline on streaming? Parameter Server API Push same param? DataSource Transformations DataSink DataStream of input data DataStream of Worker and PS output Worker Parameter Server Write conflict! Asynchronous training!
Flink Batch vs Streaming for Offline
Async matrix factorization
Async matrix factorization
Combine offline+online!
Thank you for your attention Gábor Hermann mail@gaborhermann.com Dániel Berecz berecz.daniel@sztaki.mta.hu Source code: https://github.com/gaborhermann/flink-parameter-server https://github.com/rpalovics/Alpenglow Li et al.: "Scaling Distributed Machine Learning with the Parameter Server" 2014. Schelter et al.: "Factorbird - A Parameter Server Approach to Distributed Matrix Factorization." 2014. Gemulla et al.: "Large-scale matrix factorization with distributed stochastic gradient descent” 2011. Crammer et al.: ”Online Passive-Aggressive Algorithms” 2006.
Measurements
Flink Batch vs Streaming for Offline • Movielens 1M movie rating dataset • 6000 user, 4000 movies
Flink vs Alpenglow • Based on public LastFM crawl • Filtered to avoid spammers • Lasts 1 year • Around 45.000 users and 49.000 items

Check these out next

Flink Forward Berlin 2017: Daniel Berecz, Gabor Hermann - Parameter Server on Flink, an approach for model-serving parallel machine learning

Recommended

More Related Content

Similar to Flink Forward Berlin 2017: Daniel Berecz, Gabor Hermann - Parameter Server on Flink, an approach for model-serving parallel machine learning(9)

More from Flink Forward(20)

Recently uploaded(20)

Flink Forward Berlin 2017: Daniel Berecz, Gabor Hermann - Parameter Server on Flink, an approach for model-serving parallel machine learning