Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
@ELMANU 
BACK < 
& future: ACTORS 
AND > PIPES < 
using akka for large-scale data 
migration 
manuel BERNHART
@ELMANU 
AGENDA 
• { BACKGROUND STORY 
• } FUTURES > PIPES < ACTORS 
• | LESSONS LEARNED
@ELMANU 
who is speaking? 
• freelance software consultant 
based in Vienna 
• Vienna Scala User Group 
• web, web, web 
•...
@ELMANU 
[ { 
BACKGROUND 
STORY
@ELMANU 
talenthouse 
• www.talenthouse.com 
• based in Los Angeles 
• connecting brands and artists 
• 3+ million users
@ELMANU 
BACKGROUND STORY 
• old, slow (very slow) platform 
• re-implementation from scratch with Scala & Play 
• tight s...
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM 
DISCLAIMER: 
What follows is not intended as a 
bashing of the source system, but as a 
necessary ...
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
SOURCE SYSTEM
@ELMANU 
MIGRATION schedule 
•basically, one week-end 
•big-bang kind-of migration 
•if possible incremental migration bef...
@ELMANU 
[ } 
FUTURES > PIPES 
< ACTORS
@ELMANU 
FUTURES
@ELMANU 
FUTURES 
•scala.concurrent.Future[T] 
•holds a value of type T 
•can either fail or succeed
@ELMANU 
FUTURES: HAPPY 
PATH 
import scala.concurrent._ 
import scala.concurrent.ExecutionContext.Implicits.global 
val f...
@ELMANU 
FUTURES: SAD PATH 
import scala.concurrent._ 
import scala.concurrent.ExecutionContext.Implicits.global 
import s...
@ELMANU 
FUTURES: SAD PATH 
import scala.concurrent._ 
import scala.concurrent.ExecutionContext.Implicits.global 
import s...
@ELMANU 
FUTURES: SAD PATH 
import scala.concurrent._ 
import scala.concurrent.ExecutionContext.Implicits.global 
import s...
@ELMANU 
FUTURES: SAD PATH 
import scala.concurrent._ 
import scala.concurrent.ExecutionContext.Implicits.global 
import s...
@ELMANU 
FUTURES: SAD PATH 
•Exceptions are propagated up the chain 
•Without recover there is no guarantee that 
failure ...
@ELMANU 
COMPOSING FUTURES 
val futureA: Future[Int] = Future { 1 + 1 } 
val futureB: Future[Int] = Future { 2 + 2 } 
val ...
@ELMANU 
COMPOSING FUTURES 
val futureC: Future[Int] = for { 
a <- Future { 1 + 1 } 
b <- Future { 2 + 2 } 
} yield { 
a +...
@ELMANU 
COMPOSING FUTURES 
val futureC: Future[Int] = for { 
a <- Future { 1 + 1 } 
b <- Future { 2 + 2 } 
} yield { 
a +...
@ELMANU 
FUTURES: CALLBACKS 
import scala.concurrent._ 
import scala.concurrent.ExecutionContext.Implicits.global 
val fut...
@ELMANU 
using FUTURES 
•a Future { … } block that doesn’t do any I/O 
is code smell 
•use them in combination with the “r...
@ELMANU 
using FUTURES 
import scala.concurrent.blocking 
Future { 
blocking { 
DB.withConnection { implicit connection =>...
@ELMANU 
naming FUTURES
@ELMANU 
naming FUTURES 
“Say 
eventuallyMaybe 
one more time!”
@ELMANU 
ACTORS
@ELMANU 
ACTORS 
•lightweight objects 
•send and receive messages (mailbox) 
•can have children (supervision)
@ELMANU 
ACTORS 
Mailbox Mailbox 
akka://application/user/georgePeppard akka://application/user/audreyHepburn 
Mailbox 
ak...
@ELMANU 
ACTORS 
Mailbox Mailbox 
Holly, I'm in love with you. 
akka://application/user/georgePeppard akka://application/u...
@ELMANU 
ACTORS 
Mailbox Mailbox 
Holly, I'm in love with you. 
So what? 
akka://application/user/georgePeppard akka://app...
@ELMANU 
GETTING AN ACTOR 
import akka.actor._ 
class AudreyHepburn extends Actor { 
def receive = { ... } 
} 
val system:...
@ELMANU 
SENDING AND 
RECEIVING MESSAGES 
case class Script(text: String) 
class AudreyHepburn extends Actor { 
def receiv...
@ELMANU 
SENDING AND 
RECEIVING MESSAGES 
case class Script(text: String) 
class AudreyHepburn extends Actor { 
def receiv...
@ELMANU 
SENDING AND 
RECEIVING MESSAGES 
case class Script(text: String) 
class AudreyHepburn extends Actor { 
def receiv...
@ELMANU 
ASK PATTERN 
import akka.pattern.ask 
import scala.concurrent.ExecutionContext.Implicits.global 
import scala.con...
@ELMANU 
ASK PATTERN 
import akka.pattern.ask 
import scala.concurrent.ExecutionContext.Implicits.global 
import scala.con...
@ELMANU 
SUPERVISION 
class UserMigrator extends Actor { 
lazy val workers: ActorRef = context 
.actorOf[UserMigrationWork...
@ELMANU 
SUPERVISION 
class UserMigrator extends Actor { 
actor context 
lazy val workers: ActorRef = context 
.actorOf[Us...
@ELMANU 
SUPERVISION
@ELMANU 
SUPERVISION 
class UserMigrator extends Actor { 
lazy val workers: ActorRef = context 
.actorOf[UserMigrationWork...
@ELMANU 
PIPES
@ELMANU 
CECI EST UNE PIPE 
•Akka pattern to combine Futures and Actors 
•Sends the result of a Future to an Actor 
•Be ca...
@ELMANU 
CECI EST UNE PIPE 
class FileFetcher extends Actor { 
def receive = { 
case FetchFile(url) => 
val originalSender...
@ELMANU 
CECI EST UNE PIPE 
class FileFetcher extends Actor { 
def receive = { 
case FetchFile(url) => 
val originalSender...
@ELMANU 
CECI EST UNE PIPE 
class FileFetcher extends Actor { 
def receive = { 
case FetchFile(url) => 
val originalSender...
@ELMANU 
CECI EST UNE PIPE 
class FileFetcher extends Actor { 
def receive = { 
case FetchFile(url) => 
val originalSender...
@ELMANU 
CECI EST UNE PIPE 
class FileFetcher extends Actor { 
def receive = { 
case FetchFile(url) => 
val originalSender...
@ELMANU 
PIPES AND error 
handling 
class FileFetcher extends Actor { 
def receive = { 
case FetchFile(url) => 
val origin...
@ELMANU 
SUMMARY 
•Futures: manipulate and combine 
asynchronous operation results 
•Actors: organise complex asynchronous...
@ELMANU 
[ | 
LESSONS LEARNED
@ELMANU 
design according to 
YOUR DATA 
User migrator 
Worker Worker Worker Worker Worker
@ELMANU 
design according to 
YOUR DATA 
Item migrator 
User item 
migrator 
Item 
migration 
worker 
Item 
migration 
wor...
@ELMANU 
design according to 
YOUR DATA 
Item migrator 
User item 
migrator 
Item 
migration 
worker 
Item 
migration 
wor...
@ELMANU 
design according to 
YOUR DATA 
Item migrator 
Item 
migration 
worker 
User item 
migrator 
User item 
migrator ...
@ELMANU 
design according to 
YOUR DATA 
Item 
migration 
worker 
User item 
migrator 
User item 
migrator 
Item 
migratio...
@ELMANU 
KNOW THE limits OF 
THY SOURCE SYSTEM
@ELMANU 
KNOW THE limits OF 
THY SOURCE SYSTEM
@ELMANU 
DATA MIGRATION 
SHOULD not BE A RACE 
•Your goal is to get 
the data, not to be 
as fast as possible 
•Be gentle ...
@ELMANU 
CLOUD API 
STANDARDS 
•ISO-28601 Data formats in REST APIs 
•ISO-28700 Response times and failure 
communication ...
@ELMANU 
CLOUD API 
STANDARDS 
•ISO-28601 Data formats in REST APIs 
•ISO-28700 Response times and failure 
communication ...
@ELMANU 
NO STANDARDS! 
•The cloud is heterogenous 
•Response times, rate limits, error codes all 
different 
•Don’t even ...
@ELMANU 
RATE limits
@ELMANU 
RATE limits 
•Read the docs - most cloud API docs will 
warn you about them 
•Design your actor system so that yo...
@ELMANU 
RATE limits 
•Example: Soundcloud API 
•500 Internal Server Error after seemingly 
random amount of requests
@ELMANU 
RATE limits 
•Example: Soundcloud API 
•500 Internal Server Error after seemingly 
random amount of requests 
Mag...
@ELMANU 
BLOCKING
@ELMANU 
seriously, do not 
BLOCK 
•Seems innocent at first to block from time to 
time 
•OutOfMemory after 8 hours of mig...
@ELMANU 
MISC 
•Unstable primary IDs in source system 
•Build a lot of small tools, be pragmatic 
•sbt-tasks (http://yobri...
@ELMANU 
THE END
@ELMANU 
THE END 
QUESTIONS?
Back to the futures, actors and pipes: using Akka for large-scale data migration
Back to the futures, actors and pipes: using Akka for large-scale data migration
Back to the futures, actors and pipes: using Akka for large-scale data migration
Back to the futures, actors and pipes: using Akka for large-scale data migration
Upcoming SlideShare
Loading in …5
×

Back to the futures, actors and pipes: using Akka for large-scale data migration

12,102 views

Published on

Slides of my talk at Scala.io 2014 about large-scale data migration with Akka.

Published in: Technology

Back to the futures, actors and pipes: using Akka for large-scale data migration

  1. 1. @ELMANU BACK < & future: ACTORS AND > PIPES < using akka for large-scale data migration manuel BERNHART
  2. 2. @ELMANU AGENDA • { BACKGROUND STORY • } FUTURES > PIPES < ACTORS • | LESSONS LEARNED
  3. 3. @ELMANU who is speaking? • freelance software consultant based in Vienna • Vienna Scala User Group • web, web, web • writing a book on reactive web-applications
  4. 4. @ELMANU [ { BACKGROUND STORY
  5. 5. @ELMANU talenthouse • www.talenthouse.com • based in Los Angeles • connecting brands and artists • 3+ million users
  6. 6. @ELMANU BACKGROUND STORY • old, slow (very slow) platform • re-implementation from scratch with Scala & Play • tight schedule, a lot of data to migrate
  7. 7. @ELMANU SOURCE SYSTEM
  8. 8. @ELMANU SOURCE SYSTEM DISCLAIMER: What follows is not intended as a bashing of the source system, but as a necessary explanation of its complexity in relation to data migration.
  9. 9. @ELMANU SOURCE SYSTEM
  10. 10. @ELMANU SOURCE SYSTEM
  11. 11. @ELMANU SOURCE SYSTEM
  12. 12. @ELMANU SOURCE SYSTEM
  13. 13. @ELMANU SOURCE SYSTEM
  14. 14. @ELMANU SOURCE SYSTEM
  15. 15. @ELMANU SOURCE SYSTEM
  16. 16. @ELMANU SOURCE SYSTEM
  17. 17. @ELMANU MIGRATION schedule •basically, one week-end •big-bang kind-of migration •if possible incremental migration beforehand
  18. 18. @ELMANU [ } FUTURES > PIPES < ACTORS
  19. 19. @ELMANU FUTURES
  20. 20. @ELMANU FUTURES •scala.concurrent.Future[T] •holds a value of type T •can either fail or succeed
  21. 21. @ELMANU FUTURES: HAPPY PATH import scala.concurrent._ import scala.concurrent.ExecutionContext.Implicits.global val futureSum: Future[Int] = Future { 1 + 1 } futureSum.map { sum => println("The sum is " + sum) }
  22. 22. @ELMANU FUTURES: SAD PATH import scala.concurrent._ import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ val futureDiv: Future[Int] = Future { 1 / 0 } val futurePrint: Future[Unit] = futureDiv.map { div => println("The division result is " + div) } Await.result(futurePrint, 1 second)
  23. 23. @ELMANU FUTURES: SAD PATH import scala.concurrent._ import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ val futureDiv: Future[Int] = Future { 1 / 0 } val futurePrint: Future[Unit] = futureDiv.map { div => println("The division result is " + div) } Await.result(futurePrint, 1 second) Avoid blocking if possible
  24. 24. @ELMANU FUTURES: SAD PATH import scala.concurrent._ import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ scala> Await.result(futureDiv, 1.second) java.lang.ArithmeticException: / by zero at $anonfun$1.apply$mcI$sp(<console>:11) at $anonfun$1.apply(<console>:11) at $anonfun$1.apply(<console>:11) at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24) at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24) at scala.concurrent.impl.ExecutionContextImpl $AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121) at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) val futureDiv: Future[Int] = Future { 1 / 0 } futureDiv.map { div => println("The division result is " + div) } Await.result(futureDiv, 1 second)
  25. 25. @ELMANU FUTURES: SAD PATH import scala.concurrent._ import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ val futureDiv: Future[Int] = Future { 1 / 0 } val futurePrint: Future[Unit] = futureDiv.map { div => println("The division result is " + div) }.recover { case a: java.lang.ArithmeticException => println("What on earth are you trying to do?") } Await.result(futurePrint, 1 second) Be mindful of failure
  26. 26. @ELMANU FUTURES: SAD PATH •Exceptions are propagated up the chain •Without recover there is no guarantee that failure will ever get noticed!
  27. 27. @ELMANU COMPOSING FUTURES val futureA: Future[Int] = Future { 1 + 1 } val futureB: Future[Int] = Future { 2 + 2 } val futureC: Future[Int] = for { a <- futureA b <- futureB } yield { a + b }
  28. 28. @ELMANU COMPOSING FUTURES val futureC: Future[Int] = for { a <- Future { 1 + 1 } b <- Future { 2 + 2 } } yield { a + b }
  29. 29. @ELMANU COMPOSING FUTURES val futureC: Future[Int] = for { a <- Future { 1 + 1 } b <- Future { 2 + 2 } } yield { a + b } This runs in sequence Don’t do this
  30. 30. @ELMANU FUTURES: CALLBACKS import scala.concurrent._ import scala.concurrent.ExecutionContext.Implicits.global val futureDiv: Future[Int] = Future { 1 / 0 } futureDiv.onSuccess { case result => println("Result: " + result) } futureDiv.onFailure { case t: Throwable => println("Oh no!") }
  31. 31. @ELMANU using FUTURES •a Future { … } block that doesn’t do any I/O is code smell •use them in combination with the “right” ExecutionContext set-up •when you have blocking operations, wrap them into a blocking block
  32. 32. @ELMANU using FUTURES import scala.concurrent.blocking Future { blocking { DB.withConnection { implicit connection => val query = SQL("select * from bar") query() } } }
  33. 33. @ELMANU naming FUTURES
  34. 34. @ELMANU naming FUTURES “Say eventuallyMaybe one more time!”
  35. 35. @ELMANU ACTORS
  36. 36. @ELMANU ACTORS •lightweight objects •send and receive messages (mailbox) •can have children (supervision)
  37. 37. @ELMANU ACTORS Mailbox Mailbox akka://application/user/georgePeppard akka://application/user/audreyHepburn Mailbox akka://application/user/audreyHepburn/cat
  38. 38. @ELMANU ACTORS Mailbox Mailbox Holly, I'm in love with you. akka://application/user/georgePeppard akka://application/user/audreyHepburn akka://application/user/audreyHepburn/cat
  39. 39. @ELMANU ACTORS Mailbox Mailbox Holly, I'm in love with you. So what? akka://application/user/georgePeppard akka://application/user/audreyHepburn akka://application/user/audreyHepburn/cat
  40. 40. @ELMANU GETTING AN ACTOR import akka.actor._ class AudreyHepburn extends Actor { def receive = { ... } } val system: ActorSystem = ActorSystem() val audrey: ActorRef = system.actorOf(Props[AudreyHepburn])
  41. 41. @ELMANU SENDING AND RECEIVING MESSAGES case class Script(text: String) class AudreyHepburn extends Actor { def receive = { case Script(text) => read(text) } }
  42. 42. @ELMANU SENDING AND RECEIVING MESSAGES case class Script(text: String) class AudreyHepburn extends Actor { def receive = { case Script(text) => read(text) } } audrey ! Script(breakfastAtTiffany)
  43. 43. @ELMANU SENDING AND RECEIVING MESSAGES case class Script(text: String) class AudreyHepburn extends Actor { def receive = { case Script(text) => read(text) } } audrey ! Script(breakfastAtTiffany) “tell” - fire-forget
  44. 44. @ELMANU ASK PATTERN import akka.pattern.ask import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ implicit val timeout = akka.util.Timeout(1 second) val maybeAnswer: Future[String] = audrey ? "Where should we have breakfast?"
  45. 45. @ELMANU ASK PATTERN import akka.pattern.ask import scala.concurrent.ExecutionContext.Implicits.global import scala.concurrent.duration._ implicit val timeout = akka.util.Timeout(1 second) val maybeAnswer: Future[String] = audrey ? "Where should we have breakfast?" “ask”
  46. 46. @ELMANU SUPERVISION class UserMigrator extends Actor { lazy val workers: ActorRef = context .actorOf[UserMigrationWorker] .withRouter(RoundRobinRouter(nrOfInstances = 100)) }
  47. 47. @ELMANU SUPERVISION class UserMigrator extends Actor { actor context lazy val workers: ActorRef = context .actorOf[UserMigrationWorker] .withRouter(RoundRobinRouter(nrOfInstances = 100)) } router type many children
  48. 48. @ELMANU SUPERVISION
  49. 49. @ELMANU SUPERVISION class UserMigrator extends Actor { lazy val workers: ActorRef = context .actorOf[UserMigrationWorker] .withRouter(RoundRobinRouter(nrOfInstances = 100)) override def supervisorStrategy: SupervisorStrategy = OneForOneStrategy(maxNrOfRetries = 3) { case t: Throwable => log.error(“A child died!”, t) Restart } }
  50. 50. @ELMANU PIPES
  51. 51. @ELMANU CECI EST UNE PIPE •Akka pattern to combine Futures and Actors •Sends the result of a Future to an Actor •Be careful with error handling
  52. 52. @ELMANU CECI EST UNE PIPE class FileFetcher extends Actor { def receive = { case FetchFile(url) => val originalSender = sender() val download: Future[DownloadedFile] = WS.url(url).get().map { response => DownloadedFile( url, response.ahcResponse.getResponseBodyAsBytes ) } import akka.pattern.pipe download pipeTo originalSender } }
  53. 53. @ELMANU CECI EST UNE PIPE class FileFetcher extends Actor { def receive = { case FetchFile(url) => val originalSender = sender() val download: Future[DownloadedFile] = WS.url(url).get().map { response => DownloadedFile( url, response.ahcResponse.getResponseBodyAsBytes ) } import akka.pattern.pipe download pipeTo originalSender } } This is how you pipe
  54. 54. @ELMANU CECI EST UNE PIPE class FileFetcher extends Actor { def receive = { case FetchFile(url) => val originalSender = sender() val download: Future[DownloadedFile] = WS.url(url).get().map { response => DownloadedFile( url, response.ahcResponse.getResponseBodyAsBytes ) } import akka.pattern.pipe download pipeTo originalSender } } Keep reference to original sender - what follows is a Future!
  55. 55. @ELMANU CECI EST UNE PIPE class FileFetcher extends Actor { def receive = { case FetchFile(url) => val originalSender = sender() val download: Future[DownloadedFile] = WS.url(url).get().map { response => DownloadedFile( url, response.ahcResponse.getResponseBodyAsBytes ) } import akka.pattern.pipe download pipeTo originalSender } } Wrap your result into something you can easily match against
  56. 56. @ELMANU CECI EST UNE PIPE class FileFetcher extends Actor { def receive = { case FetchFile(url) => val originalSender = sender val download: Future[Array[Byte]] = WS.url(url).get().map { response => DownloadedFile( url, response.ahcResponse.getResponseBodyAsBytes ) } import akka.pattern.pipe download pipeTo originalSender } } Will this work?
  57. 57. @ELMANU PIPES AND error handling class FileFetcher extends Actor { def receive = { case FetchFile(url) => val originalSender = sender() val download = WS.url(url).get().map { response => DownloadedFile(...) } recover { case t: Throwable => DownloadFileFailure(url, t) } Don’t forget to recover! import akka.pattern.pipe download pipeTo originalSender } }
  58. 58. @ELMANU SUMMARY •Futures: manipulate and combine asynchronous operation results •Actors: organise complex asynchronous flows, deal with failure via supervision •Pipes: deal with results of asynchronous computation inside of actors
  59. 59. @ELMANU [ | LESSONS LEARNED
  60. 60. @ELMANU design according to YOUR DATA User migrator Worker Worker Worker Worker Worker
  61. 61. @ELMANU design according to YOUR DATA Item migrator User item migrator Item migration worker Item migration worker User item migrator Item migration worker Item migration worker User item migrator Item migration worker Item migration worker design A
  62. 62. @ELMANU design according to YOUR DATA Item migrator User item migrator Item migration worker Item migration worker User item migrator Item migration worker Item migration worker User item migrator Item migration worker Item migration worker design A Not all users have the same amount of items
  63. 63. @ELMANU design according to YOUR DATA Item migrator Item migration worker User item migrator User item migrator User item migrator Item migration worker Item migration worker Item migration worker Item migration worker Item migration worker File fetcher File fetcher File uploader File uploader Soundcloud worker design B
  64. 64. @ELMANU design according to YOUR DATA Item migration worker User item migrator User item migrator Item migration worker Item migration worker Item migration worker Item migration worker Item migration worker File fetcher File fetcher File uploader File uploader Soundcloud worker Pools of actors design B Item migrator User item migrator
  65. 65. @ELMANU KNOW THE limits OF THY SOURCE SYSTEM
  66. 66. @ELMANU KNOW THE limits OF THY SOURCE SYSTEM
  67. 67. @ELMANU DATA MIGRATION SHOULD not BE A RACE •Your goal is to get the data, not to be as fast as possible •Be gentle to the legacy system(s)
  68. 68. @ELMANU CLOUD API STANDARDS •ISO-28601 Data formats in REST APIs •ISO-28700 Response times and failure communication of REST APIs •ISO-28701 Rate limits in REST APIs and HTTP error codes
  69. 69. @ELMANU CLOUD API STANDARDS •ISO-28601 Data formats in REST APIs •ISO-28700 Response times and failure communication of REST APIs •ISO-28701 Rate limits in REST APIs and HTTP error codes DREAM ON
  70. 70. @ELMANU NO STANDARDS! •The cloud is heterogenous •Response times, rate limits, error codes all different •Don’t even try to treat all systems the same
  71. 71. @ELMANU RATE limits
  72. 72. @ELMANU RATE limits •Read the docs - most cloud API docs will warn you about them •Design your actor system so that you can queue if necessary •Keep track of migration status
  73. 73. @ELMANU RATE limits •Example: Soundcloud API •500 Internal Server Error after seemingly random amount of requests
  74. 74. @ELMANU RATE limits •Example: Soundcloud API •500 Internal Server Error after seemingly random amount of requests Magic User-Agent WS .url("http://api.soundcloud.com/resolve.json") .withHeaders("User-Agent" -> “FOOBAR”) // the magic ingredient that // opens the door to Soundcloud
  75. 75. @ELMANU BLOCKING
  76. 76. @ELMANU seriously, do not BLOCK •Seems innocent at first to block from time to time •OutOfMemory after 8 hours of migration run is not very funny •You will end up rewriting your whole code to be async anyway
  77. 77. @ELMANU MISC •Unstable primary IDs in source system •Build a lot of small tools, be pragmatic •sbt-tasks (http://yobriefca.se/sbt-tasks/)
  78. 78. @ELMANU THE END
  79. 79. @ELMANU THE END QUESTIONS?

×