Scalable Applications with Scala

1,101 views
987 views

Published on

Published in: Technology, News & Politics
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,101
On SlideShare
0
From Embeds
0
Number of Embeds
357
Actions
Shares
0
Downloads
14
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Scalable Applications with Scala

  1. 1. Scalable Applications with Scala Nimrod Argov Tikal Knowledge
  2. 2. What is a Scalable Application?
  3. 3. What is a Scalable Application? Scalability is the ability of a system, network, or process to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth. -- wikipedia.org
  4. 4. What is a Scalable Application? Administrative - More People using the system, or more programmers writing it
  5. 5. What is a Scalable Application? Functional - Adding new functionality with minimal effort
  6. 6. What is a Scalable Application? Geographical - Maintaining performance, usefulness or usability regardless of locale specific user concentration
  7. 7. What is a Scalable Application? Load - The ability of a system to expand and contract to accommodate heavier or lighter loads
  8. 8. Concurrency / Parallelism Optimizing CPU Usage Efficiency
  9. 9. Scala Scala brings together the Functional and Object Oriented worlds, where until now both sides considered the other to be totally wrong, or the work of the devil. -- Martin Oderski
  10. 10. Fun!
  11. 11. Java java.util.concurrent Locking Mutable Shared State
  12. 12. Java When is it enough?
  13. 13. Java When is it enough? Too Little - Race Conditions Too Much - Loss of Parallelism / Deadlocks
  14. 14. Immutability Concurrency requires Immutability Classes should be immutable unless there's a very good reason to make them mutable....If a class cannot be made immutable, limit its mutability as much as possible. -- Effective Java (Joshua Bloch)
  15. 15. Immutability //java public final class Student private final int id; private final String name; public Student(int id, String name){ this.id = id; this.name = name; } .. getters… }
  16. 16. Immutability //java public final class Student private final int id; private final String name; public Student(int id, String name){ this.id = id; this.name = name; } .. getters… }
  17. 17. Immutability //java List<Student> students = new ArrayList<Student>(); List<Student> immutableStudents = Collections.unmodifiableList(students); List<Students> ohBother = new CopyOnWriteArrayList<Student>();
  18. 18. Immutability //java List<Student> students = new ArrayList<Student>(); List<Student> immutableStudents = Collections.unmodifiableList(students); students.add(new Student(15, “Harry Potter”)); immutableStudents now shows a different list!
  19. 19. Scala Collections ● Mutable/Immutable variants ● Immutable variant is imported by default ● Operations are Fast ● Cost of object creation is highly exaggerated by programmers
  20. 20. Scala Collections ● Set ● Map ● List ● Stream ● Vector ● Stack ● Queue ● Tree
  21. 21. Scala Collections Many Operations: ● ++ (add all) ● collect ● combinations ● contains ● count ● diff ● endsWith ● ● filter ● find ● flatMap ● flatten ● forEach ● intersect ● lift ● ● min ● max ● mkString ● partition ● permutations ● reduce ● reverse
  22. 22. Scala Parallel Collections Motivation “The hope was, and still is, that implicit parallelism behind a collections abstraction will bring reliable parallel execution one step closer to the workflow of mainstream developers.” --Scala Documentation
  23. 23. Scala Parallel Collections Example: Calculate function over list of numbers
  24. 24. Scala Parallel Collections Example: Calculate function over list of numbers Java: // Somehow cut the list to N parts
  25. 25. Scala Parallel Collections Example: Calculate function over list of numbers Java: // Somehow cut the list to N parts // Create ForkJoinTask (or Just a Runnable if pre JDK 6)
  26. 26. Scala Parallel Collections Example: Calculate function over list of numbers Java: // Somehow cut the list to N parts // Create ForkJoinTask (or Just a Runnable if pre JDK 6) // Use ForkJoinPool to run the tasks // Wait for all to finish (join)
  27. 27. Scala Parallel Collections public class Operation extends RecursiveAction{ private List<Integer> numbers; private List<Integer> results; private int start, end; public Operation(List<Integer> nums){ numbers = nums; start = 0; end = numbers.size(); } protected void doComputation(){ for(int i = start; i < end; i++) results.set(i, calculate(number)); } }
  28. 28. Scala Parallel Collections public class Operation extends RecursiveAction{ protected static int threshold = 1000; ... public Operation(List<Integer> nums){ … } protected void doComputation(){ … } protected void compute(){ if (numbers.length < threshold) doComputation(); else invokeAll(new SumOperation(numbers, 0, numbers.size() / 2), new SumOperation(numbers, numbers.size() / 2, numbers.size()); }
  29. 29. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_)
  30. 30. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95
  31. 31. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95
  32. 32. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 11 22 66 11 55 22 66 55 99 33 88 88 55 22 77 11
  33. 33. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 11 22 66 11 55 22 66 55 99 33 88 88 55 22 77 11 11 22 66 11 55 22 66 55 99 33 88 88 55 22 77 11
  34. 34. Scala Parallel Collections • ParArray • ParVector • mutable.ParHashMap • mutable.ParHashSet • immutable.ParHashMap • immutable.ParHashSet • ParRange
  35. 35. Scala Parallel Collections Each collection defines: ● Splitter ● Combiner Default # of threads: how many cores have you got? Underlying implementation depends on configuration
  36. 36. Scala Parallel Collections ForkJoinTaskSupport (JDK 1.6 or higher) ThreadPoolTaskSupport ExecutionContextTaskSupport
  37. 37. Scala Parallel Collections <<< WARNING… WARNING… >>> Side Effects or Non-Associative operations will create non-deterministic results!
  38. 38. Scala Parallel Collections var sum = 0; List(1 to 10000).par.forEach(sum += _) List(1 to 10000).par.reduce(_-_)
  39. 39. Scala Futures Futures provide a nice way to reason about performing many operations in parallel, in an efficient and non- blocking way. -- Scala Documentation
  40. 40. Scala Futures Future Promise
  41. 41. Scala Futures val p = Promise[T] val f: Future[T] = p.future
  42. 42. Java Futures val p = Promise[T] val f: Future[T] = p.future // Java-like variant: doSomeWork() f.get()
  43. 43. Efficient?
  44. 44. Efficient? Also, Java Futures are NOT Composable
  45. 45. What We Want
  46. 46. Callbacks Thread 1 val p = Promise[T] val f: Future[T] = p.future Thread 2 f onComplete { case Success(calcResult) = println(calcResult) case Failure(t) = println(“Error: “ + t.getMessage) }
  47. 47. Callbacks Thread 1 val p = Promise[T] val f: Future[T] = p.future Thread 2 f onComplete{ case Success(calcResult) = println(calcResult) case Failure(t) = println(“Error: “ + t.getMessage) }
  48. 48. Try Try[T] Success[T](value: T) Failure[T](t: throwable)
  49. 49. Try Try[T] Success[T](value: T) Failure[T](t: throwable) Not Asynchronous
  50. 50. Try val result: Try[int] = calcSomeValue result match{ case Success(n) = println(“The result is “ + n) case Failure(t) = println(“Error: “ + t.getMessage) }
  51. 51. Monadic Combinators val result: Int = calcSomeValue getOrElse -1 calcSomeValue.map(doSomethingWithVal).recover (handleException)
  52. 52. val rateQuote = future { service.getCurrentValue(USD) } val purchase = rateQuote map { quote => if (isProfitable(quote)) service.buy(amount, quote) else throw new Exception("not profitable") } purchase onSuccess { case _ => println("Purchased " + amount + " USD") } Future Composition
  53. 53. val usdQuote = future { service.getCurrentValue(USD) } val chfQuote = future { service.getCurrentValue(CHF) } val purchase = for { usd <- usdQuote chf <- chfQuote if isProfitable(usd, chf) } yield service.buy(amount, chf) purchase onSuccess { case _ => println("Purchased " + amount + " CHF") } Future Composition
  54. 54. def generate = future { (random, random) } def calculate(x:(Double, Double)) = future{ pow(x._1, x._2) } def filter(y:Any) = y.toString.contains("22222") while (true) { for { i <- generate j <- calculate(i) if filter(j) } println(j) } Pipe Example

×