Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Tools for Making Machine Learning more Reactive

18 views

Published on

Presented at the Reactive New York Meetup on 5/3/18.
Video here: https://livestream.com/accounts/14330061/events/8189196/videos/174325372

Published in: Software
  • Be the first to comment

  • Be the first to like this

Tools for Making Machine Learning more Reactive

  1. 1. Tools for Making Machine Learning more Reactive Jeff Smith @jeffksmithjr jeffsmith.tech
  2. 2. Reactive Machine Learning
  3. 3. Background Survey
  4. 4. Reactive Systems
  5. 5. Reactive Strategies
  6. 6. Machine Learning Systems
  7. 7. Reactive Machine Learning
  8. 8. Reactive at Intent Media
  9. 9. Language Survey
  10. 10. Languages ● Scala ● Erlang/Elixir ● Python
  11. 11. Responding
  12. 12. Agriculture
  13. 13. Akka ● Scala & Java ● Actor model ● Concurrency ● Distribution ● Supervision
  14. 14. Model Microservices
  15. 15. Model Microservice in Akka HTTP case class Prediction(id: Long, timestamp: Long, value: Double) trait Protocols extends DefaultJsonProtocol { implicit val predictionFormat = jsonFormat3(Prediction) }
  16. 16. Model Microservice in Akka HTTP def model(features: Map[Char, Double]) = { val coefficients = ('a' to 'z').zip(1 to 26).toMap val predictionValue = features.map { case (identifier, value) => coefficients.getOrElse(identifier, 0) * value }.sum / features.size Prediction(Random.nextLong(), System.currentTimeMillis(), predictionValue) } def parseFeatures(features: String): Map[String, Double] = { features.parseJson.convertTo[Map[Char, Double]] } def predict(features: String): Prediction = { model(parseFeatures(features)) }
  17. 17. Model Microservice in Akka HTTP trait Service extends Protocols { implicit val system: ActorSystem implicit def executor: ExecutionContextExecutor implicit val materializer: Materializer val logger: LoggingAdapter val routes = { logRequestResult("model-service") { pathPrefix("predict") { (get & path(Segment)) { features: String => complete { ToResponseMarshallable(predict(features))}}}}}}
  18. 18. Model Serialization
  19. 19. Random Forest in Spark MLlib
  20. 20. MLlib Model Serialization { "class": "org.apache.spark.ml.PipelineModel", "timestamp": 1467653845388, "sparkVersion": "2.0.0-preview", "uid": "pipeline_6b4fb08e8fb0", "paramMap": { "stageUids": ["quantileDiscretizer_d3996173db25", "vecAssembler_2bdcd79fe1cf", "logreg_9d559ca7e208"] } }
  21. 21. MLlib Model Serialization { "class": "org.apache.spark.ml.classification.LogisticRegressionModel", "timestamp": 1467653845650, "sparkVersion": "2.0.0-preview", "uid": "logreg_9d559ca7e208", "paramMap": { "threshold": 0.5, "elasticNetParam": 0.0, "fitIntercept": true, "tol": 1.0E-6, "regParam": 0.0, "maxIter": 5, "standardization": true, "featuresCol": "features", "rawPredictionCol": "rawPrediction", "predictionCol": "prediction", "probabilityCol": "probability", "labelCol": "label"}}
  22. 22. Parquet
  23. 23. Packaging
  24. 24. Containerizing with sbt-docker dockerfile in docker := { val jarFile: File = sbt.Keys.`package`.in(Compile, packageBin).value val classpath = (managedClasspath in Compile).value val mainClass = "com.reactivemachinelearning.PredictiveService" val jarTarget = s"/app/${jarFile.getName}" val classpathString = classpath.files.map("/app/" + _.getName) .mkString(":") + ":" + jarTarget new Dockerfile { from("java") add(classpath.files, "/app/") add(jarFile, jarTarget) entryPoint("java", "-cp", classpathString, mainClass) } }
  25. 25. Containerizing with sbt-docker FROM java ADD 0/spark-core_2.11-2.0.0-preview.jar 1/avro-mapred-1.7.7-hadoop2.jar ... ADD 166/chapter-7_2.11-1.0.jar /app/chapter-7_2.11-1.0.jar ENTRYPOINT ["java", "-cp", "/app/spark-core_2.11-2.0.0-preview.jar ... "com.reactivemachinelearning.PredictiveService"]
  26. 26. Model Servers
  27. 27. Transportation
  28. 28. Model Server in http4s object Models { val modelA = HttpService { case GET -> Root / "a" / inputData => val response = true Ok(s"Model A predicted $response.") } val modelB = HttpService { case GET -> Root / "b" / inputData => val response = false Ok(s"Model B predicted $response.") } }
  29. 29. Model Server in http4s object Client { val client = PooledHttp1Client() private def call(model: String, input: String) = { val target = Uri.fromString(s"http://localhost:8080/models/$model/$input").toOption.get client.expect[String](target) } def callA(input: String) = call("a", input) def callB(input: String) = call("b", input) }
  30. 30. Model Server in http4s def splitTraffic(data: String) = { data.hashCode % 10 match { case x if x < 4 => Client.callA(data) case _ => Client.callB(data) } }
  31. 31. Model Server in http4s object ModelServer extends ServerApp { val apiService = HttpService { case GET -> Root / "predict" / inputData => val response = splitTraffic(inputData).run Ok(response) } override def server(args: List[String]): Task[Server] = { BlazeBuilder .bindLocal(8080) .mountService(apiService, "/api") .mountService(Models.modelA, "/models") .mountService(Models.modelB, "/models") .start } }
  32. 32. Model Server in http4s import scala.util.Random val modelC = HttpService { case GET -> Root / "c" / inputData => { val workingOk = Random.nextBoolean() val response = true if (workingOk) { Ok(s"Model C predicted $response.") } else { BadRequest("Model C failed to predict.") } } }
  33. 33. Model Server in http4s private def call(model: String, input: String): Task[Response] = { val target = Uri.fromString( s"http://localhost:8080/models/$model/$input" ).toOption.get client(target) } def callC(input: String) = call("c", input)
  34. 34. Model Server in http4s def splitTraffic(data: String) = { data.hashCode % 10 match { case x if x < 4 => Client.callA(data) case x if x < 6 => Client.callB(data) case _ => Client.callC(data) } } val apiService = HttpService { case GET -> Root / "predict" / inputData => val response = splitTraffic(inputData).run response match { case r: Response if r.status == Ok => Response(Ok).withBody(r.bodyAsText) case r => Response(BadRequest).withBody(r.bodyAsText) } }
  35. 35. Model Server in http4s def splitTraffic(data: String) = { data.hashCode % 10 match { case x if x < 4 => Client.callA(data) case x if x < 6 => Client.callB(data) case _ => Client.callC(data) } } val apiService = HttpService { case GET -> Root / "predict" / inputData => val response = splitTraffic(inputData).run response match { case r: Response if r.status == Ok => Response(Ok).withBody(r.bodyAsText) case r => Response(BadRequest).withBody(r.bodyAsText) } }
  36. 36. Model Server in http4s private def call(model: String, input: String) = { val target = Uri.fromString(s"http://localhost:8080/models/$model/$input").toOption.get client(target).retry(Seq(1 second), {_ => true}) }
  37. 37. BEAM
  38. 38. Erlang
  39. 39. Elixir
  40. 40. Knowledge Maintenance
  41. 41. Circuit Breakers in Erlang/Elixir def update(user_data) do {user_id, _} = user_data case verify_fuse(user_id) do :ok -> write_to_db(user_data) |> parse_db_response(user_id) :sorry -> IO.puts "This user can't be updated now. ¯_(ツ)_/¯" :sorry end end
  42. 42. Circuit Breakers in Erlang/Elixir defp verify_fuse(user_id) do case :fuse.ask(user_id, :sync) do {:error, :not_found} -> IO.puts "Installing" install_fuse(user_id) :ok :blown -> :sorry _ -> :ok end end
  43. 43. Circuit Breakers in Erlang/Elixir defp install_fuse(user_id) do :fuse.install(user_id, {{:standard, 3, 60000}, {:reset, 900000}}) end
  44. 44. Circuit Breakers in Erlang/Elixir defp write_to_db({_user_id, _data}) do Enum.random([{:ok, ""}, {:error, "The DB dropped the ball. ಠ_ಠ"}]) end
  45. 45. Circuit Breakers in Erlang/Elixir defp parse_db_response({:ok, _}, _user_id) do :ok end defp parse_db_response({:error, message}, user_id) do :fuse.melt(user_id) IO.puts "Error encountered: #{message}.nMelting the fuse!" :sorry end
  46. 46. Circuit Breakers in Erlang/Elixir def update(user_data) do {user_id, _} = user_data case verify_fuse(user_id) do :ok -> write_to_db(user_data) |> parse_db_response(user_id) :sorry -> IO.puts "This user can't be updated now. ¯_(ツ)_/¯" :sorry end end
  47. 47. Circuit Breakers in Akka class DangerousActor extends Actor with ActorLogging { import context.dispatcher val breaker = new CircuitBreaker( context.system.scheduler, maxFailures = 5, callTimeout = 10.seconds, resetTimeout = 1.minute).onOpen(notifyMeOnOpen()) def notifyMeOnOpen(): Unit = log.warning("My CircuitBreaker is now open, and will not close for one minute") }
  48. 48. Circuit Breakers in Lagom def descriptor: Descriptor = { import Service._ named("hello").withCalls( namedCall("hi", this.sayHi), namedCall("hiAgain", this.hiAgain) .withCircuitBreaker(CircuitBreaker.identifiedBy("hello2")) ) }
  49. 49. Deep Learning
  50. 50. Python
  51. 51. Galápagos Nǎo
  52. 52. Diversity in Ecosystems
  53. 53. Evolution def evolve(nets, generations) do evolve(nets, generations, &decrement/1) end def evolve(nets, generations, count_function) when generations > 0 do Task.Supervisor.async(GN.TaskSupervisor, fn -> IO.puts("Generations remaining: #{generations}") learn_generation(nets) |> select() |> evolve(count_function.(generations), count_function) end) end def evolve(nets, _generations, _count_function) do nets end
  54. 54. Evolution def learn_generation(nets) do clean_nets = strip_empties(nets) tasks = Task.Supervisor.async_stream_nolink( GN.TaskSupervisor, clean_nets, &start_and_spawn(&1), timeout: GN.Parameters.get(__MODULE__, :timeout) ) generation = for {status, net} <- tasks, status == :ok, do: net IO.puts(inspect(generation)) generation end
  55. 55. Evolution def spawn_offspring(seed_layers, mutation_rate @mutation_rate) do duplicate(seed_layers, mutation_rate) |> remove(mutation_rate) |> Enum.map(&mutate(&1, mutation_rate)) end
  56. 56. Evolution def select(pid __MODULE__, nets) do cutoffs = cutoffs(nets) for net <- nets do complexity = length(net.layers) level = Enum.min( [Enum.find_index(cutoffs, &(&1 >= complexity)) + 1, complexity_levels()]) net_acc = net.test_acc elite_acc = Map.get(get(level), :test_acc) if is_nil(elite_acc) or net_acc > elite_acc do put(pid, level, net) end end
  57. 57. Evolution iex(1)> GN.Selection.get_all() %{ 1 => %GN.Network{ id: "0c2020ad-8944-4f2c-80bd-1d92c9d26535", layers: [ dense: [64, :softrelu], batch_norm: [], activation: [:relu], dropout: [0.5], dense: [63, :relu] ], test_acc: 0.8553 }, 2 => %GN.Network{ id: "58229333-a05d-4371-8f23-e8e55c37a2ec", layers: [ dense: [64, :relu],
  58. 58. Continual Learning iex(2)> GN.Example.infinite_example() %Task{ owner: #PID<0.171.0>, pid: #PID<0.213.0>, ref: #Reference<0.1968944036.911736833.180535> } Generations remaining: infinity
  59. 59. Interactive Evolution
  60. 60. Continual Learning iex(2)> GN.Example.infinite_example() %Task{ owner: #PID<0.171.0>, pid: #PID<0.213.0>, ref: #Reference<0.1968944036.911736833.180535> } Generations remaining: infinity
  61. 61. Interactive Evolution iex(3)> GN.Selection.get_all() |> Map.get(2) |> GN.Library.put() iex(4)> GN.Library.get("02b2a947-f888-4abf-b2a5-5df25668b0ee") |> GN.Selection.put_unevaluated()
  62. 62. Galápagos Nǎo Tech Stack ● Elixir ● Apache MXNet/Gluon ● Python ● Export/ErlPort ● Docker ● Microsoft Cognitive Toolkit* ● ONNX support* * coming soon
  63. 63. Model Serialization Revisited
  64. 64. ONNX ● Open interchange format ● Focused on inference ● Broad and growing support
  65. 65. ONNX Format message AttributeProto { enum AttributeType { UNDEFINED = 0; FLOAT = 1; INT = 2; STRING = 3; TENSOR = 4; GRAPH = 5; FLOATS = 6; INTS = 7; STRINGS = 8; TENSORS = 9; GRAPHS = 10; }
  66. 66. ONNXS iex(1)> {:ok, mnist_data} = File.read "./test/examples/mnist.onnx" {:ok, <<8, 3, 18, 4, 67, 78, 84, 75, 26, 3, 50, 46, 52, 40, 1, 58, 227, 206, 1, 10, 199, 80, 18, 12, 80, 97, 114, 97, 109, 101, 116, 101, 114, 49, 57, 51, 26, 12, 80, 97, 114, 97, 109, 101, 116, 101, 114, 49, ...>>}
  67. 67. ONNXS iex(2)> mnist_struct = Onnx.ModelProto.decode(mnist_data) %Onnx.ModelProto{ doc_string: nil, domain: nil, graph: %Onnx.GraphProto{ doc_string: nil, initializer: [], input: [ %Onnx.ValueInfoProto{ doc_string: nil, name: "Input3", type: %Onnx.TypeProto{ value: {:tensor_type, %Onnx.TypeProto.Tensor{ elem_type: 1, shape: %Onnx.TensorShapeProto ...
  68. 68. ONNXS iex(3)> mnist_updated = %{mnist_struct | model_version: 2}
  69. 69. Thoughts
  70. 70. Model publishing and serving are hard problems.
  71. 71. Not all reactive technologies look like Erlang.
  72. 72. Reactive patterns are useful across toolchains.
  73. 73. Hard problems require diverse sets of solutions.
  74. 74. Open standards foster rich ecosystems.
  75. 75. Wrapping Up
  76. 76. Reactive Machine Learning Use the code rmlsnymu for 40% off the book!
  77. 77. Thanks
  78. 78. Questions
  79. 79. Tools for Making Machine Learning more Reactive Jeff Smith @jeffksmithjr jeffsmith.tech

×