Downloaded 68 times
![SESSION CREATION
AND EXECUTION
% curl -XPOST localhost:8998/sessions
-d '{"kind": "spark"}'
{
"id": 0,
"kind": "spark",
"log": [...],
"state": "idle"
}
% curl -XPOST localhost:8998/sessions/0/statements -d '{"code": "1+1"}'
{
"id": 0,
"output": {
"data": { "text/plain": "res0: Int = 2" },
"execution_count": 0,
"status": "ok"
},
"state": "available"
}](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-25-320.jpg)
![LIVY INTERPRETERS
trait Interpreter {
def state: State
def execute(code: String): Future[JValue]
def close(): Unit
}
sealed trait State
case class NotStarted() extends State
case class Starting() extends State
case class Idle() extends State
case class Running() extends State
case class Busy() extends State
case class Error() extends State
case class ShuttingDown() extends State
case class Dead() extends State](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-30-320.jpg)
![LIVY INTERPRETERS
trait Interpreter {
def state: State
def execute(code: String): Future[JValue]
def close(): Unit
}
sealed trait State
case class NotStarted() extends State
case class Starting() extends State
case class Idle() extends State
case class Running() extends State
case class Busy() extends State
case class Error() extends State
case class ShuttingDown() extends State
case class Dead() extends State](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-31-320.jpg)
![INTERPRETER MAGIC
private def executeJsonMagic(name: String): ExecuteResponse = {
sparkIMain.valueOfTerm(name) match {
case Some(value: RDD[_]) => ExecuteMagic(Extraction.decompose(Map(
"application/json" -> value.asInstanceOf[RDD[_]].take(10))))
case Some(value) => ExecuteMagic(Extraction.decompose(Map(
"application/json" -> value)))
case None => ExecuteError(f"Value $name does not exist")
}
}
case Some(value: RDD[_]) => ExecuteMagic(Extraction.decompose(Map(
"application/json" -> value.asInstanceOf[RDD[_]].take(10))))
case Some(value) => ExecuteMagic(Extraction.decompose(Map(
"application/json" -> value)))](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-36-320.jpg)
![TABLE MAGIC
"application/vnd.livy.table.v1+json": {
"headers": [
{ "name": "count", "type": "BIGINT_TYPE" },
{ "name": "name", "type": "STRING_TYPE" }
],
"data": [
[ 23407, "the" ],
[ 19540, "I" ],
[ 18358, "and" ],
...
]
}
val lines = sc.textFile("shakespeare.txt");
val counts = lines.
flatMap(line => line.split(" ")).
map(word => (word, 1)).
reduceByKey(_ + _).
sortBy(-_._2).
map { case (w, c) =>
Map("word" -> w, "count" -> c)
}
%table counts%table counts](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-37-320.jpg)
![TABLE MAGIC
"application/vnd.livy.table.v1+json": {
"headers": [
{ "name": "count", "type": "BIGINT_TYPE" },
{ "name": "name", "type": "STRING_TYPE" }
],
"data": [
[ 23407, "the" ],
[ 19540, "I" ],
[ 18358, "and" ],
...
]
}
val lines = sc.textFile("shakespeare.txt");
val counts = lines.
flatMap(line => line.split(" ")).
map(word => (word, 1)).
reduceByKey(_ + _).
sortBy(-_._2).
map { case (w, c) =>
Map("word" -> w, "count" -> c)
}
%table counts](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-38-320.jpg)
![JSON MAGIC
val lines = sc.textFile("shakespeare.txt");
val counts = lines.
flatMap(line => line.split(" ")).
map(word => (word, 1)).
reduceByKey(_ + _).
sortBy(-_._2).
map { case (w, c) =>
Map("word" -> w, "count" -> c)
}
%json counts
{
"id": 0,
"output": {
"application/json": [
{ "count": 506610, "word": "" },
{ "count": 23407, "word": "the" },
{ "count": 19540, "word": "I" },
...
]
...
}
%json counts](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-39-320.jpg)
![JSON MAGIC
val lines = sc.textFile("shakespeare.txt");
val counts = lines.
flatMap(line => line.split(" ")).
map(word => (word, 1)).
reduceByKey(_ + _).
sortBy(-_._2).
map { case (w, c) =>
Map("word" -> w, "count" -> c)
}
%json counts
{
"id": 0,
"output": {
"application/json": [
{ "count": 506610, "word": "" },
{ "count": 23407, "word": "the" },
{ "count": 19540, "word": "I" },
...
]
...
}](https://image.slidesharecdn.com/20150818scalabythebay-150819153622-lva1-app6891/85/Big-Data-Scala-by-the-Bay-Interactive-Spark-in-your-Browser-40-320.jpg)
Uploaded bygethue
Big Data Scala by the Bay: Interactive Spark in your Browser
The document outlines the development and architecture of the Hue web interface for analyzing data through Apache Hadoop, emphasizing its integration with Apache Spark. It discusses the evolution of the platform, highlighting features such as Spark streaming, machine learning, and interactive notebooks, along with the use of RESTful web services for session management. The document also details Livy, a server that facilitates Spark sessions and supports multiple programming languages, enhancing user interactions with the Hadoop ecosystem.
More Related Content
Similar to Big Data Scala by the Bay: Interactive Spark in your Browser
Big Data Scala by the Bay: Interactive Spark in your Browser
- 1. INTERACTIVE SPARK IN YOUR BROWSER RomainRigaux romain@cloudera.com Erick Tryzelaar erickt@cloudera.com
- 2. GOAL OF HUE WEB INTERFACEFOR ANALYZING DATA WITH APACHE HADOOP SIMPLIFY AND INTEGRATE FREE AND OPEN SOURCE —> WEB “EXCEL” FOR HADOOP
- 3. VIEW FROM 30K FEET HadoopWeb Server You, your colleagues and even that friend that uses IE9 ;)
- 4. WHY SPARK? SIMPLER (PYTHON,STREAMING, INTERACTIVE…) OPENS UP DATA TO SCIENCE SPARK —> MR Apache Spark Spark Streaming MLlib (machine learning) GraphX (graph) Spark SQL
- 7. WHY IN HUE? MARRIED WITHFULL HADOOP ECOSYSTEM (Hive Tables, HDFS, Job Browser…)
- 8. WHY IN HUE? Multi user,YARN, Impersonation/Security Not yet-another-app-to-install ...
- 9. • It works HISTORY V1:OOZIE THE GOOD • Submit through Oozie • Slow THE BAD
- 10. • It worksbetter HISTORY V2: SPARK IGNITER THE GOOD • Compiler Jar • Batch THE BAD
- 11. • It workseven better • Scala / Python / R shells • Jar / Py batches • Notebook UI • YARN HISTORY V3: NOTEBOOK THE GOOD • Still new THE BAD
- 12.
- 13.
- 14. Notebook with snippets WEB ARCHITECTURE Server Spark Scala CommonAPI Pig Hive Livy … HS2 Scala Hive Specific APIs AJAX create_session() execute() … REST Thrift OpenSession() ExecuteStatement() /session /sessions/{sessionId}/statements
- 15.
- 16. • REST Webserver in Scala • Interactive Spark Sessions and Batch Jobs • Type Introspection for Visualization • Running sessions in YARN local • Backends: Scala, Python, R • Open Source: https://github.com/cloudera/hue/tree/master/app s/spark/java LIVY SPARK SERVER
- 17. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker Livy Server Scalatra Session Manager Session
- 18. LIVY WEB SERVER ARCHITECTURE LivyServer YARN Master Scalatra Spark Client Session Manager Session YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1
- 19. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1 2 Livy Server Scalatra Session Manager Session
- 20. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1 2 3 Livy Server Scalatra Session Manager Session
- 21. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1 2 3 4 Livy Server Scalatra Session Manager Session
- 22. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1 2 3 4 5 Livy Server Scalatra Session Manager Session
- 23. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1 2 3 4 5 6 Livy Server Scalatra Session Manager Session
- 24. LIVY WEB SERVER ARCHITECTURE YARN Master SparkClient YARN Node Spark Interpreter Spark Context YARN Node Spark Worker YARN Node Spark Worker 1 7 2 3 4 5 6 Livy Server Scalatra Session Manager Session
- 25. SESSION CREATION AND EXECUTION %curl -XPOST localhost:8998/sessions -d '{"kind": "spark"}' { "id": 0, "kind": "spark", "log": [...], "state": "idle" } % curl -XPOST localhost:8998/sessions/0/statements -d '{"code": "1+1"}' { "id": 0, "output": { "data": { "text/plain": "res0: Int = 2" }, "execution_count": 0, "status": "ok" }, "state": "available" }
- 26.
- 27. INTERPRETERS • Pipe stdin/stdoutto a running shell • Execute the code / send to Spark workers • Perform magic operations • One interpreter by language • “Swappable” with other kernels (python, spark..) Interpreter > println(1 + 1) 2 println(1 + 1) 2
- 28. INTERPRETER FLOW CURL Hue Livy ServerLivy Session Interpreter 1+1 2 { “data”: { “application/json”: “2” } } 1+1 2
- 29. INTERPRETER FLOW CHART Receivelines Split lines Send output to server Success Incomplete Merge with next line Error Execute LineMagic! Lines left? Magic line? No Yes NoYes
- 30. LIVY INTERPRETERS trait Interpreter{ def state: State def execute(code: String): Future[JValue] def close(): Unit } sealed trait State case class NotStarted() extends State case class Starting() extends State case class Idle() extends State case class Running() extends State case class Busy() extends State case class Error() extends State case class ShuttingDown() extends State case class Dead() extends State
- 31. LIVY INTERPRETERS trait Interpreter{ def state: State def execute(code: String): Future[JValue] def close(): Unit } sealed trait State case class NotStarted() extends State case class Starting() extends State case class Idle() extends State case class Running() extends State case class Busy() extends State case class Error() extends State case class ShuttingDown() extends State case class Dead() extends State
- 32. SPARK INTERPRETER class SparkInterpeterextends Interpreter { … private var _state: State = NotStarted() private val outputStream = new ByteArrayOutputStream() private var sparkIMain: SparkIMain = _ def start() = { ... _state = Starting() sparkIMain = new SparkIMain(new Settings(), new JPrintWriter(outputStream, true)) sparkIMain.initializeSynchronous() ... Interpreter new SparkIMain(new Settings(), new JPrintWriter(outputStream, true))
- 33. SPARK INTERPRETER private varsparkContext: SparkContext = _ def start() = { ... val sparkConf = new SparkConf(true) sparkContext = new SparkContext(sparkConf) sparkIMain.beQuietDuring { sparkIMain.bind("sc", "org.apache.spark.SparkContext", sparkContext, List("""@transient""")) } _state = Idle() } sparkIMain.bind("sc", "org.apache.spark.SparkContext", sparkContext, List("""@transient"""))
- 34. EXECUTING SPARK private defexecuteLine(code: String): ExecuteResult = { code match { case MAGIC_REGEX(magic, rest) => executeMagic(magic, rest) case _ => scala.Console.withOut(outputStream) { sparkIMain.interpret(code) match { case Results.Success => ExecuteComplete(readStdout()) case Results.Incomplete => ExecuteIncomplete(readStdout()) case Results.Error => ExecuteError(readStdout()) } ... case MAGIC_REGEX(magic, rest) => case _ =>
- 35. INTERPRETER MAGIC private valMAGIC_REGEX = "^%(w+)W*(.*)".r private def executeMagic(magic: String, rest: String): ExecuteResponse = { magic match { case "json" => executeJsonMagic(rest) case "table" => executeTableMagic(rest) case _ => ExecuteError(f"Unknown magic command $magic") } } case "json" => executeJsonMagic(rest) case "table" => executeTableMagic(rest) case _ => ExecuteError(f"Unknown magic command $magic")
- 36. INTERPRETER MAGIC private defexecuteJsonMagic(name: String): ExecuteResponse = { sparkIMain.valueOfTerm(name) match { case Some(value: RDD[_]) => ExecuteMagic(Extraction.decompose(Map( "application/json" -> value.asInstanceOf[RDD[_]].take(10)))) case Some(value) => ExecuteMagic(Extraction.decompose(Map( "application/json" -> value))) case None => ExecuteError(f"Value $name does not exist") } } case Some(value: RDD[_]) => ExecuteMagic(Extraction.decompose(Map( "application/json" -> value.asInstanceOf[RDD[_]].take(10)))) case Some(value) => ExecuteMagic(Extraction.decompose(Map( "application/json" -> value)))
- 37. TABLE MAGIC "application/vnd.livy.table.v1+json": { "headers":[ { "name": "count", "type": "BIGINT_TYPE" }, { "name": "name", "type": "STRING_TYPE" } ], "data": [ [ 23407, "the" ], [ 19540, "I" ], [ 18358, "and" ], ... ] } val lines = sc.textFile("shakespeare.txt"); val counts = lines. flatMap(line => line.split(" ")). map(word => (word, 1)). reduceByKey(_ + _). sortBy(-_._2). map { case (w, c) => Map("word" -> w, "count" -> c) } %table counts%table counts
- 38. TABLE MAGIC "application/vnd.livy.table.v1+json": { "headers":[ { "name": "count", "type": "BIGINT_TYPE" }, { "name": "name", "type": "STRING_TYPE" } ], "data": [ [ 23407, "the" ], [ 19540, "I" ], [ 18358, "and" ], ... ] } val lines = sc.textFile("shakespeare.txt"); val counts = lines. flatMap(line => line.split(" ")). map(word => (word, 1)). reduceByKey(_ + _). sortBy(-_._2). map { case (w, c) => Map("word" -> w, "count" -> c) } %table counts
- 39. JSON MAGIC val lines= sc.textFile("shakespeare.txt"); val counts = lines. flatMap(line => line.split(" ")). map(word => (word, 1)). reduceByKey(_ + _). sortBy(-_._2). map { case (w, c) => Map("word" -> w, "count" -> c) } %json counts { "id": 0, "output": { "application/json": [ { "count": 506610, "word": "" }, { "count": 23407, "word": "the" }, { "count": 19540, "word": "I" }, ... ] ... } %json counts
- 40. JSON MAGIC val lines= sc.textFile("shakespeare.txt"); val counts = lines. flatMap(line => line.split(" ")). map(word => (word, 1)). reduceByKey(_ + _). sortBy(-_._2). map { case (w, c) => Map("word" -> w, "count" -> c) } %json counts { "id": 0, "output": { "application/json": [ { "count": 506610, "word": "" }, { "count": 23407, "word": "the" }, { "count": 19540, "word": "I" }, ... ] ... }
- 41. • Stability andScaling • Security • iPython/Jupyter backends and file format COMING SOON
- 42.
- 43.
Editor's Notes
- #6 Why do we want to do this? Currently it’s difficult to visualize results from Spark. Spark has a great interactive tool called “spark-shell” that allows you to interact with large datasets on the commandline. For example, here is a session where we are counting the words used by shakespeare. Running this computation is easy, but spark-shell doesn’t provide any tools for visualizing the results.
- #7 One option is to save the output to a file, then use a tool like Hue to import it into a Hive table and visualize it. We are obviously big fans of Hue, but there are still too many steps to go through to get to this point. If we want to change the script, say to filter out words like “the” and “and”, we need to go back to the shell, rerun our code snippet, save it to a file, then reimport it into the UI. It’s a slow process.
- #15 Multi languages Inherit Hue’s sharing, export/import
- #16 Hello, I’m Erick Tryzelaar, and I’m going to talk about the Livy Spark Server, which is our backend for Hue’s Notebook application.
- #17 Livy is a REST web server that allows a tool like Hue to interactively execute scala and spark commands, just like spark-shell. It goes beyond it by adding type introspection, which allows a frontend like Hue to render results in interactive visualizations. Furthermore it allows sessions to be run inside YARN to support horizontally scaling out to hundreds of active sessions. It also supports a Python and R backend. Finally, it’s fully open source, and currently being developed in Hue.
- #18 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #19 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #20 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #21 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #22 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #23 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #24 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #25 The Livy server is built upon Scalatra and Jetty. Creating a session is as simple as POSTing to a particular URL. Behind the scenes, livy will communicate with the YARN master to allocate some nodes to launch the interactive sessions. This is all done asynchronously as there’s not telling when there will be resources available to run the sessions. Once the nodes have been allocated, Livy will start an intepreter on one of the nodes which takes care of creating the Spark Context, which actually runs the spark operations. After it’s setup, the session signals to the Livy Server that it’s ready for commands. At that point, the Client can simply POST their code to a url on the livy server.
- #26 Let’s see it in action. On the left we see creating a “spark” session. You could also fill in “pyspark” and “sparkR” here if you want those sessions. On the right is us executing simple math in the session itself.
- #27 We don’t have too much time to drill down into the code, but we did want to take this moment to at least dive into how the interpreters work.
- #28 Livy’s interpreters are conceptually very simple devices. They take in one or more lines of code and execute them in a shell environment. These shells perform the computation and interact with the spark environment. They’re also abstract. As I mentioned earlier, Livy currently has 3 languages built into it: Scala, Python and R, with more to come.
- #30 Here is the interpreter loop that livy manages. First is to split up the lines and feed them one at a time into the interpreter. If the line is a regular, non-magic line, it gets executed and the result can be of three states. Success, where we’ll continue to execute the next line, incomplete, where the input is not a complete statement, such as an “if” statement with an open bracket. Or an error, which stops the execution of these lines. The other case are magic lines, which are special commands to the interpreter itself. For example, asking the interpreter to convert a value into a json type.
- #31 Now for some code. As we saw earlier, the interpreter is a simple state machine that executes code and eventually produces JSON responses by way of a Future.
- #32 Now for some code. As we saw earlier, the interpreter is a simple state machine that executes code and eventually produces JSON responses by way of a Future.
- #33 In order to implement this interface, the spark interpreter needs to first create the real interpreter, SparkIMain. It’s pretty simple to create. We just need to construct it with a buffer that acts as the interpreters Standard Output.
- #34 Once the SparkIMain has been initialized, we need to create the Spark Context that communicates with all of the spark workers. Injecting this variable into the interpreter is quite simple with this “bind” method.
- #35 Now that the session is up and running we can execute code inside of it. I’ve skipped some of the other book keeping in order to show the actual heart of the execution here. Ignore the magic case at the moment. Execution is also quite simple, we first temporarily replace standard out with our buffer, and then have the interpreter execute the code. There are three conditions for the response. First the command executed. Second, this code is incomplete because maybe it has an open parenthesis. Finally, an error if some exception occurred. Altogether quite simple and doesn’t require any changes to Spark to do this.
- #36 And now the magic. I mentioned earlier that livy supports type introspection. The way it does it is through these in-band magic commands which start with the percent command. The spark interpreter currently supports two magic commands, “json” and “table”. The “json” will convert any type into a json value, and “table” will convert any type into a table-ish object that’s used for our visualization.
- #37 Here is our json magic. it takes advantage of json4s’s Extraction.decompose to try to convert values. We special case RDDs since they can’t be directly transformed into json. Instead we just pull out the first 10 items so we can at least show something.
- #38 The table magic does something similar, but it’s a bit large to compress into slides. We’ll see it’s results next.
- #39 The table magic does something similar, but it’s a bit large to compress into slides. We’ll see it’s results next.
- #40 Finally here it is in action. Here we’re taking our shakespeare code from earlier. If we run this snippet inside livy, it returns an output mimetype of application/json, with the results inlined without encoding in the output.
- #41 Finally here it is in action. Here we’re taking our shakespeare code from earlier. If we run this snippet inside livy, it returns an output mimetype of application/json, with the results inlined without encoding in the output.
- #43 Fingers crossed for a lot of reasons, it’s master and the VM was broken till 4 AM. Next: learn more