Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Polyglot Applications with GraalVM

667 views

Published on

The JVM is already a runtime for many languages. With the optimizing Graal compiler added to Java 11 and the language implementations in Truffle for Ruby, Python, JavaScript, and R it becomes possible to run them natively on the JVM, even exchanging data between them.

Michael Hunger explains the concepts behind Truffle and Graal and uses a practical example to show how you can use Python and JavaScript for “stored procedures” in a JVM-based database.
He demonstrates how to optimize the startup time of your application and container images by precompiling it to machine-code and examines its limits and the difference it makes. But nothing is perfect—Michael discusses the limitations and compares performances for the full picture.

Presentation at OSCON, PDX 2019.
https://conferences.oreilly.com/oscon/oscon-or/public/schedule/detail/76092

Published in: Software
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (2019 Update) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download Full EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download Full doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download PDF EBOOK here { https://soo.gd/irt2 } ......................................................................................................................... Download EPUB Ebook here { https://soo.gd/irt2 } ......................................................................................................................... Download doc Ebook here { https://soo.gd/irt2 } ......................................................................................................................... ......................................................................................................................... ................................................................................................................................... eBook is an electronic version of a traditional print book THIS can be read by using a personal computer or by using an eBook reader. (An eBook reader can be a software application for use on a computer such as Microsoft's free Reader application, or a book-sized computer THIS is used solely as a reading device such as Nuvomedia's Rocket eBook.) Users can purchase an eBook on diskette or CD, but the most popular method of getting an eBook is to purchase a downloadable file of the eBook (or other reading material) from a Web site (such as Barnes and Noble) to be read from the user's computer or reading device. Generally, an eBook can be downloaded in five minutes or less ......................................................................................................................... .............. Browse by Genre Available eBooks .............................................................................................................................. Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, ......................................................................................................................... ......................................................................................................................... .....BEST SELLER FOR EBOOK RECOMMEND............................................................. ......................................................................................................................... Blowout: Corrupted Democracy, Rogue State Russia, and the Richest, Most Destructive Industry on Earth,-- The Ride of a Lifetime: Lessons Learned from 15 Years as CEO of the Walt Disney Company,-- Call Sign Chaos: Learning to Lead,-- StrengthsFinder 2.0,-- Stillness Is the Key,-- She Said: Breaking the Sexual Harassment Story THIS Helped Ignite a Movement,-- Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones,-- Everything Is Figureoutable,-- What It Takes: Lessons in the Pursuit of Excellence,-- Rich Dad Poor Dad: What the Rich Teach Their Kids About Money THIS the Poor and Middle Class Do Not!,-- The Total Money Makeover: Classic Edition: A Proven Plan for Financial Fitness,-- Shut Up and Listen!: Hard Business Truths THIS Will Help You Succeed, ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • accessibility Books Library allowing access to top content, including thousands of title from favorite author, plus the ability to read or download a huge selection of books for your pc or smartphone within minutes.........ACCESS WEBSITE Over for All Ebooks ..... (Unlimited) ......................................................................................................................... Download FULL PDF EBOOK here { https://urlzs.com/UABbn } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • ..............ACCESS that WEBSITE Over for All Ebooks ................ ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { https://urlzs.com/UABbn } ......................................................................................................................... Download Full EPUB Ebook here { https://urlzs.com/UABbn } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • If you want to download or read this book, copy link or url below in the New tab ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { https://urlzs.com/UABbn } .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • If you want to download or read this book, Copy link or url below in the New tab ......................................................................................................................... DOWNLOAD FULL PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download Doc Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... .........................................................................................................................
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Polyglot Applications with GraalVM

  1. 1. POLYGLOT WITH GRAALVM O S C O N 2 0 1 9 / M I C H A E L H U N G E R / @ M E S I R I I
  2. 2. MICHAEL HUNGER Caretaker General, Neo4j Head of Neo4j Labs Disturber of the Peace Java Champion (graphs)-[:ARE]->(everywhere) Twitter & Medium: @mesirii
  3. 3. WRITING ABOUT GRAAL SINCE 2014
  4. 4. WARNING: WHIRLWIND TOUR DONT‘T READ THE CODE
  5. 5. POLYGLOT? NOT EVERYONE IS A JAVA DEVELOPER!
  6. 6. POLYGLOT? We have: • Isn‘t the JVM already polyglot? –Scala, Groovy, Kotlin, Clojure, Frege … –JRuby, Jython, … L We want: • More languages, better performance
  7. 7. WHY SHOULD I CARE?
  8. 8. WHATS IN FOR ME?• Better JVM performance • Maintainable JIT compiler • Faster evolution of Java • With Truffle Language Runtime –Run JavaScript, Ruby, R, Python, LLVM code efficiently on the JVM • With Substrate VM –Binaries for Language Runtimes –AOT compiled native images of your applications Lego box
  9. 9. GETTING STARTED WHO READS THE INSTRUCTIONS
  10. 10. HOW CAN I USE IT?• Dedicated GraalVM Download or using sdkman gu (graal-updater) Utility js/node, ruby, python, R runtimes native-image tool • Java 11 with command line flags -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+UseJVMCICompiler https://www.graalvm.org/downloads/
  11. 11. JAVA 11 sdk use java 11.0.1-open java -Diterations=3 CountUppercase I‘m happy to be back in Portland, OR for OSCON 2019 -XX:+UnlockExperimentalVMOptions -XX:+UseJVMCICompiler -Dgraal.PrintCompilation=true
  12. 12. GRAALVMsdk install java 19.1.0-grl gu install R python ruby native-image gu list ComponentId Version Component name ----------------------------------- graalvm 19.1.0 GraalVM Core R 19.1.0 FastR native-image 19.1.0 Native Image python 19.1.0 Graal.Python ruby 19.1.0 TruffleRuby java –version OpenJDK 64-Bit GraalVM CE 19.1.0 (build 25.212-b03-jvmci-20-b04, mixed mode) Lego Instructions built
  13. 13. PYTHON # graalpython fac.py 2500 import sys def fac(n): if n==1: return 1 else: return fac(n-1)*n x = int(sys.argv[1]) print("Factorial for {} is {}" .format(x,fac(x)))
  14. 14. LLVM BITCODE#include <stdio.h> int main() { printf("Hello from GraalVM!n"); return 0; } clang -c -O1 -emit-llvm hello.c lli hello.bc
  15. 15. R PLOTTINGR --version:graalvm data <- "https://raw.githubusercontent.com/selva86/datasets/master/proglanguages.csv" library(ggplot2) library(treemapify) proglangs <- read.csv(data) ggplot(proglangs, aes(area = value, fill = parent, label = id, subgroup = parent)) + geom_treemap() + geom_treemap_subgroup_border() + geom_treemap_subgroup_text() + geom_treemap_text()
  16. 16. JS POLYGLOT node --version:graalvm node --jvm const BigInteger = Java.type("java.math.BigInteger") let a = new BigInteger("10") .add(new BigInteger("8") .multiply(new BigInteger("4"))) console.log(a.toString()) > 42
  17. 17. GRAAL COMPILER OPTIMIZING COMPILER IN JAVA
  18. 18. VISION STATEMENT Create an extensible, modular, dynamic, and aggressive compiler using object- oriented and reflective Java programming, a graph- based and visualizable intermediate representation, and Java snippets. —ThomasWürthinger
  19. 19. G R A A L ! ? • JIT-Compiler implemented in Java !?! • Aggressively optimizing –e.g. inlining POJOS/DTOS –Inlining streams –Speeds up many typical Java/Scala programs • Uses compute graph for optimization • New compiler interface (JVMCI)
  20. 20. GRAAL COMPILER OPTIMIZATIONS
  21. 21. GRAALVM BOX OF JOY GRAAL ❤ TRUFFLE ❤ SUBSTRATE
  22. 22. GraalVM is a high-performance, embeddable, polyglotVirtual Machine for running applications written in JavaScript, Python, Ruby, R, JVM-based languages like Java, Scala, Kotlin, and LLVM-based languages such as C and C++. Additionally, GraalVM allows efficient interoperability between programming languages and compiling Java applications ahead-of-time into native executables for faster startup time and lower memory overhead. https://github.com/oracle/graal/releases BIGGER ON THE INSIDE
  23. 23. • Collection of Research Projects – TruffleRuby / FastR • Maxine (Research)VM in Java • „A Joke?“ • „Java-on-Java“ John Rose – Project Metropolis • Power Combo: – Substrate-VM – Truffle – Graal Compiler – AOT Compilation HISTORY
  24. 24. GRAALVM • Oracle Labs Project • Versions – 19.1.1 (quarterly release) • Integrated – JVM 1.8.x – Node.js 10.x / ECMAScript 2019 – LLVM bitcode runtime • Supports – Truffle Runtime – Language Packs (via gu) – Native Image AOT • Editions – Community (GPL v2 w/ CP-Exception – Enterprise (faster, sandboxing, commercial support) – Oracle Database Engine
  25. 25. NATIVE IMAGE MACHINE CODE BABY
  26. 26. NATIVE IMAGE • Aggressive Ahead of time compilation (AOT) • Extremely fast startup time • Small binary executables for current OS • Class initialization during build • For FaaS, Commandline • Microservices: Micronaut, Helidon, Quarkus, Spring (soon) • No classloading / class metadata • Limitations: – no reflection, no later classloading, no initializer dynamics – Slow build https://medium.com/graalvm/lightweight-cloud-native-java-applications- 35d56bc45673
  27. 27. TRUFFLE L A N G U A G E R U N T I M E
  28. 28. TRUFFLE • Language Runtime • API & Type system • Implement language constructs • Annotated Java Methods – Tooling,Testing • Generic or specialized operations
  29. 29. TRUFFLE + GRAAL
  30. 30. TRUFFLE GRAAL• Integrates with Graal Compiler • Partial Evaluation • Optimize special cases based on steady state assumption • Deoptimize (trap) on failed assumptions
  31. 31. SAMPLE LANGUAGE T R U F F L E E X A M P L E L A N G U A G E
  32. 32. TRUFFLE: ADDITION-NODE (SL) @NodeInfo(shortName = "+") public abstract class SLAdditionNode extends SLBinaryNode { @Fallback protected Object typeError(Object left, Object right) { throw SLException.typeError(this, left, right); } @Specialization(rewriteOn = ArithmeticException.class) protected long add(long left, long right) { return Math.addExact(left, right); } @Specialization @TruffleBoundary protected SLBigNumber add(SLBigNumber left, SLBigNumber right) { return new SLBigNumber(left.getValue().add(right.getValue())); } @Specialization(guards = "isString(left, right)") @TruffleBoundary protected String add(Object left, Object right) { return left.toString() + right.toString(); } protected boolean isString(Object a, Object b) {…}
  33. 33. LANGUAGES BOX OF COLORS
  34. 34. JAVASCRIP T • Main target language via graaljs • Replacement for Rhino/Nashorn • EcmaScript 2019 & Node.js (10.15.2) compat • 90% of 95k npm packages • Graaljs can run slowly w/o Graal • ScriptEngine support • org.graalvm.js:js/js-scriptengine https://www.graalvm.org/docs/reference-manual/languages/js/
  35. 35. GRAAL PYTHON • Early stage support Python 3.7 • Goal: „SciPy“ support • No python debugger, but GraalVMs https://www.graalvm.org/docs/reference-manual/languages/python/
  36. 36. FAST-R • Compatible with GNU R (3.5.1) • Much faster than other R implementations • R/Rscript • Install packages (e.g. ggplot2, Shiny) • Minimal: graphics package • Compatibility checker • Tools (debugger, profiler) • Java based Graphics https://www.graalvm.org/docs/reference-manual/languages/r/
  37. 37. TRUFFLE RUBY • Initial research project • Quite complete coverage (2.6.2) incl. c-extensions • Parallel threads • Faster than MRI/JRuby (up to 31x) • Recent: fibers • Missing: suspend, continuation, fork https://www.graalvm.org/docs/reference-manual/languages/ruby/
  38. 38. LLVM • LLVM 6.0.0 Bitcode • Via Sulong a LLVM implementation in Java via Truffle • Can use native libraries • lli to execute LLVM Bitcode • Sandbox in GraalVM Enterprise – sandbox libraries – virtualize syscalls – memory on managed heap https://medium.com/graalvm/safe-and-sandboxed-execution-of- native-code-f6096b35c360
  39. 39. POLYGLOT DO WHAT YOU WANT THURSDAY
  40. 40. TRUFFLE • Based on Truffle Implementations of dynamic languages • Joint underlying API / Typesystem • Context – eval – bind – invoke • Source • Value.* docs.oracle.com/en/graalvm/enterprise/19/sdk/org/graalvm/polyglot/Context.html
  41. 41. VALUE • The „Any“ type across languages – Scalar – List/Array – Host/Proxy Object w/ Members – Function/Invoke/Executable • Provides some semantics and conversions • Removes need for (de)serialization • Thread safety depends on language support docs.oracle.com/en/graalvm/enterprise/19/sdk/org/graalvm/polyglot/Value.html
  42. 42. CAPABILITIES Polyglot support: --polyglot Eval • Polyglot.eval("python","21*2") • polyglot.eval(language="ruby", file="./my_ruby_file.rb") Export / Import • polyglot.import_value(„name“) • Polyglot.export_value(„name“,value) • ctx.getBindings.put/getMember(name,value) Object Access • Object.size() / call / invoke • Object[name] / • Via Value.*
  43. 43. JAVA INTEROP JVM Support: --jvm Flag Import Java.import “java.util.UUID“ java.type("java.math.BigInteger") from java.util import ArrayList Helpers • isFunction / isObject / isSymbol / isNull / instance_of Access • allow<Host/Native/Polyglot/*>Access
  44. 44. POLYGLOT EXAMPLES
  45. 45. POLYGLOT EXAMPLE (1) // gu install ruby python R // groovy PolyTest1.groovy @Grab("org.graalvm.sdk:graal-sdk:19.1.0") import org.graalvm.polyglot.* ctx = Context.newBuilder().allowAllAccess(true).build() ctx.eval("js", "print('Hello JavaScript!');") ctx.eval("R", "print('Hello R!');"); ctx.eval("ruby", "puts 'Hello Ruby!'"); ctx.eval("python", "print('Hello Python!')");
  46. 46. POLYGLOT EXAMPLE (PYTHON-JAVA) import java generic = java.type('org.testcontainers.containers.GenericContainer') container = generic('nginx') container.setExposedPorts([80]) container.start(); print('%s:%s' % (container.getContainerIpAddress(), container.getMappedPort(80))); https://medium.com/graalvm/using-testcontainers-from-a-node-js-application-3aa2273bf3bb
  47. 47. POLYGLOT EXAMPLE (C) #include <stdio.h> #include <curl/curl.h> long request() { CURL *curl = curl_easy_init(); long response_code = -1; if(curl) { CURLcode res; curl_easy_setopt(curl, CURLOPT_URL, "http://example.com"); res = curl_easy_perform(curl); if(res == CURLE_OK) { curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code); } curl_easy_cleanup(curl); } return response_code; }
  48. 48. POLYGLOT EXAMPLE (JAVA+C-LLVM) // clang -c -O1 -emit-llvm use-curl.c && groovy CurlTest.groovy @Grab("org.graalvm.sdk:graal-sdk:1.9.10") import org.graalvm.polyglot.* polyglot = Context.newBuilder() .allowAllAccess(true) .option("llvm.libraries", "/usr/lib/libcurl.dylib") .build() source = Source .newBuilder("llvm", new File("use-curl.bc")) .build() result = polyglot.eval(source) responseValue = result.getMember("request").execute() responseCode = responseValue.asLong() print(responseCode)
  49. 49. R+PYTHON pycode <- ‚ library polyglot PI=polyglot.import("PI") def area(radius): return 2*radius*PI area ' export("PI",pi) area <- eval.polyglot("python",pycode) print(area(5))
  50. 50. JAVA+R @Value(value = "classpath:plot.R") private Resource rSource; @Autowired private Function<DataHolder, String> plotFunction; @Bean Function<DataHolder, String> getPlotFunction(@Autowired Context ctx) { Context ctx = Context.newBuilder().allowAllAccess(true).build(); Source source = Source.newBuilder("R", rSource.getURL()).build(); return ctx.eval(source).as(Function.class); } @RequestMapping(value = "/load", produces = "image/svg+xml") public synchronized ResponseEntity<String> load() { HttpHeaders responseHeaders = new HttpHeaders(); responseHeaders.set("Refresh", "1"); double load = getOperatingSystemMXBean().getSystemLoadAverage(); String svg = plotFunction.apply(new DataHolder(load)); return new ResponseEntity<String>(svg,responseHeaders,HttpStatus.OK); }
  51. 51. POLYGLOT EXAMPLE (JAVA+R) library(ggplot2) data <<- numeric(100) function(dataHolder) { svg() data <<- c(data[2:100],dataHolder$value) plot <- ggplot(data= data.frame(systemLoad=data, time =-99:0), aes(x=time, y=systemLoad, group=1)) + geom_line(color="orange") + expand_limits(x=0, y=0) print(plot) svg.off() }
  52. 52. POLYGLOT SHELL Context context = Context.newBuilder().allowAllAccess(true).build(); Set<String> languages = context.getEngine().getLanguages().keySet(); out.println("Shell for " + languages + ":"); String language = languages.iterator().next(); while (true) { out.print(language + "> "); String line = input.readLine(); if (line == null) break; else if (languages.contains(line)) language = line; else { Source source = Source.newBuilder(language, line, "<shell>") .interactive(true).buildLiteral(); context.eval(source); } }
  53. 53. POLYGLOT OPTIONS Print GraalVM version information and exit. --version:graalvm Print GraalVM version information and continue execution. --show-version:graalvm Start with Debugger --inspect Run using the native launcher with limited Java access --native Run on the JVM with Java access. --jvm Run with all other guest languages accessible. --polyglot Pass options to the host VM --vm.[option] Options for all installed languages, tools, host VM --help:languages,tools,vm,expert Internal options for debugging language impl and tools. --help:internal
  54. 54. TOOLING MAKING OUR LIFES EASIER
  55. 55. DEBUGGER
  56. 56. CUSTOM JVISUALVM
  57. 57. IDEAL VISUALIZER
  58. 58. EXTENDING DATABASES MORE POWER TO THE MACHINE
  59. 59. WHY? • Allow “programmatic“ extensions • Move processing to data, avoid network transfers • Not just procedures in – PL/SQL – Java • Reuse existing code – public packages (e.g. validators, data science, visualization) – domain specific code • Because you can!
  60. 60. EXTENDING DATABASES• „Multilingual Engine“ – native library integration • Available from Oracle Labs • For Oracle and MySQL • Deploy custom functions in JavaScript, R, Python with dbjs dbjs deploy -u <user> -p <pass> -c localhost:1521/ORCLCDB <package> • Create statically and call from SQL https://www.graalvm.org/docs/examples/mle-oracle/
  61. 61. SQL FUNCTION IN JAVASCRIPT CREATE OR REPLACE JAVASCRIPT SOURCE NAMED "hello.js" AS module.exports.greet = function(a) { return "Hello " + a; }; CREATE OR REPLACE FUNCTION greet(a IN VARCHAR2) RETURN VARCHAR2 AS LANGUAGE JAVASCRIPT NAME 'hello.js.greet(a string) return string‘; SELECT greet('GraalVM') FROM DUAL; Hello GraalVM
  62. 62. NEO4J (GRAPHS)-[:ARE]->(EVERYWHERE)
  63. 63. NEO4J IN A TWEET@Neo4j is an open-source native graph database designed to store, manage and query highly connected data efficiently with the Cypher Query Language. It runs transactional and analytic workloads, supports visualization and is extendable with custom functions. neo4j.com/developer
  64. 64. NEO4J • Runs on the JVM • Full Stack database • Declarative Graph Query Language • Binary Protocol • Drivers for most languages • Visual Browser • Integrations like GraphQL, Kafka • Custom Procedures and Functions
  65. 65. OSCON CONFERENCE GRAPH https://github.com/neo4j-examples/oscon-graph
  66. 66. TALK RECOMMENDATION // top 10 talks (that I‘ve not see) // that peers who like the same talks I did also liked MATCH (me:User {name:“Michael“}) -[:FAVORITED]->(:Event)<-[:FAVORITED]-(peer:User) -[:FAVORITED]->(reco:Event) WHERE NOT (me)-[:FAVORITED]->(reco) RETURN reco.name, count(*) AS freq ORDER BY freq DESC LIMIT 10
  67. 67. TALK RECOMMENDATION
  68. 68. NEO4J CUSTOM PROCEDURES P U T YO U R S M A RT S I N T H E DATA B A S E
  69. 69. NEO4J CUSTOM FUNCTIONS• Annotated Java Methods • Loaded at startup • Computation or Aggregation with Functions • Data processing & streaming with Procedures • Accessible from Cypher Query Language
  70. 70. CUSTOM FUNCTION @UserFunction @Description(„Generates an UUID“) public String uuid() { return UUID.randomUUID().toString(); } CREATE (:Event {id: uuid(), name:“Graph Algorithms“ });
  71. 71. POLYGLOT FUNCTIONS U S I N G T R U F F L E A P I S & G R A A L V M
  72. 72. POLYGLOT FUNCTIONS • Using Truffle API • Declare + invoke function with params, or • Execute code with bindings 1. Use Context to run polyglot code 2. Use Cypher to declare dynamic language functions 3. Install language files from directory 4. Store code in db to restore at restart / in cluster
  73. 73. EVAL CODE A D H O C E X E C U T I O N
  74. 74. EVAL CODE PROCEDURE @Procedure(„scripts.execute“) public Object executeCode(String lang, String code, Map<String,Object> props) { Context ctx = Context.newBuilder().allowAllAccess(true).build(); Bindings bindings = ctx.getBindings(); props.forEach(bindings::putMember); bindings.putMember("label", ctx.eval("js", "s => org.neo4j.graphdb.Label.label(s)")); bindings.putMember("db", graphDatabase); return ctx.eval(lang, code).asHostObject(); }
  75. 75. E VA L C O D E
  76. 76. EVAL CODE CALL scripts.execute(' Java.import "org.neo4j.graphdb.Label db = Polyglot.import("db") props = db.findNode(Label.label("Event"), "name",“OSCON") .getAllProperties().entrySet().toArray() Polyglot.as_enumerable(props) .map{|e| "#{e.getKey()} -> #{e.getValue()}"} .join(",") ',{},'ruby')
  77. 77. REGISTER FUNCTIONS DY N A M I C A L LY
  78. 78. INSTALL FUNCTION (2) Context ctx = Context.newBuilder().allowAllAccess(true).build() @Procedure(„scripts.register“) public void registerFunction(String lang, String name, String code) { Value function = ctx.eval(lang, code); ctx.getBindings(lang) .putMember(name, function); }
  79. 79. INVOKE EXISTING FUNCTION (2) @UserFunction(„scripts.run“) public Object executeFun(String lang, String name, Object…args) { return ctx.getBindings(lang) .getMember(name) .execute(params).asHostObject(); }
  80. 80. REGISTER NATIVE FUNCTION (2) @UserFunction public void registerFunction(String code, String name, String lang) { ctx.getBindings(lang).putMember(name, ctx.eval(code)); procedures.register(new BasicUserFunction(signature(name)) { @Override public AnyValue apply(org.neo4j.proc.Context c, AnyValue[] input) { return resultOf(context.getBindings(lang).getMember(name) .execute(paramsFor(input))); } }); }
  81. 81. SCRIPT DIRECTORY A U TO L O A D S C R I P T F I L E S
  82. 82. SOURCE WATCHER watchService = FileSystems.getDefault().newWatchService(); pathName = new File(scriptsDir, target.getDirName()) pathToWatch = getTargetPath(pathName); pathToWatch.register(watchService, ENTRY_CREATE,ENTRY_DELETE, ENTRY_MODIFY); public void run() { WatchKey watchKey; while ((watchKey = watchService.take()) != null) { for (WatchEvent<?> event : watchKey.pollEvents()) updateScript(event.kind(), event.context()); } }
  83. 83. IMPLEMENT SCRIPT FUNCTION (1) public class ScriptFunction implements CallableUserFunction { private final UserFunctionSignature signature; private final String name; private final String sourceCode; private transient volatile Source source; public ScriptFunction(String language, String name, String sourceCode) { this.name = name; this.sourceCode = sourceCode; this.source = Source.newBuilder(this.language, this.sourceCode, this.name).build(); this.signature = generateSignature(); } private UserFunctionSignature generateSignature() { final QualifiedName qualifiedName = new QualifiedName(Arrays.asList("scripts", "fn"), name); final List<FieldSignature> input = IntStream.range(0, numberOfArguments); .mapToObj(i -> FieldSignature.inputField("p" + i, NTAny, DefaultParameterValue.nullValue(NTAny))) .collect(toList()); return new UserFunctionSignature(qualifiedName, input, NTAny, null, new String[0], null, false); }
  84. 84. IMPLEMENT SCRIPT FUNCTION (2) @Override public AnyValue apply(Context ctx, AnyValue[] input) throws ProcedureException { try (org.graalvm.polyglot.Context context = PolyglotContext.newInstance()) { GraphDatabaseAPI db = ctx.get(Context.DATABASE_API); Log log = ctx.get(Context.DEPENDENCY_RESOLVER).resolveDependency(LogService.class) .getUserLog(ScriptFunction.class); Value bindings = context.getPolyglotBindings(); bindings.putMember("db", db); bindings.putMember("log", log); Value langBindings = context.getBindings(language); langBindings.putMember(name, context.eval(source)); return resultFor(langBindings.getMember(name).invoke(input)); } }
  85. 85. IMPLEMENT SCRIPT FUNCTION (3) private AnyValue resultFor(Value result) { if (result.isNull()) { return null; } if (result.isNumber()) { return ValueUtils.asAnyValue(result.asDouble()); } if (result.isBoolean()) { return ValueUtils.asAnyValue(result.asBoolean()); } if (result.isHostObject()) { return ValueUtils.asAnyValue(result.asHostObject()); } return ValueUtils.asAnyValue(result.asString()); }
  86. 86. NEXT STEPS
  87. 87. NEXT STEPS• Better bi-directional conversions • Handle Isolation / Threading • Integrate with Python / R ~ – Graph processing / algorithms – Data Science / ML / graph_net – Large Scale Plotting – Investigate GraphBlas (C-Library) • Allow installing packages (e.g npm) • Move into Neo4j Labs library
  88. 88. THE GOOD AND EXCELLENT
  89. 89. THE GOOD • It works! – All the languages – JVM Interop – Polyglot Interop • Continuous improvements by the Oracle Team • Lots of great docs & articles • Adding those database extensions was really easy
  90. 90. THE BAD AND UGLY
  91. 91. THE BAD • Python support still preliminary • Too many polyglot indirections are costly • Interop a bit kludgy • Error messages not that helpful • No automatic mapping of collection types (map/hash/dict) and (list/collection) – only for arrays
  92. 92. FIND MORE ALL THE THINGS
  93. 93. MORE• graalvm.org • graalvm.org/docs • medium.com/graalvm ~/graalvm-ten-things-12d9111f307d • github.com/oracle/graal • graalvm.org/docs/reference-manual/compatibility – package compat checker • youtu.be/a-XEZobXspo – 3hr deep dive • neo4j.com/developer • r.neo4j.com/algo-book • github.com/neo4j-contrib/neo4j-script-procedures Twitter & Medium: @mesirii
  94. 94. Graphs in AI and ML Alicia Frame, Senior Data Scientist, Neo4j Jake Graham, Lead Product Manager for AI and Graph Analytics Intro to Neo4j for Developers Jennifer Reif, Developer Relations Engineer, Neo4j Neo4j Bolt Driver Architecture Now and inThe Future Nigel Small,Tech Lead for Neo4j Drivers team GRANDstack: Graphs ALL the Way Down William Lyon, Software Engineer, Neo4j Labs All-new SDN-RX: Reactive Spring Data Neo4j Gerrit Meier, Software Engineer, Neo4j Graph Embeddings Alicia Frame, Senior Data Scientist, Neo4j Graph ModelingTips andTricks Max De Marzi, Field Engineer and Graph Expert, Neo4j APOC Pearls -The bestTips andTricks Michael Hunger, Director of Neo4j Labs Visualizing Graph Data in JavaScript Will Lyon, Software Engineer, Neo4j Labs Creating a Data Marvel (Comics) with Spring and Neo4j Jennifer Reif, Developer Relations Engineer, Neo4j Keynote and Announcements Emil Eifrem, CEO and Co-Creator of Neo4j October 10 8am NYC, 1pm London, 530pm Mumbai
  95. 95. PLEASE RATE MY SESSION
  96. 96. THANK YOU! QUESTIONS IN HALLWAY

×