Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

"ClojureScript journey: from little script, to CLI program, to AWS Lambda function" Alexander Khokhlov

229 views

Published on

In this talk, I’d like to show that engineer, in order to make progress, should develop its own “outside the box” thinking. Experienced programmer regardless of the language ought to look at things from various standpoints outside the commonly used paradigm. This allows her to choose the proper strategy which fits the task, customer’s requirements, saves time and money. Having our product as an example, I’d like to show new language and new methods, which are not that frequently used in the mainstream. I believe this will broaden the horizon of the conference audience.

Published in: Software
  • Be the first to comment

  • Be the first to like this

"ClojureScript journey: from little script, to CLI program, to AWS Lambda function" Alexander Khokhlov

  1. 1. Alexander Khokhlov @nots_ioNots.io ClojureScript journey From little script, to CLI program, to AWS Lambda function
  2. 2. Co-Founder Nots.io 01
  3. 3. README.md Code Comments JavaDoc GDocs Confluence / DokuWiki / Wiki system
  4. 4. Nots.io Add docs for block of code, function, module, file, commit or branch 01
  5. 5. Notes, Tied to Code Easy to Discover Easy to Explore Easy to Get Scope Easy to Ask and Discuss 01 • Nots.io
  6. 6. We Track Relevance You always know what’s fresh and what’s not. Promotes keeping docs up-to-date. Rewarding when everything is ✅ 01 • Nots.io
  7. 7. Discuss with your Team You won’t loose a dispute that is written down. It’s tied and has context 01 • Nots.io
  8. 8. And many more Integration with GitHub IDE/Editors plugins Markdown formatting @mentions GitHub PR as a Note Attachments One-on-one conversations … 01 • Nots.io
  9. 9. 02 https://medium.com/basecs/reading-code-right-with-some-help-from-the-lexer-63d0be3d21d The task: Detect comment scope
  10. 10. https://medium.com/basecs/reading-code-right-with-some-help-from-the-lexer-63d0be3d21d 02 • The task Analyze! AST is the best But tokens are good enough
  11. 11. Tokenizers ANTLR
 Pygments
 vscode-textmate 02 • The task
  12. 12. vscode-textmate 02 • The task https://medium.com/basecs/reading-code-right-with-some-help-from-the-lexer-63d0be3d21d var vsctm = require('vscode-textmate'); var registry = new vsctm.Registry({ loadGrammar: function (scopeName) { var path = ‘./javascript.tmbundle/Syntaxes/JavaScript.plist'; if (path) { return new Promise((c, e) => { fs.readFile(path, (error, content) => { if (error) { e(error); } else { var rawGrammar = vsctm.parseRawGrammar( content.toString(), path); c(rawGrammar); }});});} return null; }}); // Load the JavaScript grammar and any other grammars included by it async. registry.loadGrammar('source.js').then(grammar => { // at this point `grammar` is available... var lineTokens = grammar.tokenizeLine( 'function add(a,b) { return a+b; }'); for (var i = 0; i < lineTokens.tokens.length; i++) { var token = lineTokens.tokens[i]; console.log('Token from ' + token.startIndex + ‘ to ' + token.endIndex); } }); Sample
  13. 13. vscode-textmate 02 • The task https://medium.com/basecs/reading-code-right-with-some-help-from-the-lexer-63d0be3d21d { tokens: [ { startIndex: 0, endIndex: 1, scopes: [ 'source.js', 'string.quoted.single.js', 'punctuation.definition.string.begin.js' ], text: "'", line: 0 }, { startIndex: 1, endIndex: 11, scopes: [ 'source.js', 'string.quoted.single.js' ], text: 'use strict', line: 0 }, { startIndex: 11, endIndex: 12, scopes: [ 'source.js', 'string.quoted.single.js', 'punctuation.definition.string.end.js' ], text: "'", line: 0 }, { startIndex: 12, endIndex: 13, scopes: [ 'source.js', 'punctuation.terminator.statement.js' ], text: ';', line: 0 }, Output
  14. 14. Tool which works great with sequences 02 • The task 🤔
  15. 15. ClojureScript 03
  16. 16. 03 • CLJS LISP
  17. 17. 03 • CLJS LISP
  18. 18. 03 • CLJS Dialect of LISP Dynamic Immutable Persistent Compiled to JS Homoiconic Data-Driven
  19. 19. 03 • CLJS Data-driven
  20. 20. 03 • CLJS Composable
  21. 21. 03 • CLJS 98 functions to work with collections🔥 Isn’t that enough?
  22. 22. 03 • CLJS "It is better to have 100 functions operate on one data structure than 10 functions on 10 data structures." 
 —Alan Perlis
  23. 23. 03 • CLJS All together now
  24. 24. 03 • CLJS Interop ;; Globals ;; alert("Hello!") (js/alert "Hello!") ;; Function Call ;; "string".toUpperCase() (.toUpperCase "string") ;; Properties ;; "string".length (.-length "string") Interoperability with JS
  25. 25. 03 • CLJS Interop ;; Chain calls ;; "string".toUpperCase().charCodeAt(1).toString() (.toString (.charCodeAt (.toUpperCase "string") 1)) (.. "string" (toUpperCase) (charCodeAt 1) (toString)) (.. "string" toUpperCase (charCodeAt 1) toString) (-> "string" .toUpperCase (.charCodeAt 1) .toString) ;; Chain properties ;; document.body.lastChild.innerHTML.length (.. js/document -body -lastChild -innerHTML -length) (-> js/document .-body .-lastChild .-innerHTML .-length) Interoperability with JS
  26. 26. 03 • CLJS Interop (ns myapp) (defn ^:export func [a] (str "Hey, " a)) ;; in JS: ;; myapp.func("NodeUA!");
  27. 27. How we did it 04 🏗
  28. 28. https://medium.com/basecs/reading-code-right-with-some-help-from-the-lexer-63d0be3d21d Tokenize 04 • How we did it
  29. 29. vscode-textmate (require '[cljs.build.api :as b]) (b/build "src" {:main 'notsapp.citation.core :optimizations :simple :target :nodejs :npm-deps {:vscode-textmate “4.1.1”} :install-deps true :output-to "notsapp_citation.js"}) 04 • How we did it
  30. 30. Get tokens (ns notsapp.citation.registry (:require [vscode-textmate :as vstm] [cljs-node-io.core :as io] [cljs-node-io.fs :as fs])) (def reg (new vstm/Registry)) (defn load-all-grammars [] (->> (fs/readdir “grammars") (filter #(re-find #".json$" %)) (map #(let [grammar-path (str "grammars/" %) grammar (io/slurp grammar-path)] (->> (vstm/parseRawGrammar grammar grammar-path) (.addGrammar reg)))))) (defn tokenize-file [file source scope-name] (when-let [grammar-promise (.grammarForScopeName reg scope-name)] (.then grammar-promise #(.tokenizeLine % source))))) 04 • How we did it
  31. 31. (defn common-block-scopes [input-tokens] (let [last-idx (-> input-tokens count dec)] (->> input-tokens (keep-indexed (fn [idx token] (when (or (re-find #"((r)?n){2,}" (-> token :text (str/replace #"[ t]+" ""))) (= idx last-idx)) idx))) (cons 0) distinct (partition 2 1) (map (fn [[start end]] (subvec input-tokens (inc start) end)))))) Transform stream of tokens 04 • How we did it
  32. 32. Transform to CLI 05 📺
  33. 33. 05 • Transform to CLI Transform to CLI deps.edn {:deps {org.clojure/clojure {:mvn/version "1.9.0"} org.clojure/core.async {:mvn/version "0.4.490"} org.clojure/clojurescript {:mvn/version "1.10.439"} cljs-node-io {:mvn/version "1.1.2"} org.clojure/tools.cli {:mvn/version "0.4.1"} }}
  34. 34. Transform to CLI clojure.tools.cli (ns notsapp.citation.core (:require [cljs.nodejs :as nodejs] [clojure.tools.cli :as cli])) (def cli-options [["-l" "--lang LANG" "Language of a source code file passed via stdin"] ["-s" "--scope LINENUMBER" "Get the scope by given line number" :parse-fn #(js/parseInt %)] ["-c" "--comments" "Show comments scopes"] ["-h" "--help"]]) (defn -main [& args] (let [opts (cli/parse-opts args cli-options) file (-> opts :arguments first) lang (-> opts :options :lang) scope-line-number (-> opts :options :scope) show-comments? (-> opts :options :comments)] (.exit nodejs/process 0)) (set! *main-cli-fn* -main) 05 • Transform to CLI
  35. 35. Transform to CLI stdin (defn read [] (.on stdin "readable" (fn on-readable [] (let [string (loop [buf (.alloc js/Buffer 0)] (if-let [data (.read stdin)] (recur (.concat js/Buffer #js [buf data])) ;else (.toString buf "utf8")))] (.removeListener stdin "readable" on-readable) string)))) 05 • Transform to CLI
  36. 36. Transform to CLI stdout (ns notsapp.citation.stdout (:require [clojure.string :as str] [cljs.nodejs :as nodejs])) (def stdout (.-stdout nodejs/process)) (defn write [data] (let [buf (.from js/Buffer data) data-len (.-length buf) len-buf (.alloc js/Buffer 4)] (.writeUInt32BE len-buf data-len 0) (->> #js [len-buf buf] (.concat js/Buffer) (.write stdout)))) 05 • Transform to CLI
  37. 37. Compile & exec > clj build.clj > node notsapp_citation.js (require '[cljs.build.api :as b]) (b/build "src" {:main 'notsapp.citation.core :optimizations :simple :target :nodejs :npm-deps {:vscode-textmate “4.1.1”} :install-deps true :output-to "notsapp_citation.js"})build.clj 05 • Transform to CLI
  38. 38. 06 Houston …
  39. 39. 06 • AWS Lambda AWS Lambda
  40. 40. Surprisingly simple to transform ; WAS ;(set! *main-cli-fn* -main) ;NOW (set! (.-exports js/module) #js {:scopelambda scopelambda}) (defn scopelambda [event ctx cb] (if-let [body (.parse js/JSON (.-body event))] (cb nil #js {:statusCode 200 :headers #js {"Content-Type" "text/plain"} :body "Hey There!"}) ;or else return BAD REQUEST response (cb nil #js {:statusCode 500 :headers #js {"Content-Type" "text/plain"} :body "Cannot parse request body"}))) 06 • AWS Lambda
  41. 41. Compile build.clj (require '[cljs.build.api :as b]) (b/build "src" {:main 'notsapp.citation.core :optimizations :simple :target :nodejs :npm-deps {:vscode-textmate “4.1.1”} :install-deps true :output-to "notsapp_citation.js"}) $ clj build.clj 06 • AWS Lambda
  42. 42. Deploy & exec $ serverless deploy serverless.yml package: include: - notsapp_citation.js - node_modules/** - grammars/** exclude: - src/** - .git/** - out/** functions: citation: handler: notsapp_citation.scopelambda $ serverless invoke -f citation -l 06 • AWS Lambda
  43. 43. 07 Testing
  44. 44. 07 • Testing Testing Entry point (ns notsapp.citation.core-test (:require [cljs.test :as t] [cljs.nodejs :as nodejs] [notsapp.citation.js-test])) (nodejs/enable-util-print!) (defn -main [& args] (t/run-tests 'notsapp.citation.js-test)) (set! *main-cli-fn* -main)
  45. 45. Testing The test (ns notsapp.citation.js-test (:require [cljs.test :refer-macros [deftest is] :as t])) (t/use-fixtures :once {:before load-all-grammars}) (deftest js-scopes-arrow (is (= (js-scope-arrow-funciton) [[ 2, 2 ], [ 4, 5 ], [ 7, 8 ]]))) 07 • Testing
  46. 46. Testing Async tests (deftest js-scopes-arrow (t/async done (-> (tokenize-and-prepare-file "samples/js/arrow-functions.js") (.then (fn [tokens] (let [scope (js-scope-arrow tokens)] (is (= scope [[2 2] [4 5] [7 8]]))) (done)))))) 07 • Testing
  47. 47. Testing Build & Execute > clj build.clj > node notsapp_tests.js (require '[cljs.build.api :as b]) (b/build (b/inputs "test" "src") {:main 'notsapp.citation.core-test :optimizations :none :target :nodejs :output-to "notsapp_tests.js" :npm-deps {:vscode-textmate "3.3.3" } :install-deps true :output-dir "out_tests"}) 07 • Testing
  48. 48. Final thoughts 08
  49. 49. 08 • Final thoughts Pros Simple, Elegant & Readable Easy to reason about Small compassable libraries, not frameworks Rich collection manipulation functions FP, immutability, purity, first-class functions High level data manipulation CLJ/CLJS code reuse Macros Seamless interop with JS/Node.js
  50. 50. Cons Learning curve, but it’s just an initial hump Need to study FP Good to deep dive into CLJ philosophy Additional JS code added by CLJS itself Still tangled error messages Compiled code is hard to read Need time to fully master the language Know what you do 08 • Final thoughts
  51. 51. Best Fit If the task is dedicated If the team is skilled enough If you see limit of your current stack If you’re curious enough If you want to broaden yours horizons If you want to be 10x productive In our opinion 08 • Final thoughts
  52. 52. Thank you 🤘
  53. 53. Alexander Khokhlov point@nots.io Nots.io nots.io blog.nots.io @nots_io facebook.com/nots.io

×