Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Hermetic Systems (without Docker)

462 views

Published on

Slides for a talk I gave at Dutch Clojure Days on March 25th, 2017. @wkf on Twitter.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Building Hermetic Systems (without Docker)

  1. 1. Building Hermetic Systems (without Docker) Will Farrell @wkf
  2. 2. Who am I? I’m the Lead Software Architect at MojoTech in Providence, RI, that’s who. We’re a software consultancy, and we’ve built many web and mobile apps.
  3. 3. What can you expect to learn? • How hermeticity is an example of functional design thinking • How to identify “leaks” in not-so-hermetic systems • How to leverage Clojure to build and manage hermetic systems
  4. 4. What are hermetic systems?
  5. 5. Airtight
  6. 6. Pure
  7. 7. Why is hermeticity desirable? ;; ... (ns determinism (:require [hermeticity])) (defn system [well-defined-input] well-defined-output) ;; ...
  8. 8. –Every Developer Ever “It works on my machine.”
  9. 9. What about Docker?
  10. 10. –Every Developer Ever “Let’s use Docker.”
  11. 11. –Every Developer Ever, Eventually “Let’s never use Docker again.”
  12. 12. xkcd Why are there leaks? • The JVM • The operating system • The electricity • …you get it
  13. 13. Which leaks can we fix?
  14. 14. External Libraries
  15. 15. What’s wrong with libraries? (defproject motrics "0.1.0-SNAPSHOT" :description "Mojotech Metrics" :url "https://github.com/mojotech/motrics" :license {:name "Eclipse Public License" :url "http://www.eclipse.org/legal/epl-v10.html"} :repositories {"elasticsearch-releases" "https://maven.elasticsearch.org/releases"} :dependencies [[org.clojure/clojure "1.7.0"] [com.stuartsierra/component "0.3.1"] [com.taoensso/timbre "4.2.1"] [com.taoensso/carmine "2.12.2"] [com.github.kstyrc/embedded-redis "0.6"] [org.elasticsearch/elasticsearch "2.2.0"] [org.elasticsearch.plugin/shield "2.2.0"] [environ "1.0.2"] [tentacles "0.5.1"] [suspendable "0.1.0"]] :main ^:skip-aot motrics.core :target-path "target/%s" :test-paths ["src/test"] :source-paths ["src/main"] :min-lein-version "2.0.0" :profiles {:uberjar {:aot :all} :dev {:jvm-opts ["-Xmx300m"] :source-paths ["src/dev"] :repl-options {:init-ns motrics.repl} :dependencies [[reloaded.repl "0.2.1"]]}})
  16. 16. Remember these?
  17. 17. External Services
  18. 18. What’s wrong with services? $ ping api.github.com ping: cannot resolve api.github.com: Unknown host ping: cannot resolve api.github.com: Unknown host ping: cannot resolve api.github.com: Unknown host ping: cannot resolve api.github.com: Unknown host ping: cannot resolve api.github.com: Unknown host ping: cannot resolve api.github.com: Unknown host ping: cannot resolve api.github.com: Unknown host
  19. 19. Should we rewrite everything? (No)
  20. 20. What are components? (defrecord ExampleComponent [options cache database scheduler] component/Lifecycle (start [this] (println ";; Starting ExampleComponent") ;; In the 'start' method, a component may assume that its ;; dependencies are available and have already been started. (assoc this :admin (get-user database "admin"))) (stop [this] (println ";; Stopping ExampleComponent") ;; Likewise, in the 'stop' method, a component may assume that its ;; dependencies will not be stopped until AFTER it is stopped. this))
  21. 21. What are systems? (defn example-system [config-options] (let [{:keys [host port]} config-options] (component/system-map :db (new-database host port) :scheduler (new-scheduler) :app (component/using (example-component config-options) {:database :db :scheduler :scheduler}))))
  22. 22. What about embedding?
  23. 23. Embed Elasticsearch (defrecord EmbeddedElasticsearchServer [config] component/Lifecycle (component/start [this] (update this :node #(or % (-> (:elasticsearch-server config) (update :settings (partial merge default-node-settings)) (elasticsearch/new-node) (elasticsearch/start-node))))) (component/stop [this] (some-> this :node elasticsearch/stop-node) (dissoc this :node)) suspendable/Suspendable (suspendable/suspend [this] this) (suspendable/resume [this that] (if (= (-> this :config :elasticsearch-server) (-> that :config :elasticsearch-server)) (assoc this :node (:node that)) (do (component/stop that) (component/start this)))))
  24. 24. REPL namespace (ns my-app.repl (:require [reloaded.repl :as repl :refer [system start stop go reset reset-all]] [com.stuartsierra.component :as component] [my-app.elasticsearch :refer [new-embedded-elasticsearch-server]] [my-app.core :as my-app] [my-app.config :as config]))
  25. 25. Manage services (def config (merge config/config {:elasticsearch-server {:settings {"cluster.name" "elasticsearch" "http.enabled" "true" "http.host" "127.0.0.1" "http.port" "9200" "transport.host" "127.0.0.1" "transport.tcp.port" "9300"}}})) (repl/set-init! #(-> (component/system-map :worker-service (my-app/new-worker-service config) :embedded-elasticsearch-server (new-embedded-elasticsearch-server config)) (component/system-using {:worker-service [:embedded-elasticsearch-server]})))
  26. 26. Embeddable Services • datomic • zookeeper • h2 • kafka • elasticsearch • rabbitmq • postgresql • redis • neo4j • and more!
  27. 27. What about embedding?
  28. 28. What about embedding?
  29. 29. What about embedding?
  30. 30. Are we done?
  31. 31. Entropy
  32. 32. What’s wrong with entropy? (str/reverse “entropy”) ;; That was easy!
  33. 33. Where do we get entropy? xkcd
  34. 34. How do we reproduce random? (defn random-seq "Generate a lazy sequence of random numbers based on an intial seed." [seed n] (let [r (java.util.Random. seed)] (repeatedly #(.nextInt r n)))) (take 5 (random-seq 1 10)) ;;-> (2 5 7 1 8) (take 5 (random-seq 1 10)) ;;-> (2 5 7 1 8) (take 5 (random-seq 2 10)) ;;-> (3 6 5 5 9)
  35. 35. Can randomness be a service?
  36. 36. So, can randomness be a service?
  37. 37. Time
  38. 38. What’s wrong with time?
  39. 39. What’s wrong with time?
  40. 40. Can’t we just use Datomic?
  41. 41. How can we model time? (defn application [from-tick to-tick] “Run our important application.” (let [clock (new-clock from-tick to-tick)] ;; ... )) ;; Run from tick 0 to 100 (application 0 100) ;; Run from tick 0 to 200 (application 0 200) ;; Run from tick 200 to 300 (application 200 300)
  42. 42. But how do we design a system?
  43. 43. –Will Wright (he made SimCity) “Reticulating splines.”
  44. 44. So now our system is perfect?
  45. 45. What did we learn? • How to evaluate systems and find leaks • How to plug leaks when you find them • How to make good trade-offs when building hermetic systems
  46. 46. Questions?
  47. 47. Thanks! Will Farrell @wkf

×