September 19th joint meetup with Serverless Chicago user group at RedShelf - Serverless in AWS. "Go Serverless from your iPad: Building a Data-driven REST API with AWS CodeStar, Lambda, and Cognitect’s Datomic and Vase" - Chris Johnson Bidler, CTO at Centriq Technology, Inc

1. AWS CODESTAR AND SERVERLESS -
BUILD A REST API FROM YOUR IPAD!
AWS CHICAGO SEPTEMBER 19, 2017

2. INTRO
YOUR HUMBLE NARRATOR
▸ Chris Johnson Bidler
▸ CTO at Centriq Technology, Inc.
▸ Previously Transunion, SPS Commerce, McKesson
▸ Lots of Java -> Quite a bit of Devops -> Architecture
▸ AWS Certified Associate Developer, SA, Devops
Engineer
▸ chris@centriqhome.com

3. INTRO
WHAT SHALL WE TALK ABOUT ?
▸ Datomic
▸ Overview of the tool and use cases
▸ Overview of Datomic deployment in AWS
▸ Vase - REST API layer atop Datomic
▸ AWS CodeStar
▸ Overview of what it is and how it works
▸ Pros and cons
▸ Example of a Serverless use case (CodeStar + AWS SAM)

4. DATOMIC
WEIRD NEW TECHNOLOGY I: DATOMIC
▸ https://www.datomic.com
▸ Database of immutable facts rather than mutable records
▸ Each fact is comprised of:
▸ Entity
▸ Attribute
▸ Value
▸ Time

5. DATOMIC
EXAMPLE FACT - JOHN LIKES PIZZA
e: <opaque id ref for “John”>,
a: “likes”,
v: [“Pizza”],
t: 100
EXAMPLE FACT - JOHN LIKES PIZZA AND ICE CREAM
e: <opaque id ref for “John”>,
a: “likes”,
v: [“Pizza”, “Ice Cream”],
t: 191

6. DATOMIC
EXAMPLE FACT - JOHN LIKES MIXING THINGS UP
e: <opaque id ref for “John”>,
a: “likes”,
v: [“Pizza”, “Ice Cream”, “Beer”],
t: 224
EXAMPLE FACT - JOHN HAD A BAD EXPERIENCE
e: <opaque id ref for “John”>,
a: “likes”,
v: [“Pizza”],
t: 301

7. DATOMIC
USEFUL PROPERTIES OF DATOMIC
▸ All incoming events are given a consistent, total order by
the transactor (of which more anon)
▸ Reads can scale arbitrarily - peers read directly from
storage and don’t need to coordinate with anything to pull
reads
▸ Writes create a consistent state that can be replayed,
queried, etc. (“what was every value of the db at the time
this bug occurred”)

8. DATOMIC
QUERYING A DATOMIC DATABASE
▸ Datomic exposes a logic programming query model called
Datalog (modeled on Prolog/Cascalog)
▸ Allows you to ask interesting questions about the db as a
whole
▸ Since queries are against an immutable set of facts, even
queries that consider a large amount of data can be quite
fast

9. DATOMIC
QUERYING A DATOMIC DATABASE
;; find all the Stewart or Stuart first names
(d/q '[:find [?e ...]
:in $ [?name ...]
:where [?e :user/firstName ?name]]
db
["Stewart" "Stuart"])

10. DATOMIC
DATOMIC ON AWS
▸ There can only be one transactor for any
given Datomic db
▸ It is a fungible, ephemeral resource
▸ Datomic provides Cloudformation
templates to stand up and run
transactors in HA/failover mode
▸ The storage is where the data actually lives
▸ You’re free to use e.g. MongoDB or
Cassandra, etc., but why not use
DynamoDB?
▸ Peers are long-running and service client
requests

11. COGNITECT VASE
COGNITECT VASE - DATOMIC DATA AS REST APIS
▸ Writing a layer to expose useful views into a Datomic DB
used to be something of a challenge
▸ Cognitect recently released Vase, a tool for writing data-
driven APIs atop a Datomic database
▸ Runs in Pedestal (analogous to Flask, Express, etc.)
▸ Composes with existing Pedestal projects or can be
deployed standalone

12. COGNITECT VASE
COGNITECT VASE - DATOMIC DATA AS REST APIS
"/user" {:get #vase/query {:name :example-app.v1/user-page
;; All params are required to perform the query
:params [email]
:query [:find ?e
:in $ ?email
:where
[?e :user/email ?email]]}}
"/users-count" {:get #vase/query {:name :example-app.v1/users-count
:params []
:query [:find (count-distinct ?id)
:where [?e :user/id ?id]]}}

13. COGNITECT VASE
COGNITECT VASE - DATOMIC DATA AS REST APIS
▸ Additional behavior can be configured per route:
users-count-as-of/:when" {:get [#vase/intercept {:name example-app.v1/db-as-of
:doc "Takes a timestamp and updates the db for this request to be
at timestamp"
:enter (fn [ctx]
(update-in ctx [:request :db] (constantly (datomic.api/as
(get-in ctx [:request :db])
(clojure.instant/read-instant-date (get-in ctx [:reque
rams :when]))))))}
#vase/query {:name :centriq-bi.v1/users-count-as-of
:params []
:query [:find (count-distinct ?id)
:where [?e :user/id ?id]]}]}

14. AWS CODESTAR
AWS CODESTAR
▸ Goal: Can we get a Heroku-like experience on top of AWS
services?
▸ Of course it is possible to roll something like this
yourself
▸ You’d need source control, CodeBuild to run CI, some
kind of deployment management
▸ It is of course possible to wire up everything yourself
with CodePipeline, CodeBuild, Cloudformation, etc.

15. AWS CODESTAR
OR …AWS CODESTAR

16. AWS CODESTAR

17. AWS CODESTAR
TOOLS INTEGRATION

18. AWS CODESTAR
GITHUB USERNAME IS YOUR SSH KEY ID

19. AWS CODESTAR

20. AWS CODESTAR

21. AWS CODESTAR
CODESTAR PROS AND CONS
▸ Pros
▸ Press button, receive configured project
▸ Lots of options for deployment type, language, tooling
▸ AWS services can be bolted into template.yml

22. AWS CODESTAR
CODESTAR PROS AND CONS
▸ Cons
▸ CodeCommit is not Github
▸ Issues, PR review tools, wiki dearly missed
▸ Only project management is JIRA
▸ This will almost certainly improve

23. PUTTING THE TWO TOGETHER
DATOMIC BACKEND, LAMBDA FRONTEND
▸ Datomic running on DynamoDB with a transactor and a
peer (these ephemeral, managed servers are your only
servers)
▸ Pedestal/Vase
▸ CodeStar Java Spring -> Lambda project
▸ Single-handler-for-/*

24. PUTTING THE TWO TOGETHER
TESTING LOCALLY
▸ Can run Datomic
▸ Against ephemeral in-memory datastore
▸ Against dynamodb-local
▸ Could then ingest a backup from production
▸ Can test Lambda function with Serverless Local