A technical presentation of the EMBL-EBI Skills Mapper based on Neo4j and AWS Lambda.

1. An experiment with AWS Lambda
EBI Skills Mapper
Daniel Vaughan
@DanielVaughan
www.ebi.ac.uk

2. The Problem
• Map the skills and interests of our developers
• Easy to interact with and no registration
• Enable finding patterns in the data

3. The Graph Database
Amazon EC2
t2.micro with
8GB SSD
+ +

4. AWS Lambda
Lambda
• Language runtime vs. container or VM
• Functions are unit of deployment
• Java 8, Node.js or Python 2.7
• Linux containers + packing algorithm
• Scaling handling for you
• Stateless – permanent storage is
elsewhere
• Pay per request

5. Getting email in
SES
(Simple Email Service)
Lambda SNS
(Simple Notification Service)
 
To: i.am.using@gcscc.site
Subject: nodejs aws-lambda


6. Processing email
SNS
(Simple Notification Service)
Lambda Graph Database
 
Lambda

DynamoDB


7. Gathering more information
SNS
(Simple Notification Service)

Graph Database

Lambda
Stack Overflow API
 
 
EBI Intranet

8. Getting information out
API Gateway Lambda Graph Database





9. Serving the UI
Route 53
(gcscc.site)
Webpages on S3 API Gateway





10. The Result

11. The Graph

12. What I liked
• Low costs
• Very nearly free and will be even with thousands of users
• Clean architecture
• Forced to have independent functions that do one thing
• Hugely powerful and flexible toolkit
• Feels productive as a developer
• ~1 feature per hour (40 hours total)

13. What I didn’t like
• Tools are still emerging
• node-lambda used for local execution, testing and
deployment
• Can see managing many functions and will get tricky
• Setting up plumbing like SNS events and permissions not
automated (for me)
• A slight delay in cold start of Node.js functions but better
than Java
• Error handling needs a bit of thought

14. Future plans
• Cloud Formation for Lambda
• API Caching (very small $ cost)
• Tuning (turn up the dial)
• Editing web frontend with login and API authentication
• I am optimistic Lambda will become production ready

15. Thanks to
Jenny Cham
UX, prototyping and
testing
Ken Hawkins
Visual design and
styling

16. Try it out
To: i.learned.from@gcscc.site
Subject: dvaughan@ebi.ac.uk aws-lambda