This document summarizes a presentation about the Boss database, a petascale database for neuroscience powered by serverless analytics on AWS. It discusses how serverless computing allows running code without managing servers, paying only for compute time used. It then describes the JHU APL/IARPA MiCRONS project which uses AWS serverless services like Lambda, Step Functions, DynamoDB and others to build a database for large-scale neuroscience research that is scalable, flexible and easy to manage.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Ben Snively, AWS Sr Solutions Architect, Data and Analytics
Dean Kleissas, Research Engineer, JHU APL
June 13, 2017
The Boss: A Petascale Database for
Large-Scale Neuroscience Powered
by Serverless Advanced Analytics

2. Agenda
• AWS Introduction (10 mins)
• What is serverless
• Why serverless analytics
• JHU APL/IARPA MiCRONS (35 mins)
• Introduction - Reshaping research
• Why’s it hard/Why the cloud?
• Architecture Deep Dive
• Considerations
• Demonstration

3. Cloud architecture evolution
Virtualized Managed Serverless
Virtualized
Servers
Managed
Platforms
Serverless
Analytics

4. No servers to provision
or manage
Scales with usage
Never pay for idle Availability and fault
tolerance built in
Serverless characteristics

5. Orchestration/Transform
AWS Big Data Services
Amazon EMR
Amazon
EC2
Amazon
S3
Amazon
Redshift
Amazon
DynamoDB
AWS DMS (CDC)
AWS Lambda
Amazon Kinesis
Analytics
Amazon Athena
Amazon
QuickSight
Amazon Aurora
AWS Glue AWS Step
Functions
Amazon Kinesis
Streams
Ingest/ Collect Store Analyze/ Process
Visualization/
Consume
AWS
Snowball
ISV
Connectors
Amazon
Kinesis
Firehose
S3 Transfer
Acceleration
Amazon
Elasticsearch
Service

6. Serverless computing - AWS Lambda
Run code without provisioning or managing servers – pay
only for the compute time you consume.

7. Continuous
Scaling
No Servers to
Manage
Subsecond
Metering
Benefits of AWS Lambda
AWS Lambda handles:
• Operations and
management
• Provisioning and
utilization
• Scaling
• Availability and fault
tolerance
Automatically scales your
application, running code in
response to each trigger
Your code runs in parallel and
processes each trigger
individually, scaling precisely
with the size of the workload
Pricing
• CPU and Network
scaled based on
RAM (128 MB to
1500 MB)
• $0.20 per
1M requests
• Price per 100ms

8. λλ
λ DBMS
λ
λ
λ
λ
λ
λ λ
λ
λ
Queue
Modern
app

10. Benefits of AWS Step Functions
Diagnose and
debug problems
faster
Adapt to change
Easy to connect and
coordinate
distributed components
and microservices to
quickly create apps
Manages the
operations and
infrastructure of
service coordination
to ensure availability
at scale, and
under failure
Productivity Agility Resilience

11. Application Lifecycle in AWS Step Functions
Visualize in the
console
Define in JSON Monitor
executions

12. Amazon
DynamoDB
Fast & Flexible NoSQL Database Service
• NoSQL Database
• Seamless scalability
• Zero admin
• Single digit millisecond latency

13. Amazon DynamoDB
Document or Key-Value Scales to Any WorkloadFully Managed NoSQL
Access Control Event Driven ProgrammingFast and Consistent

14. Dean Kleissas,
Research Engineer, JHU APL

15. JHU APL/IARPA MiCRONS
Introduction - Reshaping
research

16. • Why’s it hard/Why the cloud?

17. Architecture Deep Dive

18. • Serverless Lessons Learned
and Considerations

19. • Demonstration

20. Thank you!