SlideShare a Scribd company logo
1 of 14
Download to read offline
Engineering Genomic Big
Data Analytics at A Global
Scale
© 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
Elio Gerardi
NetApp Cloud Solutions Architect
July 17, 2018
Data analytics driving transformation of genomics industry
© 2018 NetApp, Inc. All rights reserved.
… while enabling
data-driven digital
transformation
MASSIVE DATA GROWTH
100 million to two billion genomes
driving 40 Exabytes of data by 2025*
EFFICIENT RESOURCE USAGE
Budget constraints driving optimization of
infrastructure with the flexibility to freely
scale as business demands
CONSTRAINED PACE OF
INNOVATION
Demand for on-demand
resources at odds with
performance requirements
2
1000GB=1TB, 1000TB=1PT, 1000PT=1EB
* Source: PLOSBiology
▪ Time is money – need to process more
sequences, faster
▪ Need the flexibility of the cloud to scale for
an increasing genomic workload
▪ Need a high-performance
analytic environment
▪ Need to spend less time managing
infrastructure and control growing cost
Challenge of doing genomic sequencing at scale
3 © 2018 NetApp, Inc. All rights reserved. --- NETAPP CONFIDENTIAL ---
Using sequence data to improve health for people around the world.
Current cloud sequencing architecture
▪ Massive file-based data set for analysis
▪ AWS global data center locations enable
global sequencing footprint
▪ Data set supported across roll-your-own
NFS services on Amazon Elastic Compute
Cloud (EC2) and Amazon Elastic Block
Storage (EBS)
▪ Current architecture supports 48,000
parallel sequences
▪ Always looking for more performance
Reduce complexity and increase performance with NetApp Cloud Volumes
4 © 2018 NetApp, Inc. All rights reserved.
Sequencer
Output Data
NFS Servers
(Amazon EC2/Amazon EBS)
Sequencing Codes
NetApp
Cloud Volumes
Cloud-native file
services with fast
performance
Finally run any workload in AWS - even legacy apps – for maximum agility
Multiprotocol support in a multi-tenant service from AWS
Shift expense from CapEx to OpEx
Data protection without application performance impact
Clone Dev/Ops workspaces instantly and space-efficiently for faster releases
Replicate across geographies for global data access and protection
Get workloads to AWS cloud
months or even years sooner
NetApp Cloud Volumes Service overview
Cloud-native file services built to run on AWS
6 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
Availability and durability
▪ High durability (7 nines)
▪ High availability (4 nines)
Scalability
▪ 100 TB per file volume
▪ 100s of shares per subscription – 1,000s of clients per share
▪ Multiple VPCs supported
Rich data management
(add-on services)
▪ NetApp Snapshot copies, restore, and clones
▪ Cross-region and hybrid-cloud backup and replication with NetApp data movers
Security
▪ Data at rest encryption
▪ Secure multitenancy, Active Directory, and (LDAP) integration export policies
▪ File share access can be restricted based on instance or IP range (through export policies)
Performance
▪ Multiple service levels (one per volume) to match application needs
▪ Low latency, consistent performance
Simple, on-demand
file share
▪ NFS v3, v4, and v4.1
▪ SMB 3.0 and 3.1.1
Cloud Volumes Provides Excellent Performance
7 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
Cloud Volumes Cloud Volumes Value
Latency milliseconds Ability to run applications that demand high performance
Per volume
Throughput (1TB)
16MB (Standard)
65MB (Premium)
130MB (Extreme)
Control cost: No need to overprovision capacity to achieve
performance
• Performance numbers based on NetApp test environment with multiple
instances connecting to one session
Consistency
Consistent latency
across capacities
NetApp service level objectives (SLO) provide application user
expectations all the time
Flexibility
Non-disruptive
performance level
change
Flexibility to switch service levels on the fly – at any time –
without requiring a data migration.
▪ File-based dataset resides on AWS-hosted
NetApp Cloud Volumes
▪ Analytics running on Amazon EC2 services;
Amazon EBS; Amazon VPC; MongoDB
▪ Saving time and money, no longer need to
manage custom infrastructure
▪ Complete cloud offering with no versions or software to
manage
▪ Cloud Volumes service offers advanced data
management—built in
High-performance, cloud-based sequence analysis
Reduce complexity and increase performance with NetApp Cloud Volumes
8 © 2018 NetApp, Inc. All rights reserved.
Cloud Volumes
Sequencer
Output Data
Sequencing Codes
▪ 20X more sequencing capacity with the same
dataset compared to previous capability
▪ Able to run a large calculation in 45 minutes​
with NetApp Cloud Volumes in AWS
▪ 3x faster than previous capabilities for​ three NFS
servers and15 Amazon EBS volumes​
▪ Easy onboarding: 30TB and 2M files in place
in less than 2 days​
Results with NetApp Cloud Volumes
9 © 2018 NetApp, Inc. All rights reserved. --- NETAPP CONFIDENTIAL ---
Increasing sequencing capacity for more rapid genome results
Cloud Volumes Demo
10 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
NetApp Cloud Data Services
Inspire innovation in the cloud with industry-leading data services stack
© 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
ONTAP Data Management
Backup, Protection, Security, and so on
NetApp Cloud Controls
NetApp Cloud Insights
Cloud
Storage
Software-as-a-
Service (SaaS)
Applications
On-
Premises
11
25 years of on-
premises and
private cloud
storage solutions
• Modernize with All
Flash Arrays
• Build clouds with
hyper-converged
infrastructure (HCI)
NETAPP
DATA
FABRIC
12 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
Disaster
Recovery
Backup and
Archive
Analytics and
Data Lakes
DevOps
App Dev
Production
Workloads
File
Services
SaaS
Applications
Data Services that Accelerate All Your Cloud Workloads
Summary
▪ Sign up and try Cloud Volumes for AWS today
https://netapp.com/aws
▪ Learn more at www.netapp.com/cloudvolumes
NetApp Cloud Volumes delivering
fast performance for genomic
sequencing…
…While saving cost and
manpower in the process
Cloud-native file services with
extreme performance
14
Thank You
© 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —

More Related Content

What's hot

Builders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LCBuilders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LCAmazon Web Services LATAM
 
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...Amazon Web Services
 
Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...
Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...
Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...Amazon Web Services
 
Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...
Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...
Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...Amazon Web Services
 
DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS
DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECSDEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS
DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECSAmazon Web Services
 
Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...
Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...
Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...Amazon Web Services
 
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Amazon Web Services
 
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018Amazon Web Services
 
AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...
AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...
AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...AWS Germany
 
Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...
Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...
Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...Amazon Web Services
 
EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...
EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...
EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...Amazon Web Services
 
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018Amazon Web Services
 
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...Amazon Web Services
 
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...Chris Fregly
 
AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...
AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...
AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...AWS Germany
 
High Performance Computing on AWS
High Performance Computing on AWSHigh Performance Computing on AWS
High Performance Computing on AWSAmazon Web Services
 
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...Amazon Web Services
 

What's hot (20)

Builders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LCBuilders' Day - Building Data Lakes for Analytics On AWS LC
Builders' Day - Building Data Lakes for Analytics On AWS LC
 
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
Deep Dive on Amazon Elastic Block Storage (Amazon EBS) (STG310-R1) - AWS re:I...
 
Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...
Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...
Metrics-Driven Performance Tuning for AWS Glue ETL Jobs (ANT331) - AWS re:Inv...
 
Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...
Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...
Back Up and Manage On-Premises and Cloud-Native Workloads with Rubrik on AWS ...
 
Builders' Day - What's New on EC2
Builders' Day - What's New on EC2Builders' Day - What's New on EC2
Builders' Day - What's New on EC2
 
DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS
DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECSDEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS
DEM19 Advanced Auto Scaling and Deployment Tools for Kubernetes and ECS
 
Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...
Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...
Tape Is a Four Letter Word: Back Up to the Cloud in Under an Hour (STG201) - ...
 
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP371) - AWS re:Invent 2018
 
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
Optimizing Amazon EBS for Performance (CMP317-R2) - AWS re:Invent 2018
 
AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...
AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...
AWS Summit Berlin 2013 - Optimizing your AWS applications and usage to reduce...
 
AWS re:Invent Recap
AWS re:Invent RecapAWS re:Invent Recap
AWS re:Invent Recap
 
Amazon Aurora 深度探討
Amazon Aurora 深度探討Amazon Aurora 深度探討
Amazon Aurora 深度探討
 
Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...
Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...
Moving Out of the Data Center to Reach More Customer Targets (IOT222-S) - AWS...
 
EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...
EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...
EUT302_Data Ingestion at Seismic Scale Best Practices for Processing Petabyte...
 
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
Ask Me Anything about Amazon Aurora (DAT369-R1) - AWS re:Invent 2018
 
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
[NEW LAUNCH!] AWS IoT Device Tester: Enable Your Edge Devices for AWS IoT (IO...
 
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
PipelineAI + AWS SageMaker + Distributed TensorFlow + AI Model Training and S...
 
AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...
AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...
AWS Summit Berlin 2013 - Euroforum - Moving an Entire Physical Data Center in...
 
High Performance Computing on AWS
High Performance Computing on AWSHigh Performance Computing on AWS
High Performance Computing on AWS
 
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
Accelerate Your Analytic Queries with Amazon Aurora Parallel Query (DAT362) -...
 

Similar to Engineering Genomic Big Data Analytics at A Global Scale

Containers and Kubernetes
Containers and KubernetesContainers and Kubernetes
Containers and KubernetesAltoros
 
Lessons learned processing 70 billion data points a day using the hybrid cloud
Lessons learned processing 70 billion data points a day using the hybrid cloudLessons learned processing 70 billion data points a day using the hybrid cloud
Lessons learned processing 70 billion data points a day using the hybrid cloudDataWorks Summit
 
NetApp Cloud Data Services & AWS Empower Your Cloud Champions
NetApp Cloud Data Services & AWS Empower Your Cloud ChampionsNetApp Cloud Data Services & AWS Empower Your Cloud Champions
NetApp Cloud Data Services & AWS Empower Your Cloud ChampionsAmazon Web Services
 
NetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the Cloud
NetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the CloudNetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the Cloud
NetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the CloudVeritas Technologies LLC
 
NetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloud
NetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloudNetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloud
NetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloudVeritas Technologies LLC
 
Slides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out InfrastructureSlides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out InfrastructureNetApp
 
NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...
NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...
NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...NetApp
 
IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015Doug O'Flaherty
 
OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017
OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017
OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017Cloud Native Day Tel Aviv
 
Denver Big Data Analytics Day
Denver Big Data Analytics DayDenver Big Data Analytics Day
Denver Big Data Analytics DayZivaro Inc
 
NetApp IT Data Center Strategies to Enable Digital Transformation
NetApp IT Data Center Strategies to Enable Digital TransformationNetApp IT Data Center Strategies to Enable Digital Transformation
NetApp IT Data Center Strategies to Enable Digital TransformationNetApp
 
Macroview Netapp Overview
Macroview Netapp OverviewMacroview Netapp Overview
Macroview Netapp OverviewAlex Tsui
 
Scale Your Load Balancer from 0 to 1 million TPS on Azure
Scale Your Load Balancer from 0 to 1 million TPS on AzureScale Your Load Balancer from 0 to 1 million TPS on Azure
Scale Your Load Balancer from 0 to 1 million TPS on AzureAvi Networks
 
Enabling big data & AI workloads on the object store at DBS
Enabling big data & AI workloads on the object store at DBS Enabling big data & AI workloads on the object store at DBS
Enabling big data & AI workloads on the object store at DBS Alluxio, Inc.
 
Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...NetAppUK
 
Solving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalSolving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalAvere Systems
 
AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...
AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...
AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...Amazon Web Services
 

Similar to Engineering Genomic Big Data Analytics at A Global Scale (20)

Containers and Kubernetes
Containers and KubernetesContainers and Kubernetes
Containers and Kubernetes
 
Lessons learned processing 70 billion data points a day using the hybrid cloud
Lessons learned processing 70 billion data points a day using the hybrid cloudLessons learned processing 70 billion data points a day using the hybrid cloud
Lessons learned processing 70 billion data points a day using the hybrid cloud
 
NetApp Cloud Data Services & AWS Empower Your Cloud Champions
NetApp Cloud Data Services & AWS Empower Your Cloud ChampionsNetApp Cloud Data Services & AWS Empower Your Cloud Champions
NetApp Cloud Data Services & AWS Empower Your Cloud Champions
 
NetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the Cloud
NetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the CloudNetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the Cloud
NetBackup CloudCatalyst: Efficient, Cost-Effective Deduplication to the Cloud
 
NetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloud
NetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloudNetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloud
NetBackup CloudCatalyst – efficient, cost-effective deduplication to the cloud
 
Slides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out InfrastructureSlides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
Slides: Start Small, Grow Big with a Unified Scale-Out Infrastructure
 
OCI Overview
OCI OverviewOCI Overview
OCI Overview
 
NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...
NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...
NetApp IT Efficiencies Gained with Flash, NetApp ONTAP, OnCommand Insight, Al...
 
IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015IBM Spectrum Scale Overview november 2015
IBM Spectrum Scale Overview november 2015
 
OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017
OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017
OpenStack and NetApp - Chen Reuven - OpenStack Day Israel 2017
 
Denver Big Data Analytics Day
Denver Big Data Analytics DayDenver Big Data Analytics Day
Denver Big Data Analytics Day
 
NetApp IT Data Center Strategies to Enable Digital Transformation
NetApp IT Data Center Strategies to Enable Digital TransformationNetApp IT Data Center Strategies to Enable Digital Transformation
NetApp IT Data Center Strategies to Enable Digital Transformation
 
AWSome Insider's View of NetBackup 8.1
AWSome Insider's View of NetBackup 8.1AWSome Insider's View of NetBackup 8.1
AWSome Insider's View of NetBackup 8.1
 
Macroview Netapp Overview
Macroview Netapp OverviewMacroview Netapp Overview
Macroview Netapp Overview
 
Scale Your Load Balancer from 0 to 1 million TPS on Azure
Scale Your Load Balancer from 0 to 1 million TPS on AzureScale Your Load Balancer from 0 to 1 million TPS on Azure
Scale Your Load Balancer from 0 to 1 million TPS on Azure
 
Enabling big data & AI workloads on the object store at DBS
Enabling big data & AI workloads on the object store at DBS Enabling big data & AI workloads on the object store at DBS
Enabling big data & AI workloads on the object store at DBS
 
Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...Converged Everything, Converged Infrastructure delivering business value and ...
Converged Everything, Converged Infrastructure delivering business value and ...
 
Solving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalSolving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute final
 
AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...
AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...
AWS re:Invent 2016: Optimizing workloads in SAP HANA with Amazon EC2 X1 Insta...
 
AWS Storage State of the Union
AWS Storage State of the UnionAWS Storage State of the Union
AWS Storage State of the Union
 

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Amazon Web Services
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Amazon Web Services
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateAmazon Web Services
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSAmazon Web Services
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Amazon Web Services
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Amazon Web Services
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...Amazon Web Services
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsAmazon Web Services
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareAmazon Web Services
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSAmazon Web Services
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAmazon Web Services
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareAmazon Web Services
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWSAmazon Web Services
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckAmazon Web Services
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without serversAmazon Web Services
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceAmazon Web Services
 

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...
 
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
Big Data per le Startup: come creare applicazioni Big Data in modalità Server...
 
Esegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS FargateEsegui pod serverless con Amazon EKS e AWS Fargate
Esegui pod serverless con Amazon EKS e AWS Fargate
 
Costruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWSCostruire Applicazioni Moderne con AWS
Costruire Applicazioni Moderne con AWS
 
Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot Come spendere fino al 90% in meno con i container e le istanze spot
Come spendere fino al 90% in meno con i container e le istanze spot
 
Open banking as a service
Open banking as a serviceOpen banking as a service
Open banking as a service
 
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...
 
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...OpsWorks Configuration Management: automatizza la gestione e i deployment del...
OpsWorks Configuration Management: automatizza la gestione e i deployment del...
 
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows WorkloadsMicrosoft Active Directory su AWS per supportare i tuoi Windows Workloads
Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads
 
Computer Vision con AWS
Computer Vision con AWSComputer Vision con AWS
Computer Vision con AWS
 
Database Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatareDatabase Oracle e VMware Cloud on AWS i miti da sfatare
Database Oracle e VMware Cloud on AWS i miti da sfatare
 
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJSCrea la tua prima serverless ledger-based app con QLDB e NodeJS
Crea la tua prima serverless ledger-based app con QLDB e NodeJS
 
API moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e webAPI moderne real-time per applicazioni mobili e web
API moderne real-time per applicazioni mobili e web
 
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatareDatabase Oracle e VMware Cloud™ on AWS: i miti da sfatare
Database Oracle e VMware Cloud™ on AWS: i miti da sfatare
 
Tools for building your MVP on AWS
Tools for building your MVP on AWSTools for building your MVP on AWS
Tools for building your MVP on AWS
 
How to Build a Winning Pitch Deck
How to Build a Winning Pitch DeckHow to Build a Winning Pitch Deck
How to Build a Winning Pitch Deck
 
Building a web application without servers
Building a web application without serversBuilding a web application without servers
Building a web application without servers
 
Fundraising Essentials
Fundraising EssentialsFundraising Essentials
Fundraising Essentials
 
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
AWS_HK_StartupDay_Building Interactive websites while automating for efficien...
 
Introduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container ServiceIntroduzione a Amazon Elastic Container Service
Introduzione a Amazon Elastic Container Service
 

Engineering Genomic Big Data Analytics at A Global Scale

  • 1. Engineering Genomic Big Data Analytics at A Global Scale © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL — Elio Gerardi NetApp Cloud Solutions Architect July 17, 2018
  • 2. Data analytics driving transformation of genomics industry © 2018 NetApp, Inc. All rights reserved. … while enabling data-driven digital transformation MASSIVE DATA GROWTH 100 million to two billion genomes driving 40 Exabytes of data by 2025* EFFICIENT RESOURCE USAGE Budget constraints driving optimization of infrastructure with the flexibility to freely scale as business demands CONSTRAINED PACE OF INNOVATION Demand for on-demand resources at odds with performance requirements 2 1000GB=1TB, 1000TB=1PT, 1000PT=1EB * Source: PLOSBiology
  • 3. ▪ Time is money – need to process more sequences, faster ▪ Need the flexibility of the cloud to scale for an increasing genomic workload ▪ Need a high-performance analytic environment ▪ Need to spend less time managing infrastructure and control growing cost Challenge of doing genomic sequencing at scale 3 © 2018 NetApp, Inc. All rights reserved. --- NETAPP CONFIDENTIAL --- Using sequence data to improve health for people around the world.
  • 4. Current cloud sequencing architecture ▪ Massive file-based data set for analysis ▪ AWS global data center locations enable global sequencing footprint ▪ Data set supported across roll-your-own NFS services on Amazon Elastic Compute Cloud (EC2) and Amazon Elastic Block Storage (EBS) ▪ Current architecture supports 48,000 parallel sequences ▪ Always looking for more performance Reduce complexity and increase performance with NetApp Cloud Volumes 4 © 2018 NetApp, Inc. All rights reserved. Sequencer Output Data NFS Servers (Amazon EC2/Amazon EBS) Sequencing Codes
  • 5. NetApp Cloud Volumes Cloud-native file services with fast performance Finally run any workload in AWS - even legacy apps – for maximum agility Multiprotocol support in a multi-tenant service from AWS Shift expense from CapEx to OpEx Data protection without application performance impact Clone Dev/Ops workspaces instantly and space-efficiently for faster releases Replicate across geographies for global data access and protection Get workloads to AWS cloud months or even years sooner
  • 6. NetApp Cloud Volumes Service overview Cloud-native file services built to run on AWS 6 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL — Availability and durability ▪ High durability (7 nines) ▪ High availability (4 nines) Scalability ▪ 100 TB per file volume ▪ 100s of shares per subscription – 1,000s of clients per share ▪ Multiple VPCs supported Rich data management (add-on services) ▪ NetApp Snapshot copies, restore, and clones ▪ Cross-region and hybrid-cloud backup and replication with NetApp data movers Security ▪ Data at rest encryption ▪ Secure multitenancy, Active Directory, and (LDAP) integration export policies ▪ File share access can be restricted based on instance or IP range (through export policies) Performance ▪ Multiple service levels (one per volume) to match application needs ▪ Low latency, consistent performance Simple, on-demand file share ▪ NFS v3, v4, and v4.1 ▪ SMB 3.0 and 3.1.1
  • 7. Cloud Volumes Provides Excellent Performance 7 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL — Cloud Volumes Cloud Volumes Value Latency milliseconds Ability to run applications that demand high performance Per volume Throughput (1TB) 16MB (Standard) 65MB (Premium) 130MB (Extreme) Control cost: No need to overprovision capacity to achieve performance • Performance numbers based on NetApp test environment with multiple instances connecting to one session Consistency Consistent latency across capacities NetApp service level objectives (SLO) provide application user expectations all the time Flexibility Non-disruptive performance level change Flexibility to switch service levels on the fly – at any time – without requiring a data migration.
  • 8. ▪ File-based dataset resides on AWS-hosted NetApp Cloud Volumes ▪ Analytics running on Amazon EC2 services; Amazon EBS; Amazon VPC; MongoDB ▪ Saving time and money, no longer need to manage custom infrastructure ▪ Complete cloud offering with no versions or software to manage ▪ Cloud Volumes service offers advanced data management—built in High-performance, cloud-based sequence analysis Reduce complexity and increase performance with NetApp Cloud Volumes 8 © 2018 NetApp, Inc. All rights reserved. Cloud Volumes Sequencer Output Data Sequencing Codes
  • 9. ▪ 20X more sequencing capacity with the same dataset compared to previous capability ▪ Able to run a large calculation in 45 minutes​ with NetApp Cloud Volumes in AWS ▪ 3x faster than previous capabilities for​ three NFS servers and15 Amazon EBS volumes​ ▪ Easy onboarding: 30TB and 2M files in place in less than 2 days​ Results with NetApp Cloud Volumes 9 © 2018 NetApp, Inc. All rights reserved. --- NETAPP CONFIDENTIAL --- Increasing sequencing capacity for more rapid genome results
  • 10. Cloud Volumes Demo 10 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —
  • 11. NetApp Cloud Data Services Inspire innovation in the cloud with industry-leading data services stack © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL — ONTAP Data Management Backup, Protection, Security, and so on NetApp Cloud Controls NetApp Cloud Insights Cloud Storage Software-as-a- Service (SaaS) Applications On- Premises 11 25 years of on- premises and private cloud storage solutions • Modernize with All Flash Arrays • Build clouds with hyper-converged infrastructure (HCI) NETAPP DATA FABRIC
  • 12. 12 © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL — Disaster Recovery Backup and Archive Analytics and Data Lakes DevOps App Dev Production Workloads File Services SaaS Applications Data Services that Accelerate All Your Cloud Workloads
  • 13. Summary ▪ Sign up and try Cloud Volumes for AWS today https://netapp.com/aws ▪ Learn more at www.netapp.com/cloudvolumes NetApp Cloud Volumes delivering fast performance for genomic sequencing… …While saving cost and manpower in the process Cloud-native file services with extreme performance
  • 14. 14 Thank You © 2018 NetApp, Inc. All rights reserved. — NETAPP CONFIDENTIAL —