Harness the Power of Hybrid Cloud with AWS and Avere
•Download as PPTX, PDF•
1 like•839 views
Avere offers an agile, enterprise hybrid cloud platform that lets you leverage cloud compute, storage, or both and keep data where it makes the most sense—without lofty storage costs, latency, and security concerns. In this webinar, you’ll learn how organizations with complex IT environments like a leading Cancer Center harness hybrid cloud environments to simplify data storage and management, and improve both security and performance. Find out how Avere and AWS hybrid cloud solutions can eliminate roadblocks and open opportunities to more efficiently meet the needs of your users and customers.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Harness the Power of Hybrid Cloud with AWS and Avere
Similar to Harness the Power of Hybrid Cloud with AWS and Avere (20)
More from Amazon Web Services
More from Amazon Web Services (20)
Recently uploaded
Recently uploaded (20)
Harness the Power of Hybrid Cloud with AWS and Avere
- 1. How a Leading Cancer Research Center is Building a World-Class Hybrid Infrastructure
- 2. Ian Scofield, Partner Solutions Architect, AWS Jeff Tabor, Sr. Director of Product Marketing, Avere Systems Dirk Petersen, Scientific Computing Director, Cancer Research Center Today’s Presenters
- 3. Learning Objectives Understand the benefits of Avere + AWS hybrid cloud solutions. Learn why a leading cancer research center decided to embrace the cloud as a long- term strategy for IT infrastructure and how they are building a hybrid cloud architecture with Avere and AWS. Understand best practices for and benefits of building and leveraging hybrid cloud infrastructures to simplify data management, increase efficiency, and scale performance with high availability.
- 5. Why choose AWS for storage Compelling Economics Pay as you go No upfront investment No commitment No risky capacity planning No need to provision for redundancy or overhead Reduce risk Durable and Secure Avoid risks of physical media handling Easy to Use Self service administration SDKs for simple integration Speed, Agility, Scale Reduce time to market Focus on your business, not your infrastructure
- 6. Amazon EFS File Amazon EBS Amazon EC2 Instance Store Block Amazon S3/SIA Amazon Glacier Object Data Transfer AWS Direct Connect AWS Snowball ISV Connectors Amazon Kinesis Firehose S3 Transfer Acceleration Storage Gateway Storage is a platform: AWS Storage Maturity
- 7. Amazon Simple Storage Service (Amazon S3) • Highly scalable object storage • Store and retrieve data from anywhere on the web • Automatically replicated across multiple AZs • Highly durable (11 9’s of durability) • Bucket policies and ACLs • Server and Client side encryption
- 8. Choice of storage classes on Amazon S3 Active data Archive dataInfrequently accessed data Standard Standard - Infrequent Access Amazon Glacier
- 9. Amazon Elastic Compute Cloud (Amazon EC2) • Virtual servers in the cloud • Procure new servers in minutes • Elasticity • Support various distributions of Linux as well as Microsoft Windows Server (2003 R2 - 2016) • Complete control of your host operating system
- 10. • On-premises storage and compute can have the scale and durability of the cloud, without disrupting existing investments or systems. • Move workloads between on-premises and the cloud as computing and cost needs change. • Extend your on-premises network configuration into the AWS Cloud so that AWS resources operate as if they are part of your existing corporate network AWS Hybrid Architecture Capabilities
- 11. Key AWS Certifications and Assurance Programs
- 12. Avere Systems
- 13. Avere Hybrid Cloud NAS Cloud Compute On-Prem Compute Cloud Storage On-Prem Storage NAS Object Bucket 1 Bucket 2 Bucket n Virtual Compute Farm Virtual FXT File Storage for Private Object NAS Optimization Cloud NAS Physical FXT
- 14. Complete NAS Functionality • Powerful GUI and storage analytics • NFS, SMB, multi-protocol data access • Snapshots • Global namespace • Data migration • Mirroring • Compression • Encryption Avere GUI showing client-side NFS access to object storage
- 15. Avere FXT Product Line Hardware r3.2xlarge r3.8xlarge FXT 5200 FXT 5400 FXT 5600 DRAM (GB) 61 244 128 256 384 SSD (TB) 1TB or 4TB EBS General Purpose (SSD) - 4.8 9.6 SAS (TB) - - 7.2 - - Network High 10GbE 4x10GbE, 4x1GbE Performance 5600 5400 5200r3.2xlarge AWS r3.8xlarge AWS Virtual FXT Physical FXT
- 16. Avere FXT 5600: Fastest and Most Efficient NAS High performance • 96GB/s max • 9.6M Getattr IOPS max • 2.4M Read IOPS max All-Flash tier • High density • 1RU per FXT Highly efficient • NAS and public/private object storage support • Better economics FXT Nodes 3 4 5 6 12 24 Throughput (GB/s) 12 16 20 24 48 96 Getattr Performance (IOPS, in 1000s) 1200 1600 2000 2400 4800 9600 Read Performance (IOPS, in 1000s) 300 400 500 600 1200 2400 SSD, in FXT cluster (TB) 29 38 48 58 115 230
- 17. The Avere Hybrid Cloud NAS Solution + AWS Avere FXT and vFXT high-performance file system products enhance your current environment with easy access to AWS • Add storage capacity to existing NAS architectures with Amazon S3 without sacrificing performance • Add performance to file-based applications with Amazon EC2 compute instances and SSDs • Build a cloud strategy that makes sense for your business
- 18. Hybrid Cloud Use Case: Cancer Research Center
- 19. Cancer Research Center Overview • 3000 employees • 200+ independent research groups • Gov’t funded • More than just cancer research…HIV, etc. • Roadmap: Move to the public cloud in 5 years Dirk Petersen – Scientific Computing Director
- 20. Compute Challenges • Workloads are bursty. – Gov’t grants – Publication cycle – Annual meetings • 200+ programs operating in parallel, sharing CPUs. • 60% CPU core utilization on average. • Target: Do NOT cross pink line (80% util.). • Pending: Shown in yellow. Yellow jobs are waiting should go to cloud
- 21. • Elasticity in storage required. – Needed for “ad hoc” growth (~50TB) • Legacy apps require POSIX file system with fast consistency. – NFS for compute farm and servers – SMB/CIFS for scientist workstations • Cloud built for global scale and provides eventual consistency. • POSIX needed until next-gen cloud apps (and scientists) take over. Storage Challenges
- 22. “Focus on Science, not Information Technology (IT)” • Move to the AWS in 5 years: – When POSIX file system required, use Avere. – In cloud, security and compliance come for free. – In Seattle job market, competition for IT talent is high. • Use phased approach: – Budgets are limited. – Need to get value out of existing capital. – CAPEX is more complex than OPEX.
- 23. PHASE – Primary Need • Phase 1 – Lower cost NAS on-premises. • Phase 2 – Cloud compute CPUs added when needed. • Phase 3 – Cloud storage for POSIX file system. • Phase 4 – Compute and storage 100% in the cloud.
- 24. Phase 1 – File Storage for Private Object Challenges • Scale-out NAS is expensive • Need object storage for scale • Need POSIX for legacy apps • Enable AWS S3 use in future Solution • Avere provides NFSv3 and SMB for POSIX file system • Private object storage provides scaling and resiliency • TCO is significantly lower than scale-out NAS Cloud Compute On-Prem Compute Cloud Storage On-Prem Storage NAS Object Bucket 1 Bucket 2 Bucket n Virtual Compute Farm Virtual FXT File Storage for Private Object Physical FXT
- 25. Phase 2 – Cloud Bursting Challenges • Owning compute on premises is inefficient • ”Too much waiting” during busy times • “Wasteful” during slow times Solution • Cloud bursting with Avere vFXT and AWS EC2 • Pay only for what you use • Go as wide (e.g. 100,000 CPUs) as needed at the time • No waiting, no waste Cloud Compute On-Prem Compute Cloud Storage On-Prem Storage NAS Object Bucket 1 Bucket 2 Bucket n Virtual Compute Farm Virtual FXT File Storage for Private Object Physical FXT
- 26. Phase 3 – Cloud Storage Gateway Challenges • Storage elasticity required • Research teams request 10-50TB on ”ad hoc” basis • Need POSIX for legacy apps Solution • Cloud storage gateway with Avere and AWS S3 • Avere provides: performance, low latency, NFSv3, SMB • AWS S3 provides: unlimited storage scaling, low TCO, global resiliency Cloud Compute On-Prem Compute Cloud Storage On-Prem Storage NAS Object Bucket 1 Bucket 2 Bucket n Virtual Compute Farm Virtual FXT File Storage for Private Object Physical FXT
- 27. Phase 4 – Cloud NAS Challenges • Want “utility” model with no on- premises resources • Deploying and fixing hardware is difficult, requires staff • Building data centers is expensive Solution • Cloud NAS with Avere vFXT and AWS EC2 and S3 • Pay only for what is used • Software-only maintenance • No touching of hardware Cloud Compute Cloud Storage Bucket 1 Bucket 2 Bucket n Virtual Compute Farm Virtual FXT Cloud NAS On-Prem Compute On-Prem Storage NAS Object File Storage for Private Object Physical FXT Focus on science, not IT
- 29. • Work with cloud experts – Avere and AWS have “been there, done that” – Avere professional services • Migrating on-prem apps to cloud has surprising challenges – Slurm Workload Manager not documented – If hybrid not necessary, use CfnCluster • Avere Hybrid Cloud NAS was the right decision for us – Provides near enterprise performance and access to AWS – Meets the complex compute and storage demands of cancer research – Using rsync with many small files is still a challenge, but improving – VPN / VPC pretty low (50% of 1G), Avere caching adding more value Lessons Learned
- 30. Q & A
Editor's Notes
- AWS gives you reliable, durable backup storage without the up-front capital expenditures and complex capacity-planning burden of on-premises storage. Amazon storage services remove the need for complex and time-consuming capacity planning, ongoing negotiations with multiple hardware and software vendors, specialized training, and maintenance of offsite facilities or transportation of storage media to third party offsite locations.
- AWS offers a variety of low cost storage choices, many of which have pay-as-you-go options for added flexibility. Popular storage choices include the Amazon Simple Storage Service (S3), Elastic Block Store (EBS), Amazon Elastic Compute Cloud (Amazon EC2) and Amazon CloudFront. The suite of transfer services that support customers in their migrations means more choice. Large batches, incremental changes, constant streams or seamless integration are all part of the storage offering. Today we’re going to talk about two of the newest ways to do cloud data migration, Snowball and S3 Transfer Acceleration. Note to presenters: Disk Transfer service is not EOL but has been deprecated out of the transfer services story in favor of Snowball. Snowball has already surpassed the amount of data imported over the lifetime of the disk transfer service. EFS is in preview and due before the end of the year
- Files are stored as objects and organized into high-level folders called buckets
- Files are stored as objects and organized into high-level folders called buckets
- Instance sizes and types: Pricing options: on-demand, reserved, spot, dedicated
- We are also certified and accredited by a wide range of regulators and industry bodies. Here is a list of key bodies that have either certified us, or we have a workbook of guidance showing you how to validate an AWS environment against these standards. every customer benefits from the tough scrutiny of other AWS customers! Top Row (left to right) ISO 27001 Information Security Management ISO 9001 Quality Management Systems Requirements American Institute of Certified Professional Accounts (SOC 1, SOC 2, SOC 3 reports) Payment Card Industry Data Security Standard (PCI-DSS) Federal Information Security Management Cloud Security Alliance Middle Row: TUV Trust IT – independent certification body for the German Federal Office for Information Security (BSI) IT Baseline protection methodology (IT Grundschutz) UK G-Cloud Digital Marketplace HIPAA (Health Information Portability and Accountability Act) Federal Information Processing Standards 140-2 Americans with Disabilities Act Section 508 Motion Pictures of America Association Bottom Row: US International Traffic in Arms Regulations Department of Defense Cloud Security Model Criminal Justice Information Systems (CJIS) Security Policy Federal Risk Authorization Management Program (FedRAMP) Australian Information Risk Assurance Program US Department of Education (FERPA) <FOR MORE IN DEPTH QUESTIONS REFER THE CUSTOMER TO http://aws.amazon.com/compliance FOR MORE DETAILS>
- Let me tell you more about what we mean by the Avere Hybrid Cloud. This slide has a diagram that we like to use. It contains all the resources that are available to you when designing your IT/NAS environment. The bottom shows all the on-prem resources. On the left you have your on-prem compute including users workstations and a compute farm and on the right you have on-prem storage including NAS and object storage. The top shows cloud resources from a service provided like Amazon, Google, or Microsoft. On the left you have the compute cloud where you can rent servers to run your apps and on the right you have the storage cloud where you can rent capacity to store your data. Avere products get deployed in 5 different use cases in this hybrid cloud infrastructure. Starting at the bottom, with our physical FXT we have the NAS Optimization use case. NAS Optimization is where we started as a company. In the NAS Optimization use case, we place an All-Flash performance tier infront of your existing disk-based NAS systems. The FXT clusters use caching to automatically accelerate active date, cluster to scale performance (add more CPUs and DRAM) and capacity (add more SSDs), and hide the latency to the core storage which sometimes is deployed over a WAN. Many of our customers are using Avere in this way to accelerate the performance of rendering, genomic analysis, financial simulations, software tools and binaries respository, and more. (CLICK) In the File Storage for Private Object use case, we have found many customers who are looking to move from NAS to private object stoage. They like private object for it’s efficiency, simplicity, and resiliency. However, they don’t like it’s perfromance or object-based API interface. In this use case, the FXT cluster accelerates the perfromance of the private object storage in the same way as in the NAS optimization use case. But, in addition, the FXT cluster provides familar NAS protocols with translation to object APIs on the storage side, so customerrs can use object storage without re-writing their applications or changing their data access methods. (CLICK) The Cloud Storage Gateway use case is similar to the File Storage for Private Object use case with the added benefit that customers can begin to build fewer data centers and move their data to the cloud. Latency is one of the key challenges that needs to be overcome in this use case and that’s what the physical FXT cluster provides. On access, data is cached on premises on the FXT cluster so all subsequent accesses occur at low latency. An FXT cluster can have as much as 480TB of total caching capacity, so large volumes of data can be stored on premises to avoid the latency of the cloud. (CLICK) The Cloud Bursting use case enables customers to take advantage of the near infinite supply of compute nodes available in the clouds offered by AWS, GCP, and MS Azure. At peak times, customers can add from 100s to many 10s of thousands of compute cores to run rendering, genomics, and financial batch jobs in the compute cloud. The virtual FXT is software-only version of our product that runs on high-perfromance servers and SSDs available in the compute cloud and automatically caches active data in the cloud from on-premises NAS (or object) systems and delivers this active data at low latency to the application. (CLICK) Last, the Cloud NAS use case is for when everything is moved to the cloud, both the app and the storage. This use case allows you to build fewer data centers and fully embrace the cloud. The Avere Hybrid Cloud allows you to get there in a simple, stepwise fashion, often going through either the Cloud Bursting or Cloud Gateway use cases first. The Avere Hybrid Cloud allows you to move to the cloud at your own pace and under the terms you define.
- NAS Simplicity Avere C2N delivers NAS simplicity and performance with a purpose-built file system designed for object storage. Each C2N system has an internal all-Flash performance tier built from proven Avere FXT Edge filer technology that scales to keep pace with growing performance demands. C2N provides a rich set of NAS features that make it an easy fit in existing environments. NFS, SMB, and multi-protocol support enable C2N to work seamlessly in Linux, Unix, Windows, and Mac environments with existing file-based applications. Protect against everything from accidental deletion to site-wide failures with C2N snapshots and mirroring. Compression and encryption ensure data is stored efficiently and securely. A powerful GUI and data analytics provide the visibility and control needed to manage the entire NAS environment with ease.
- Let me tell you more about what we mean by the Avere Hybrid Cloud. This slide has a diagram that we like to use. It contains all the resources that are available to you when designing your IT/NAS environment. The bottom shows all the on-prem resources. On the left you have your on-prem compute including users workstations and a compute farm and on the right you have on-prem storage including NAS and object storage. The top shows cloud resources from a service provided like Amazon, Google, or Microsoft. On the left you have the compute cloud where you can rent servers to run your apps and on the right you have the storage cloud where you can rent capacity to store your data. Avere products get deployed in 5 different use cases in this hybrid cloud infrastructure. Starting at the bottom, with our physical FXT we have the NAS Optimization use case. (CLICK) NAS Optimization is where we started as a company. In the NAS Optimization use case, we place an All-Flash performance tier infront of your existing disk-based NAS systems. The FXT clusters use caching to automatically accelerate active date, cluster to scale performance (add more CPUs and DRAM) and capacity (add more SSDs), and hide the latency to the core storage which sometimes is deployed over a WAN. Many of our customers are using Avere in this way to accelerate the performance of rendering, genomic analysis, financial simulations, software tools and binaries respository, and more. (CLICK) In the File Storage for Private Object use case, we have found many customers who are looking to move from NAS to private object stoage. They like private object for it’s efficiency, simplicity, and resiliency. However, they don’t like it’s perfromance or object-based API interface. In this use case, the FXT cluster accelerates the perfromance of the private object storage in the same way as in the NAS optimization use case. But, in addition, the FXT cluster provides familar NAS protocols with translation to object APIs on the storage side, so customerrs can use object storage without re-writing their applications or changing their data access methods. (CLICK) The Cloud Storage Gateway use case is similar to the File Storage for Private Object use case with the added benefit that customers can begin to build fewer data centers and move their data to the cloud. Latency is one of the key challenges that needs to be overcome in this use case and that’s what the physical FXT cluster provides. On access, data is cached on premises on the FXT cluster so all subsequent accesses occur at low latency. An FXT cluster can have as much as 480TB of total caching capacity, so large volumes of data can be stored on premises to avoid the latency of the cloud. (CLICK) The Cloud Bursting use case enables customers to take advantage of the near infinite supply of compute nodes available in the clouds offered by AWS, GCP, and MS Azure. At peak times, customers can add from 100s to many 10s of thousands of compute cores to run rendering, genomics, and financial batch jobs in the compute cloud. The virtual FXT is software-only version of our product that runs on high-perfromance servers and SSDs available in the compute cloud and automatically caches active data in the cloud from on-premises NAS (or object) systems and delivers this active data at low latency to the application. (CLICK) Last, the Cloud NAS use case is for when everything is moved to the cloud, both the app and the storage. This use case allows you to build fewer data centers and fully embrace the cloud. The Avere Hybrid Cloud allows you to get there in a simple, stepwise fashion, often going through either the Cloud Bursting or Cloud Gateway use cases first. The Avere Hybrid Cloud allows you to move to the cloud at your own pace and under the terms you define.
- With Avere, you can bring together the best of scale-out NAS and cloud gateways for a best-in-class NAS experience with flexibility, control, and cost-savings. Large on-premises infrastructures are using Avere to create an agile architecture that provides freedom for data to move between on-premises and cloud storage and take advantage of unlimited cloud compute resources at a much lower total cost of ownership. Cloud NAS is powered by the Avere Virtual Edge filer (vFXT), a software-only file server that bridges the storage cloud and the compute cloud. The Avere vFXT provides a fully functional NAS to support both NFS and SMB applications running in the cloud.