Amazon EC2 F1 Developing Cloud-Scale Accelerations Sep 13, 2017
•
2 likes•555 views
Learn how to develop a cloud scale FPGA accelerations using AWS F1 instances, using AWS Marketplace and F1 Partner Network.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Amazon EC2 F1 Developing Cloud-Scale Accelerations Sep 13, 2017
Similar to Amazon EC2 F1 Developing Cloud-Scale Accelerations Sep 13, 2017 (20)
Recently uploaded
Recently uploaded (20)
Amazon EC2 F1 Developing Cloud-Scale Accelerations Sep 13, 2017
- 1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Gadi Hutt, Amazon Web Services September 13, 2017 Developing Cloud Scale FPGA Accelerations Using AWS F1 F1
- 2. What is Elastic Cloud Compute (EC2)?
- 3. Amazon AWS Cloud Agility and speed of innovation Elasticity Scale up or down quickly, as needed Breadth of functionality Go global in minutes Cost savings
- 4. Responding requires a new model Focus on differentiating your company Innovate at start-up like speed Reduce risk
- 5. Up to 8 Xilinx UltraScale+ 16nm VU9P FPGA devices in a single instance The f1.16xlarge size provides: 8 FPGAs, each with over 2 million customer-accessible FPGA programmable logic cells and over 5000 programmable DSP blocks Each of the 8 FPGAs has 4 DDR-4 interfaces, with each interface accessing a 16GiB, 72-bit wide, ECC-protected memory Instance Size FPGAs DDR-4 (GiB) vCPUs Instance Memory (GiB) NVMe Instance Storage (GB) Network Bandwidth f1.2xlarge 1 4 x 16 8 122 1 x 470 Up to 10 Gbps f1.16xlarge 8 32 x 16 64 976 4 x 940 25 Gbps F1 FPGA Instance Types on AWS
- 6. The FPGA Developer AMI Use Xilinx Vivado and a hardware description language (Verilog or VHDL for RTL) with the HDK to describe and simulate your custom FPGA logic Developing Applications for F1 Xilinx Vivado for custom logic development Virtual JTAG for interactive debugging
- 8. Create the Amazon FPGA Image (AFI) Generate an encrypted AFI using the generated DCP
- 9. Create the Amazon FPGA Image (AFI) Generate an encrypted AFI using the generated DCP
- 10. OpenCL Generally Available for F1 Familiar development experience to accelerate C/C++ applications 50+ F1 code examples available that span multiple domains: security, image processing and accelerated algorithms Including working examples ported from Intel (Altera) Already supported on the FPGA Developer AMI, no need to upgrade/install
- 11. Amazon Machine Image (AMI) Amazon FPGA Image (AFI) CPU Application DDR-4 Attached Memory DDR-4 Attached Memory DDR-4 Attached Memory DDR-4 Attached Memory DDR-4 Attached Memory DDR-4 Attached Memory DDR-4 Attached Memory DDR-4 Attached Memory PCIe DDR Controllers Launch Instance and Load AFI An F1 instance can have any number of AFIs An AFI can be loaded into the FPGA in seconds FPGA Acceleration Using F1 EC2 F1
- 12. Added Support for a graphical development flow FPGA Developer AMI upgraded to the latest Vivado (2017.1) improving the synthesis quality and runtime capabilities Improved FPGA utilization and made it easier to close timing Support FPGA initiated read/write over PCI (PCI-M) Doubled the URAM capacity Expanded DMA support EC2 F1 Rapid Pace of Innovation
- 13. Financial computing Genomics Sequencing Engineering simulations Image and video processing Big data and machine learning Security, Compression …and more F1 Use Cases and Partners
- 14. Connecting FPGA Partners with AWS Users on AWS F1
- 15. Connecting FPGA Partners with AWS Users
- 16. Connecting FPGA Technology Partners with End customers
- 17. Leverage Xilinx FPGA-based performance with CPU-like ease-of-use for lightning-fast insight from all your data and business analytics applications on AWS cloud Bringing the Power of FPGAs to Everyone
- 18. Standard Application Stacks Leverage Ryft to Gain the Advantages of FPGA Technology with Zero Learning Curve 18 Ryft makes FPGAs easy-to-use by tightly coupling industry standard software interfaces/APIs with fast FPGA-accelerated primitives for: Real-time performance: Eliminate data preparation bottlenecks—including indexing and transformation. Low Latency Operations: Seamlessly and constantly swap Ryft Analytics Primitives in-and-out during the course of normal operation with Xilinx’s superior partial reconfiguration technology. Purpose-built heterogeneous compute: Ensure the right compute architecture—CPU and/or Xilinx FPGA—is used to achieve maximum performance. Legacy system acceleration: Amplify existing ecosystem performance via the Ryft Open API and growing Library of Connectors. Flexibility: Employ a variety of cloud, hybrid and on- premises offerings to select the architecture, performance, pricing model and deployment method that fit your needs.25Gigabit Enhanced Network
- 19. Ryft Technology Architecture on AWS F1 19 For more details, see Amazon’s blog at https://aws.amazon.com/blogs/aws/ec2-f1- instances-with-fpgas-now-generally-available/, and a whitepaper developed with Xilinx at https://www.xilinx.com/support/documentation/product-briefs/ryft-aws-f1.pdf
- 20. Toolkit Powered by Ryft’s Heterogeneous Compute DELIVERS MORE SOPHISTICATED, FASTER CLOUD-BASED DATA SEARCH & ANALYSIS Seamlessly integrate with current analytics environment via a series of APIs and connectors Get more accurate and actionable insights Turning big data into small data Ryft’s FPGA-Accelerated Cloud Accelerators Speed Critical Decisions 20 Elasticsearch Powered by Ryft’s Heterogeneous Compute EXTENDS AND EXPANDS ELASTICSEARCH PERFORMANCE & CAPABILITIES Enhance Regular Expression capabilities Increase the power of edit distance Speed search and analysis across structured & unstructured data Augment wildcard searches to include leading wildcard characters Geospatial (Google Earth) Instantaneous social media geospatial observations without database or indexing Data Visualization (Tableau) Existing visualization ecosystems can now seamlessly and simultaneously operate on unstructured and structured data Elasticsearch (ELK Stack) Elasticsearch runs 100x faster and gets new functionality such as Regex, edit distance and hamming fuzzy search
- 21. Benchmark comparison of Elasticsearch on Ryft Cloud with FPGA-acceleration vs. on CPU Ryft Supercharges Analysis by 91X to Render Big Data Relevant Now 21
- 22. 22 Publish Your Solution Via AWS Marketplace All our customers are interested in capabilities in the Cloud Instead of leading with our on-premises equipment we are leading with Cloud based solutions Create your own listing in AWS Marketplace via the self service listing mechanism Use “private” listing mode to validate documentation URLs and single & cluster instance creation
- 24. 100%Yo Y AWS Consulting Partners 130%Yo Y AWS Managed Service Partners O n A W S M a r k e t p l a c e G r o w t h U s e A P N p a r t n e r s o l u t i o n s & s e r v i c e s 90%+ Fortune 100 60% Partners Headquartered Outside U.S. 370M EC2 Hours Per Month AWS Partner Network
- 25. Amazon EC2 FPGA Deployment via Marketplace Amazon Machine Image (AMI) Amazon FPGA Image (AFI) AFI is secured, encrypted, dynamically loaded into the FPGA - can’t be copied or downloaded Customers AWS Marketplace Delivering FPGA Partner Solutions via AWS Marketplace
- 26. AWS Marketplace Discover, Procure, Deploy, and Manage Software in the Cloud
- 27. AWS Partner Network (APN) T h e A P N f u r t h e r e n a b l e s AW S C u s t o m e r s t o i d e n t i f y h i g h - q u a l i t y A P N P a r t n e r s w h o d e l i v e r v a l u e - a d d e d s e r v i c e s a n d s o l u t i o n s o n AW S t h r o u g h s p e c i f i c A P N P a r t n e r P r o g r a m s a n d g o - t o - m a r k e t ( G T M ) o p p o r t u n i t i e s . F1 https://aws.amazon.com/ec2/instance-types/f1/partners/
- 28. AWS Global Infrastructure 16 Regions – 42 Availability Zones – 74 Edge Locations Region & Number of Availability Zones AWS GovCloud (2) EU Ireland (3) US West Frankfurt (3) Oregon (3) London (2) Northern California (3) Asia Pacific US East Singapore (2) N. Virginia (6), Ohio (3) Sydney (3), Tokyo (3), Seoul (2), Mumbai (2) Canada Central (2) China Beijing (2) South America São Paulo (3) Announced Regions China, France, Hong Kong, Sweden AWS GovCloud (US-East)
- 29. AWS Global Infrastructure 16 Regions – 42 Availability Zones – 74 Edge Locations Region & Number of Availability Zones AWS GovCloud (2) EU Ireland (3) US West Frankfurt (3) Oregon (3) London (2) Northern California (3) Asia Pacific US East Singapore (2) N. Virginia (6), Ohio (3) Sydney (3), Tokyo (3), Seoul (2), Mumbai (2) Canada Central (2) China Beijing (2) South America São Paulo (3) Announced Regions China, France, Hong Kong, Sweden AWS GovCloud (US-East) F 1 F 1 F 1
Editor's Notes
- Since 2006, AWS has grown to serve millions of active customers every month in more than 190 countries, and we’ve developed into a $15B run rate business faster than any technology company in history. We’ve gotten here, because we keep customer obsession and constant innovation behind everything we do. We started working on bringing FPGAs to AWS cloud because we strongly believed it will democratize hardware accelerations, and as a result will bring new use-cases and more compute options to AWS users. Since we launched in April this year, we have seen good growth of users and use-cases, and we are starting to see the fruits of the hard work our customers have put in, and I will share some of these success stories later on. Today I will mainly focus on the value and benefits of building cloud-scale, Elastic FPGA accelerated applications.
- So let’s start by defining What is cloud computing? "cloud computing" refers to the on-demand delivery of IT resources via the Internet with pay-as-you-go pricing. Instead of buying, owning and maintaining your own datacenters and servers, with AWS, organizations can acquire technology such as compute, storage, databases, and other services securely ,on an as-needed basis, on a global scale.
- There are five reasons companies are moving so quickly to the AWS cloud. The first is Increase speed and agility AWS lets you quickly spin up resources when you need them, deploy hundreds or even thousands of servers in minutes. This means you can very quickly develop and roll out new applications, teams can experiment and innovate more quickly and frequently. F1 developers can run their development flows on high-performance servers on AWS cloud, and accelerate productivity by focusing on the application, and not on maintaining and tracking servers and licenses. In addition, our team has implemented all the non-differentiating, heavy lifting HW integration work around Pcie and DRAM, and system health monitoring enabling developers to focus on their differentiating application code and launch products at an accelerated pace. The second reason is cost savings. AWS allows customers to trade capital expense for variable expense, paying for IT as they consume it. And, the variable expense is much lower than what customers can do for themselves because of AWS’s economies of scale. For example, Dow Jones has estimated that migrating its global data centers to AWS will contribute to savings of $100 million in infrastructure costs. Or the Commonwealth Bank of Australia, that cut its storage costs by 50% and is estimating it will save hundreds of millions of dollars. EC2 F1 developers pay $1.65/hr, compared to thousands or more for on-premises appliance, we already see that lowering the entry bar to develop on these massive FPGAs already brings new developers that were not able to do so without our scale. The third reason is elasticity. Customers used to over-provision to ensure they had enough capacity to handle their business operations at the peak levels of activity, if they missed their peek predictions, it would have taken several months to increase capacity. Now, they can provision the number of resources that they actually need, knowing they can instantly scale up or down along with the needs of their business, which also reduces cost and improves the customer’s ability to meet their user’s demands. The same applies to F1: AWS users buying services from F1 partners and developers can scale their accelerated solutions independently, they don’t need to call anyone to provide them with more servers… The fourth reason is the breadth of functionality that exists in AWS. Simply put ,We have more features and services than any other cloud provider. And, we continue to add new capabilities and new services at an accelerating pace. In 2011, we released over 80 significant services and features; in 2012, nearly 160; in 2013, 280, in 2014, 516, in 2015, we launched 722. And last year, we launched over a thousand new services and features. This year we are on track to exceed that. Customers benefit from this continual evolution, innovation and iteration, because they get the newest/latest features or enhancements instantly. No need to upgrade, deploy, or to migrate. The fifth reason is that AWS enables customers to deploy globally in minutes. When we talk about global presence we use the term Regions, which are physical locations around the world where we cluster datacenters. We call each group of logical datacenters an Availability Zone. Using AWS, customers can leverage 43 Availability Zones across 16 geographic regions worldwide. We started F1 running in our US-East region, and just announced last week F1 availability in the EU and US-West. And, we don’t plan to stop there.
- So what does this all mean to you, and your business? The move to the cloud calls for a new model of engagement. Many companies are looking at the cloud as a way to double down on investments that support the core mission of the company and differentiate it from competitors. As a result, CEOs/CIOs and COOs are looking at new ways to innovate that allow for more experimentation and more customer engagement. And they are doing it in a way that meaningfully reduces their security and compliance risks. With F1, as a first-class-citizen in the family of accelerated compute instances on AWS EC2, we see a wave of new companies that are born in the cloud, as well as enterprises that plan to build or transition from their traditional FPGA appliance-based deployments to cloud enabled FPGA virtual appliances and services. They are able to focus on their core differentiating value, contact millions of potential customer base through F1 partner network and AWS Marketplace. For AWS customers, using F1 based applications and services, brings a boost in performance and lower cost compared to CPU and in some cases GPU workloads.
- Before we further discuss how to build application and services on top of F1, lets run through a quick instance overview, and how to develop, create and eventually deploy FPGA based accelerations on AWS FPGAs. We launched with 2 new instance types Supporting up to 8 Xilinx VU9P FPGAs, with high performance host configuration: up to 64 vCPUs, with 976GB DRAM and 940GB of local NVMe SSDs Each FPGA has x16 PCIe Gen3 connectivity to the host, with 4 dedicated DDR4 interfaces to a total of 64GB 72bit wide ECC protected memory All instances come with enhanced networking, Elastic Network Adapter , and in fact, we’ve just announced last week that we have upgraded the F1.16XL networking bandwidth from 20Gbit per second to 25Gbps.
- Our FPGA Developer AMI, is a VM that can be launched from AWS marketplace , it already includes Xilinx Vivado at NO ADDITIONAL COST. Combined with our HDK, that is publically available on github, it allows F1 developers to describe and simulate their custom FPGA logic Developing in the cloud introduced new challenges, as there is no physical hardware for developers to work with: Virtual JTAG allow interactive debugging, vLED and vDIP switches. Build strategies and scripts to automate the simulation and build process Customers can choose to skip simulation all together, and run different build strategies recipes, to find out which one will close timing at the desired performance. You can even do that in parallel, since you can scale as many compute servers you need.
- The FPGA developer AMI is available globally in all of AWS 16 regions. Note that Software/hr column ,at $0 Also there is a large variety of EC2 compute instance selection, which will depend on your design size, and compute needs. If you optimize for cost, you can go for as little as a few cents with our T2 micro, or if you need to complete a large design that requires a lot DRAM and compute, you can choose on of the c4 , R4 or m4 instances that are more compute/memory optimized and range between 50 cents to a dollar or so, per hour.
- Now lets look at the development flow for F1, developers start with our HDK, which is available on Github. Used in combination with Xilinx Vivado for synthesis, and the Amazon FPGA developer AMI, to create the custom logic, this is the developer sandbox, where all the magic happens After simulation and timing closure, users submit the custom logic in a form of an encrypted design checkpoint file to AWS for ingestion using an EC2 API. In the background, AWS creates the AFI using partial reconfiguration to mesh the custom logic with AWS monitoring and interface implementation contained in the F1 Shell. Its important to note that the AFI is securely stored at AWS, and no one has access to it, not even the AFI developer. When an API is called to load the AFI, our backend services loads it to the FPGA, but there is no point where someone can take the AFI and save or copy it. The AFI owner is the only one who can load that AFI, and other AWS users need to get permission, only from the AFI owner to be able to use that AFI. At this point the AFI is ready for testing on F1, and published at MP or shared with other AWS users if you wish to do so. So this HDK flow is how F1 development flow looked like in the in the past 6 months.. Today, I am excited to announce
- that F1 OpenCL is generally available for all OpenCL is a parallel flow to our HDK flow Now you can take C/C++ code and accelerate it on top of F1. Developers with little to no FPGA experience can quickly get started on F1 instances without learning how to program FPGA devices.
- Developers will find a familiar development experience to supercharge their applications. Falcon Computing, an early OpenCL / SDx preview customer reported that the Quality of results is comparable to manually optimized implementation. This enabled them to provide their Merlin compiler, available on AWS MP today, which they say enables a 100% C/C++ to FPGA flow. No OpenCL or FPGA experience required. We already have more than 50 examples available on our github, including OpenCL applications ported from Altera open source repositories, All of this is already supported on the FPGA Developer AMI, no need to upgrade/install I invite all of you to spin up an FPGA developer AMI and start using it.
- So now that we covered how we develop AFI, lets see how the acceleration works on F1 AMI packages the OS and any associated application/drivers needed AFI is loaded using an API you can swap AFIs in and out, at runtime. This can be used create an acceleration pipeline, using F1 application that will swap AFIs as different acceleration is needed, on the fly.
- Based on customer feedback, we have also added new features and improvements into F1 HDK, AWS FPGA Shell, FPGA Developer AMI and the SDK. The FPGA developer AMI now enables a design canvas, enabling faster AFI development using a graphical flow, and leveraging pre-integrated verified IP blocks (Xiilnx IPI). FPGA Dev AMI also improving the synthesis quality and runtime capabilities The Amazon FPGA Shell provides dramatically better utilization of the Custom Logic (CL) area available to the developers. Simply put, this means that developers can now achieve higher FPGA area utilization, enabling larger acceleration logic. We added FPGA-initiated read/write over PCI to the host CPU allowing FPGA logic to initiate data transfers to/from the host CPU applications. We also improved the UltraRAM utilization, enabling the CL to utilize all of the FPGA URAM blocks, useful for designs that need low latency memory access. And finally, we have expanded the DMA support, allowing DMA transactions of any size. And added support for multi-queue in each direction
- So we’ve talked about What is AWS cloud? Why companies move to AWS cloud ? and announced new ways to develop for F1. Now let’s talk about how to monetize and use F1 to accelerate your business. The companies listed, a partial list of our partner network, have been working on many different solutions that use F1. A few would be Financial: monte carlo and risk simulations can be massively accelerated on FPGAs. Now with F1, AWS customers can use this technology when needed, and many times these types of workloads come with seasonal peaks, So F1 enables users to ramp up many instances to complete the tasks, and ramp them down when not needed anymore. Video and image processing: with growing number of photos and video streams coming into the AWS cloud, we see many video and image applications customers use to analyze, process and store on AWS. Companies like Mipsology and NGCODEC already have F1 products on AWS MP for that purpose. Big data analytics, security , many more use-cases Next, I would like to introduce a few notable customer success stories to demonstrate how our partners were able, in short time, to build successful FPGA based elastic and global applications.
- Genomics in the cloud has seen meteoric growth over the past few years, as cost of processing a human genome went down from $1000 in 2014 to double digits today. Edico was founded in 2013 focused on accelerated genetics, using physical FPGA appliances that would require their customers to have on-premises datacenter and IT infrastructure to be able to deploy. Illumina is an interesting company many of you probably haven’t heard of: according to Illumina, they have processed 90% of DNA sequencing globally to date. In 2014 they decided to move to AWS, the move allowed them to expand globally, securely, reliably and accelerate their local governance compliancy requirements, as many were already supported natively by AWS. DNAnexus is another genomic company that operates a global network, providing an API-based platform for sharing and management of data and tools that accelerate genomic research, and yes, they run on AWS as well. Edico quickly realized they can not serve these large target customers with on-premises, because these companies have already chosen to run their business on AWS cloud When we started our preview, Edico ported their Dragen engine to EC2 F1, and that enabled them to offer Dragen acceleration and compressions services seamlessly to all of illumina and DNAnexus customers. This would have not happened without F1. This is a great example of how existing AWS customers consume new accelerations and services from new AWS developers on top of EC2 F1. BTW, Nature journal published an article with predictions on how much genomic data will be available in 2025, and it is measured in EXABYTES so you can see how the scale of AWS cloud, with its security, elasticity, regulatory compliance and GLOBAL reach mesh very well with F1 and our partners in this segment, we expect this trend will continue help process genomic data faster, and at lower costs. Very exciting.
- Another exciting example of the F1 ecosystem in action is the case of Atomic Rules. Atomic Rules is an AWS FPGA consulting partner, offering design services to clients seeking an F1 cloud presence. Atomic Rules received a request from Skreens, an AWS customer, to help them address a specific FPGA design challenge in order to upgrade Skreens' AWS-based application. With Amazon's F1, Atomic Rules was not only able to use their own IPs faster they were able to combine, as part of their work with Skreens, with the NGCODEC compression AFI which gave them the ability to focus on their unique application and leverage an F1 technology partner to use existing and proven F1 AFI acceleration. So in this case using F1, Atomic Rules earned a new customer, and cleverly used an existing marketplace AFI from NGCODEC to accelerate their project performance and timeline. How cool is that?
- Or the case of Tiledmedia, a VR streaming company that teamed up with NGCODEC, And say they were able to reduce encoding costs by 10X using Amazon EC2 FPGAs, compared to current software solutions. Truly amazing. With that, I would like to introduce my friend from Ryft, Pat McGarry.
- Pat is Ryft’s VP of Engineering and he will discuss with you what were they able to build using AWS and F1
- Inspiring The AWS Partner Network (APN in short) currently has tens of thousands of partners across the globe. Specifically, we have seen 110% year over year growth in consulting partners and 130% year over year growth in AWS managed services partners. Over 90% of the Fortune 100 companies are using APN partner solutions and services. APN is a global program and over 60% its members are headquartered outside of the United States, so customers all over the world can utilize partners in their home country. Another area customers use APN Partner solutions is through the AWS Marketplace where over 370 million hours of Amazon EC2 are used every month for AWS Marketplace products. As an AWS customer, the APN enables you to identify high-quality APN Partners who deliver value-added services and solutions on AWS.
- SO LET’S FOCUS ON MP AWS MP is a managed and curated software catalog with an ecommerce storefront, that helps AWS customers innovate faster and reduce costs, by making it easy to discover, evaluate, procure, immediately deploy 3rd party software solutions. Like we’ve seen with the Skreens example before. MP Customers pay for only what they use, on their AWS bill. MP gives its sellers Global reach instantly And the ability to Quickly get to market, ( free trials, Paid evaluations or even publish applications for free, as often the case with research projects. MP lets the sellers Track their application usage And AWS remit payments to the AMI owners per hourly usage, per month F1 AFI, just like the AMI on MP is secured, and MP users don’t have access to the AFI itself, as it is encrypted and loaded by AWS backend service. When an F1 based MP instance is launched. Offering can be packaged as an AMI, or SaaS, where AWS customers are using APIs to access the services. In case of F1, this means the users of the acceleration don’t even need to know its FPGA based, like the case of Edico, Ilumina and DNA NEXUS
- Since April, F1 developers published 15+ AMIs on Marketplace, ranging from complete AMI solutions, like Mipsology, Falcon Computing, Ryft and Edico, to FPGA development tools, and even research projects available at no additional cost. Marketplace allows our sellers to deploy in multiple regions, and reach to millions of potential users.
- If you interested to become an Amazon EC2 F1 instance partner you can find all the onboarding details on our F1 partner page.
- AWS serves millions of customers in more than 190 countries. We currently have 16 Regions around the world today, and we’re not close to being done. We plan to add 5 new AWS regions in the coming months in China, France, Hong kong, Sweden and AWS govcloud. There's a very, very large opportunity for cloud computing internationally and you can expect that we’ll continue to add Regions.
- Specifically for F1, We’re constantly getting feedback from customers on where they would like the next AWS Region to come live with F1 support. Since we launched in April, we already added F1 in 3 regions (with 9 availability zones), and we plan to continue to expand F1 into new regions.
- In summary: I hope you now have better understanding of the dynamics we see in the market, why enterprises are quickly moving to AWS cloud, and how can F1 play a significant role in this ‘new normal’ reality Where AWS cloud becomes the first option by a growing number of businesses. Thanks to Xilinx, Pat and the Ryft team, and to the rest of AWS EC2 F1 partners, Please feel free to approach me throughout the day, I would love to talk to you, my email is gadi@amazon. Thank you!