Ceph Day KL - Ceph on ARM
•Download as PPTX, PDF•
0 likes•128 views
This presentation discusses how ARM-based systems can provide scalable and efficient CEPH storage solutions. It outlines how the ARM ecosystem is innovating in storage, including low-cost enterprise SMB storage and energy efficient enterprise storage racks. Examples are given of CEPH implementations on ARM, such as a 504 node CEPH cluster using converged microservers and CEPH performance on ThunderX platforms comparable to x86 servers but with lower total cost of ownership. Overall, the presentation argues ARM-based systems can deliver scalable, portable, and optimized intelligent flexible cloud storage using CEPH.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (8)
Similar to Ceph Day KL - Ceph on ARM
Similar to Ceph Day KL - Ceph on ARM (20)
Recently uploaded
Recently uploaded (20)
Ceph Day KL - Ceph on ARM
- 1. CEPH on ARM: Scalable and Efficient Jeff Chu CEPH Community Days Director Enterprise Solutions August 2016
- 2. ©ARM 2016 Purpose and Agenda Purpose: Make you aware of the opportunity for increased efficiency and scale that the ARM ecosystem brings to CEPH Agenda: Tomorrow’s cloud will need a range of storage options Introduction to ARM and the ARM Ecosystem ARM and Storage; ARM and CEPH Delivering a range of CEPH storage solutions
- 3. ©ARM 2016 ARM has many meanings in the industry ARM: The Computer Architecture ARM: The Ecosystem Enabling Choice and Innovation ARM: The Company
- 4. ©ARM 2016 Industry leading ecosystem No person will make a great business who wants to do it all himself or get all the credit. Andrew Carnegie
- 5. ©ARM 2016 Core values to deliver Innovation in products, services, and business models Partnership enabling the focus key strengths and differentiators Energy efficiency making possible what was once impossible Disrupting existing markets Creating new opportunities
- 6. 7 zettabytes 44 zettabytes 1 zettabyte 2.3 zettabytes 2016 2020 IP traffic Internet storage
- 7. ©ARM 2016 1ms end to end millions connections per km2 2020: More nodes, more needs 22x bandwidth increase ~30x access nodes
- 8. ©ARM 2016 2020: Needs an intelligent flexible cloud 4x CPU compute density 6x memory bandwidth >4x interconnect >10x acceleration Data center compute growth
- 11. ©ARM 2016 Delivering an intelligent flexible cloud Scaleable Portable Optimized
- 12. ©ARM 2016 Today’s scalable ARM-based platforms Announced or shipping today 4-64 core solutions Up to 7x efficiency demonstrated
- 13. The ARM Ecosystem in Storage APM August 3, 2016
- 14. ©ARM 2016 Historically – Disk Drive and SSD Controllers Microcontrollers for managing HDDs and SSDs Real-Time processors for controlling access to spinning media
- 15. ©ARM 2016 More recently innovative storage as a service Enhanced end-to-end security S3 compatible APIs Nine 9’s data reliability Self managing scalability 3-7x lower end user cost 3x power savings
- 16. ©ARM 2016 Energy efficient enterprise storage racks 50% Reduction in system power 4.2W Maximum power per TB
- 17. ©ARM 2016 Now – Affordable enterprise SMB storage http://www.theregister.co.uk/2016/08/15/hpe_storevirtual_gets_lowcost_armpowered_v ariant/ 58% Cheaper street price >80% Less in $/GB usable capacity
- 18. ©ARM 201618 What about for CEPH? You can see these benefits for CEPH based systems
- 19. ©ARM 2016 Ceph on ARM: Jewel Release adds AARCH64 With the recent Jewel (10.2.0) release AARCH64 community builds are available! See: http://ceph.com/releases/v10-2-0-jewel- released/ Ceph on ARM Resources: • Ceph for AARCH64 officially supported in the latest upstream community release (Jewel) • Working with ecosystem to drive commercial offerings based on the community work • Partner proof points in the public domain: • 504 OSD Ceph cluster on converged Ethernet drives - http://ceph.com/community/500-osd- ceph-cluster/ • Ceph on 64-bit ARM with X-Gene - http://www.slideshare.net/Inktank_Ceph/ceph- on-64bit-arm-with-xgene (slides from Ceph day) • Partner Collaboration on ARM for SUSE Enterprise Storage - https://www.suse.com/company/press/2015/sus e-enterprise-storage-2-first-to-bring-ceph-based- storage-to-any-os.html
- 20. ©ARM 2016 Ambedded CEPH based storage solution
- 21. ©ARM 2016 Unique microserver architecture
- 22. ©ARM 2016 Decreases impact of large failure domains
- 24. ©ARM 2016 But saves significant energy (= operational cost)
- 25. ©ARM 2016 More traditional CEPH approaches…but balanced
- 26. ©ARM 2016 Applied Micro CEPH on Mudan with X- Gene®
- 27. ©ARM 2016 Application note to deploy a CEPH based cluster • Balanced CEPH system • SSDs • HDDs • Network • Leverage efficiency of X- Gene System on Chip • Integrated peripherals • Customer reported a 15% TCO benefit (HW and Power)
- 28. ©ARM 2016 Cavium CEPH on ThunderX Platform
- 29. ©ARM 2016 CEPH performance comparison
- 30. ©ARM 2016 ThunderX_ST vs x86 CEPH TCO • No HBA, chipset or NIC needed • Integrated accelerators • Integrated fabric for Ceph clusters
- 31. ©ARM 2016 Benefits of SOC integration and acceleration
- 32. ©ARM 2016 More than just an HDD Server
- 33. ©ARM 2016 WDLabs Converged Microserver
- 35. ©ARM 2016 504 Converged microservers in a CEPH Cluster
- 37. ©ARM 2016 Summary Tomorrow’s cloud will need a range of storage options – One size does not fit all The ARM ecosystem is innovating to provide efficient and scalable storage solutions Demonstrated benefits Cost Energy Efficiency Simplicity CEPH on ARM is part of Jewel and beyond – explore for yourself
- 38. The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited (or its subsidiaries) in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners. Copyright © 2016 ARM Limited ©ARM 2016
Editor's Notes
- Xu Luo Server Segment Manager Akira Shimizu Segment Marketing Manager
- As the scale of compute continues to grow we find ourselves at a time of major disruption. Not a bad disruption but an opportunity to leverage new levels of access to deliver new services across more connected devices with significantly less latency. To take advantage of this opportunity means delivering new levels of scalability and portability in the network enabling compatible services to be deployed whether from the data center or at the edge. And the scale will require significantly improved levels of efficiency and compute density. This broad range of solutions addressing such a diverse set of requirements can only be delivered by the breadth of ARM and the ARM ecosystem Today I want to make you aware of the opportunity for increased efficiency and scale of leveraging the ARM ecosystem for CEPH by first showing you why it will be crucial to tomorrow’s data center. And then share with you some examples of scale and efficency that is being delivered by the ARM ecosystem… But for those of you who may not know ARM, let me introduce you to ARM and our Ecosystem
- First thing to understand is that “ARM” is thought of in many different ways depending on the topic. ARM as a company has been around for just over 25 years now ARM is based in the UK with a global presence We license technology to our customers enabling them to add their IP and create the right semiconductor chips for their markets and customers Pick a few stats... ARM also has a very technical meaning. It is a computer architecture… The ARM Architecture As a RISC the ARM Architecture was defined with efficiency at the center... And as an Ecosystem, ARM and our partners work together to jointly develop and enable new markets, products, and opportunities… At the same time, many members of the ecosystem compete with each other This collaboration and competition allow for companies to work together where there is little differentiation and focus their resources and investments on the key technologies which differentiate them from their competitors In this manner there is the opportunity for success for everyone...
- Together with our Connected Community®, we are breaking down barriers to innovation for developers, designers and engineers, and enabling competition and choice across technology markets. We share success with our partners…. Andrew Carnegie couldn't’t have said it better…
- Fundamentally ARM and our partners are driving an almost continuous pace of innovation. Innovation in business model, technology, and product development has been a part of ARM and our partners since the beginning And as we said, our business model is dependent on partnership. Our success is dependant upon our customers’ success And this is all built upon a foundation of energy efficiency.. Disrupting existing markets And creating new opportunities
- Now… I am sure that many of you have seen this or other similar data. IP traffic continuing to grow from from 1ZB this year to 2.3ZB in 2020 and storage ballooning from 7 zettabytes today to over 44 zettabytes by 2020 But this is only part of it. Looking at the network specs for 5g and predictions about it's deployment...
- ...there will be a 30 fold increase in access nodes <click> ...in the UK, the move to HD and 4k content will drive up BW requirements by 22x with more than 75% of the traffic being video <click> ...To support IoT and access and control of real time sensors, the 5g specs are for a maximum of 1 ms end to end latency. This is a major driver for more compute at the edge of the network. <click> ...And it is the massive volume of devices, not just the growing number of consumer devices we all have but also those IoT nodes throughout our infrastructure that is driving the requirement for massive increases in the density of connections -------------------------- OLD NOTES - USE AS YOU SEE FIT The point is that it’s not just a bigger hammer – it’s a question of scale and specialisation Need to be able to express the requirements in a simple definitioin – from streaming high speed video to support for 10^6 low BW connections per sq km for sensors… The need here is to quantify some of the claims made. The latest hype topic whether this be driven from IoT, Connected things, connected home, health, augmented reality etc all will place a load on the network. Some data points: Three axis of pressure: MTC MC (Machine Type Comms Mission Critical) MTC NMC (machine Type Comms Non-Mission Critical) Mobile Broadband (subsciber driven) Use cases for each and explain diverse set of challenges. From EE UK market studies (Mobile Broadband): Video will be the driver of long term growth. There will be a 22x increase in data over the UK network between 2015 and 2030. The network in the UK will be required to carry 2200PBytes per month. 76% of the traffic will be video related. 4K video will be the majority of traffic in this timeframe with a data rate per stream of over 18Mbps. This places a demand on the network and forces carriers to consider caching this content on the edge of the network to meet latency demands. Augmented reality and gaming drive about 2/3 of the remaining traffic over the cellular network and amount to approximately 600PetaBytes per month. MTC NMC – drives bandwidth throughput and connectivity everywhere. 2 foils below is an old slide but represents the amount of data that requires to be handled. This is not latency sensitive and can be handled in the core cloud (bring in NFV/Virtualisation), but we require connectivity and bandwidth/thoughput. Another to highlight is the development of Narrow-Band IoT (NB-IoT) in 3GPP that is expected to support massive machine connectivity in wide area applications. NB-IoT will most likely be deployed in bands below 2GHz and will provide high capacity and deep coverage for an enormous numbers of connected devices. MTC MC – Need equivalent data on Machine critical requirements that would drive reduction in latency and require data to be processed and cached at the edge of the network in the edge cloud. To support such latency-critical applications, 5G should allow for an application end-to-end latency of 1ms or less. Many services will distribute computational capacity and storage close to the air interface. This will create new capabilities for real-time communication and will allow ultra-high service reliability in a variety of scenarios, ranging from entertainment, automotive, health to industrial process control. In addition to very low latency, 5G should also enable connectivity with ultra-high reliability and ultra-high availability. For critical services, such as control of critical infrastructure and traffic safety, connectivity with certain characteristics, such as a specific maximum latency, should not merely be ‘typically available.’ Loss of connectivity and deviation from quality of service requirements must be extremely rare. For example, some industrial applications might need to guarantee successful packet delivery within 1 ms with a probability higher than 99.9999 percent. This the allows us to bridge to the so-whats of what we are doing…
- So as the device that normally communicate back to the servers grow and grow. <click> to become the 10s of Billions predicted by so many <click> We will see the increasing demands placed on the required compute... In fact the future of cloud and network infrastructure…is not just about more servers and disks The future will be driven by the types of SERVICES that need to be delivered…. And the specific needs of that service. It means that if you are a cloud or network provider you need datacenters and networks that are highly reusable, highly reconfigurable and highly flexible. Where the topology, the compute, the storage can be adjusted to match the service being delivered. You will need an Intelligent Flexible Cloud…
- Until now, there’s been a clear distinction between what runs in the cloud… and what runs in the network. THAT WILL CHANGE. And I believe there will be new players and business models emerging….
- And the future is more about pushing server functionality throughout the network with some being pushed further to the edge of the network and others staying within the core datacenter This means that networking and cloud capability is delivered closer to where its actually needed…Another way to think about this is that we will have data centers throughout the network…throughout this Intelligent Flexible Cloud This means that you will need storage solutions that can scale to be deployed as needed. Already we see some CDNs deploying content at the edge to minimize the bandwidth impacts…
- For this to happen computing, storage, and networking will have to be: Scalable – we need to make sure we are delivering HW that can scale from the data center to the smallest edge and still run the same SW, services, manageability… Portable – it needs to be portable across a diverse range of workload optimized hardware. Essentially running standard server software on workload optimized hardware without any impact or even knowledge by the developer of those apps and services. The software can take advantage of the underlying acceleration as needed without impact to the delivery or rollout of the service. Really leverage truly open hardware and software interface standards. Optimized – Leveraging workload acceleration to meet the performance demands without breaking the power or size limitations but also deliver a significant increase in compute density for general workloads to be run within those constraints.
- This level of scalability is already being deliver by many of our partners in networking, servers, and storage. And they are already demonstrating huge gains in efficiency… So lets look a little closer at storage…
- As the scale of computing continues to grow with almost no limits, we are also seeing the need to deliver an increased amount of that compute at the edge...
- Historically ARM processors have been in storage but really for the function of controlling the disks themselves or managing solid state storage… But that is NOT what we are talking about today…
- Instead we are seeing companies innovate storage solutions by taking advantage of the benefits of ARM-based systems. Now they can achieve efficiencies such that they could change the dynamics around storage. One such example is an innovative storage service from a company called Cynny Space. They were able to achieve cost and power savings without impacting the reliability all while able to co-exist with established interfaces for users… -------------- ORIGINAL NOTES They innovated at the system level to achieve 3x power saving and about upto 7x lower cost to the end user. But it was just about the lower cost, they delivered a service with high reliability and key attributes like disaster recovery. They have been able to add value added differentiated features like end to end security all the way through their service from server to client and self managing scalbility that allows them to add nodes automatically. From an end user perpective the are able to coexist with established services with S3 compatible APIs.
- Another example is Huawei where they have shown significant power savings by deploying ARM based storage solutions… This is their Ocean UDS storage system where the ARM processor enables them to realize a 50% reduction in system power or a MAXIMUM of 4.2W per TB…
- … And just this week HPE announced their new product line of more affordable enterprise class storage for SMB… Delivering the same level of service at a fraction of the cost compared to using a legacy architecture. Now many of these examples are proprietary solutions so you might ask…
- …What about CEPH? <CLICK> Well I’m here to tell you that these benefits are applicable to CEPH based systems.. . TODAY! in fact…
- First of all, CEPH on ARM has been in the community for some time but as of this year with the Jewel release it is an officially supported part of the upstream community . I’m sure you have all read the blogs on this.. With this release, the momentum will even more for CEPH on ARM. That said, products have been shipping since before this release… For example…
- There are already solutions in the market using ARM and CEPH in storage solutions. Ambeded, for example, is delivering a CEPH based solution putting several microservers together to deliver a CEPH Cluster Storage Appliance… Earlier this year that product won an award at Interop 2016 for Storage… This solution is a different sort of architecture than legacy based solutions in that it distributes the work a little bit different…
- They use a micro server approach where they have a cluster of 8 ARM based micro servers each connected to a drive in a rack. They include the networking, switches, and power supplies while minimizing power… But the power savings are only 1 of the benefits…
- When you start looking at the failure domains, and the resultant rebuild times, they are minimized with this architecture. Instead of a whole server with 10s of drives going down, now you only have a single drive failure.
- And from a performance standpoint, it delivers … Now this data, like much of what I will show you was generated from our partners so I don’t have the specifics but if you have any questions I can get back to you with the answers or put you in contact with them directly…
- For ambedded the energy savings alone is huge when examined at a rack level… This is a consistent savings with what the proprietary solutions were seeing…
- Now if you want to look at an architecture that is more traditional with a number of drives per processor then you only have to look at solutions from a few of our server chip partners. The next few slides are excerpts from some presentations on CEPH storage from both Applied Micro with their x-Gene System on Chip and Cavium with their ThunderX System on Chip… Let’s start by looking at some of the recent work APM has done…
- APM has created a 1U storage platform called Mudan. It leverages their 8 core x-gene 1 SOC to connect up to 12 HDDs with 2 SSD for journaling using a mix of integrated and external peripherals. They have recently gone through some testing on this platform to demonstrate the balanced configuration of the solution… This is not a reference platform but a platform that they are selling to customers for evaluation and small deployments…
- This has all been documented this in an application note on how to deploy a CEPH cluster using X-Gene1. In essence they were able to create a configuration that equally saturates the SSDs, HDDs, and network while not overly stressing the processor and I/O…
- On the Cavium fron they recently did a head to head comparison of a 24 core ThunderX ST with a legacy 2630v3 based x86 server. Penguin computing provides a storage solution based on the Cavium Thunder X that leverages the 16 integrated SATA ports so there is no need for extra expanders which can cause bottle necks or failure points…not to mention add cost.
- They then tested that against a legacy x86 solution and demonstrated that they were able to exceed the performance of a 2630v3 based system.
- But when you look at the complexity of the HW solutions you see the areas for savings at a system level so above and beyond just the processor. In fact
- With SOC integration, the Cavium system is able cost 40-60% less than the legacy systems. Again very similar to the proprietary HPE system that was just announced… And it still achieves the same or better performance…
- Now, do you remember this HDD <CLICK> Well it is actually much more than just an HDD. It is actually what WDLabs calls a Converged Micro Server. They have integrated a processor SOC onto the control board of the HDD creating a micro server…
- This way they run Linux on the drive as well as the OSD on each drive… Similar to the Ambedded solution, the failure domain is kept at a minimum…but in this case the processor, memory, and networking are all included on the HDD itself.
- Here are some of the specifications but essentially it has everything on board that a server would… Now this is a first generation system and they have done some preliminary testing in a huge 504 drive cluster…
- With CEPH running on each micro server, they pulled together 4 PB into a single cluster. They have dual ethernet for the private CEPH network and the external network. Much of this is documented in a CEPH blog from May but here is some of the data…
- So in summary… We encourage you to explore CEPH on ARM for yourself.