Hecatonchire: Transparent Resource Aggregation                                Israeli Networking Day 2013Aidan Shribman; S...
In the talk todayMotivation: the SAP HANA in-memory database.Intro: the Hecatonchire projectCapabilities: mainly memoryagg...
Back story: SAP HANA In-Memory Real-Time AnalyticsSAP ERP R/3 (1980s)                                         SAP HANA In-...
A “gap” in Current Cloud OfferingsGuest are constrained by host sizes                            Host resources are pooled...
The Hecatonchire ProjectHecatonchires in Greek mythology means “Hundred Handed One”. The project addresses thetransparent ...
Faster to Scale-Out …                                      Author: Chaim Bendalac© 2012 SAP AG. All rights reserved.      ...
Challenges of Resources AggregationCPU (future) State is MBs in size (can be migrated sub-second) Can be virtualized usi...
VM Memory PoolingFast Remote Page AccessLow latency RDMA (Remote Direct Memory Access) page transfer protocolSupports best...
Multithreaded 2 GB Quicksort Benchmark                                    110.00%    Completion Time / Native Time        ...
SAP HANA TPC-H like OLAP Benchmark (128 GB / 40 vCPU)                                    102.00%    Completion Time / Nati...
Application Memory PoolingRequired if virtualizaiton can„t be used due toperformance of usage of exotic hardware.When usin...
Hecatonchire key takeawaysAlready provides transparent VM / applicationmemory aggregationExtends existing cloud & virtuali...
Thank you!Aidan ShribmanSAP Research Israelaidan.shribman@sap.comThe Hecatonchire Projecthttp://www.hecatonchire.comhecato...
Legal DisclaimerThe information in this presentation is confidential and proprietary to SAP and may not be disclosed witho...
Upcoming SlideShare
Loading in …5
×

Networking Israeli Day 2013 - Hecatonchire: Transparent Memory Aggregation

498 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
498
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Networking Israeli Day 2013 - Hecatonchire: Transparent Memory Aggregation

  1. 1. Hecatonchire: Transparent Resource Aggregation Israeli Networking Day 2013Aidan Shribman; Sr. Researcher; SAP Research IsraelWith the contribution of Benoit Hudzia, Roei Tell, Steve Walsh, Peter Izsak
  2. 2. In the talk todayMotivation: the SAP HANA in-memory database.Intro: the Hecatonchire projectCapabilities: mainly memoryaggregation.Numbers: quicksort & SAPHANA TCP-H benchmarks.© 2012 SAP AG. All rights reserved. 2
  3. 3. Back story: SAP HANA In-Memory Real-Time AnalyticsSAP ERP R/3 (1980s) SAP HANA In-Memory Database (2010)Moderate hardware requirements; Is optimized for slow Big multi-core large-memory systems; highstorage devices. memory/core ratio: 128 GB; 256 GB; … 1024 GB.OLTP (OnLine Transactioanl Processing) : mainly OLAP (OnLine Analytical Processing): on-the-flystoring and retrieving transactions analytics on real-time dataSingle-record access: Single-scan aggregation:SELECT * FROM Sales_Orders SELECT Country, SUM(Sales) FROM Sales_OrdersWHERE Order = ‘457’ WHERE Product=‘corn’ GROUP BY Country Order Country Product Sales 456 France corn 1000 Order Country Product Sales 456 France corn 1000 457 Italy wheat 900  457 Italy wheat 900 458 Italy corn 600 458 Italy G corn 600 459 Spain rice 800 459 Spain rice 800 Column order organization Row order organization © 2012 SAP AG. All rights reserved. 3
  4. 4. A “gap” in Current Cloud OfferingsGuest are constrained by host sizes Host resources are pooled together (future?)Limited by physical constraints (AWS high- No limitation: Better scalability; Better economics;memory is 244 GB guest atop a 256 GB host). Better performance. Guests Guests VM VM VM VM VM App VM App App App App OS App OS OS OS VM App OS OS H/W H/W H/W H/W OS H/W H/W H/W Server #1 Server #2 Server #n Server #1 Server #2 Server #n CPUs CPUs CPUs CPUs CPUs CPUs Memory Memory Memory Memory Memory Memory I/O I/O I/O I/O I/O I/O Fast RDMA Communication Fast RDMA Communication© 2012 SAP AG. All rights reserved. 4
  5. 5. The Hecatonchire ProjectHecatonchires in Greek mythology means “Hundred Handed One”. The project addresses thetransparent aggregation of compute resources such as memory, CPU and storageHecatonchire aims at extending (not replacing) existing cloud and virtualization stacks(Linux/KVM, QEMU, OpenStack) while using of commodity hardware and moderninterconnects (10 GbE or InfiniBand)Hecatonchire is an open source project hosted on github developed by SAP Research© 2012 SAP AG. All rights reserved. 5
  6. 6. Faster to Scale-Out … Author: Chaim Bendalac© 2012 SAP AG. All rights reserved. 6
  7. 7. Challenges of Resources AggregationCPU (future) State is MBs in size (can be migrated sub-second) Can be virtualized using hypervisor vCPU abstraction Difficult challenge: Implementing efficient fine-grained cache-coherency (difficult)Memory (in-progress) GBs to TBs is size (accessed in page granularity 4KB – 4MB) Can be virtualized by extending the MMU (Main Memory Unit) system Challenge: achieving near-native performance while not degrading availabilityStorage (mostly-done) TBs to PBs in size (but accessed in blocks MBs in size) Already virtualized using VFS; BLK; SCSI; iSCSI; virtio/vhost ;iSER; LIO. Continued challenge: high bandwidth and high IOPS (partially achieved)© 2012 SAP AG. All rights reserved. 7
  8. 8. VM Memory PoolingFast Remote Page AccessLow latency RDMA (Remote Direct Memory Access) page transfer protocolSupports best price/performance hardware: InfiniBand, iWARP, RoCEDemand pre-paging (pre-fetching) mechanismBuilt-in RAID-1 style Memory MirroringNo extra single points for failureMinimal overhead : 1st slave to answer synchronous; 2nd - asynchronousComplimentary to VM-based High-Availability (e.g. Kemari)Transparent for guest VMsIntegration by adding static hooks into Linux MMU (page fault, swap-out, etc.).QEMU Binds Remote Memory to Guest Linux/KVM Virtual Machine.© 2012 SAP AG. All rights reserved. 8
  9. 9. Multithreaded 2 GB Quicksort Benchmark 110.00% Completion Time / Native Time 108.00% 106.00% 104.00% 102.00% Native 100.00% Remote Mirrored 98.00% 96.00% 94.00% 75% 50% 33% 25% 20% Local Memory Percentage© 2012 SAP AG. All rights reserved. 9
  10. 10. SAP HANA TPC-H like OLAP Benchmark (128 GB / 40 vCPU) 102.00% Completion Time / Native Time 101.50% 101.00% 100.50% Native 100.00% Mirrored 99.50% 99.00% 50% 33% Local Memory Percentage© 2012 SAP AG. All rights reserved. 10
  11. 11. Application Memory PoolingRequired if virtualizaiton can„t be used due toperformance of usage of exotic hardware.When using virtualization we „remotify“ the entireguest VM address space.Here we need to „remofity“ all processesassociated with the application.© 2012 SAP AG. All rights reserved. 11
  12. 12. Hecatonchire key takeawaysAlready provides transparent VM / applicationmemory aggregationExtends existing cloud & virtualizationsolutionsReleased as open source / hosted on GitHubDeveloped by SAP Research TechnologyInfrastructure (TI) Practice© 2012 SAP AG. All rights reserved. 12
  13. 13. Thank you!Aidan ShribmanSAP Research Israelaidan.shribman@sap.comThe Hecatonchire Projecthttp://www.hecatonchire.comhecatonchire@liberlist.com
  14. 14. Legal DisclaimerThe information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission ofSAP. This presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAPhas no obligation to pursue any course of business outlined in this document or any related presentation, or to develop orrelease any functionality mentioned therein. This document, or any related presentation and SAPs strategy and possible futuredevelopments, products and or platforms directions and functionality are all subject to change and may be changed by SAP atany time for any reason without notice. The information on this document is not a commitment, promise or legal obligation todeliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied,including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. Thisdocument is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors oromissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materiallyfrom expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only asof their dates, and they should not be relied upon in making purchasing decisions.© 2012 SAP AG. All rights reserved. 14

×