Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

HARNESS project Demo

318 views

Published on

HARNESS project platform demo

Published in: Software
  • Be the first to comment

  • Be the first to like this

HARNESS project Demo

  1. 1. http://www.harness-project.euhttp://www.harness-project.eu/ The HARNESS Project Technology Demo Hardware- and Network-Enhanced Software Systems for Cloud Computing Mark Stillwell Imperial College London Software Carpentry Bootcamp
  2. 2. http://www.harness-project.euhttp://www.harness-project.eu/ Introduction Goal of HARNESS project: allow cloud providers and tenants take advantage of heterogeneous resources Key idea: heterogeneous resources as first-class entities FPGAs, GPGPUs, programmable routers, heterogeneous storage volumes Benefits: offer a wider context to make price/performance trade offs provide larger scope for resource allocation/optimization increase application performance reduce energy consumption and cost profiles
  3. 3. http://www.harness-project.euhttp://www.harness-project.eu/ Integration/Testing Testbed@Imperial axel07 .doc.ic.ac.uk maia01 .doc.ic.ac.uk infiniband GPGPU SSD MPC-X HARNESS cloud controller SSD HDD
  4. 4. http://www.harness-project.euhttp://www.harness-project.eu/ The RTM HARNESS Use Case 1. Generates an image of the sub-surface from seismic survey data  Processing ~10,000 “shots”  1 shot = 1 acoustic stimulus + ~16 s recording on ~10,000 hydrophones 2. Embarrassingly parallel between shots 3. Can project full RTM runtime from single shot
  5. 5. http://www.harness-project.euhttp://www.harness-project.eu/ HARNESS cloud phases Two distinctive phases: –Profiling: application performance model building –Production: application runs with objectives based on performance model from Profiling phase
  6. 6. http://www.harness-project.euhttp://www.harness-project.eu/ Testbed @ Imperial Axel07 Maia01 AM Demo Preview: Black-box Profiling CRS OpenStack I R M Instance: RTM VPCU = 2 DFE = 1 Memory = 1GB Host = Maia01 Front End Instance type: RTM VPCU = 1 Memory = 1GB Host = Axel07 Director Configuration 1 • 1 VM: 2VCPUs, 1G RAM, 100MB Disk, 1DFE … • 1 VM: 1VCPUs, 0.5G RAM, 200MB Disk, … Conf 1: Time 1, Cost 1 Conf 2: Time 2, Cost 2 Conf 3: Time 3, Cost 3 Conf 4: Time 4, Cost 4 Conf 5: Time 5, Cost 5 Configuration 2 • 1 VM: 1VCPUs, 2G RAM, 110MB Disk, 1DFE … • 1 VM: 2VCPUs, 1G RAM, 200MB Disk, … Configuration 3 • 1 VM: 2VCPUs, 2G RAM, 200MB Disk, 2DFE … • 1 VM: 1VCPUs, 2G RAM, 100MB Disk, … Manifest My zip app details
  7. 7. http://www.harness-project.euhttp://www.harness-project.eu/ DEMO: Cloud Tenant View Profiling - Performance Model Building 1. User uploads application + manifest via web frontend 2. Platform creates Application Manager which starts profiling to build performance model  Manifest defines the configuration space: 1 and 3 CPU cores, 2 to 4 DFEs  Each execution runs via CRS + IRMs 3. Tenant can download generated performance model
  8. 8. http://www.harness-project.euhttp://www.harness-project.eu/ Step-by-Step Production View Step1: User submits - Application zip - Application Manifest - Application performance model Step2: Director assesses user’s request, if ok creates an Application Manager instance Step3: AM requests necessary resources to CRS Step4: CRS requests the resources through the appropriate IRMs Step5: IRM creates instance(s) in the relative physical resource and provides appropriate handles to AM Step6: AM deploys the application in the instances created Step7: AM, once application up and running, provides end point to user
  9. 9. http://www.harness-project.euhttp://www.harness-project.eu/ Testbed @ Imperial Axel07 Maia01 Demo Preview CRS OpenStack I R M Instance: RTM VPCU = 2 DFE = 1 Memory = 1GB Host = Maia01 Instance type: Application Manager VPCU = 1 Memory = 0.5GB Host = Axel07 Front End Director • 1 VM: 2VCPUs, 1G RAM, 100MB Disk, 1DFE … • 1 VM: 1VCPUs, 0.5G RAM, 200MB Disk, … SLO I want run my zip app with X performance and Y budget
  10. 10. http://www.harness-project.euhttp://www.harness-project.eu/ Demo: Cloud Tenant View 1. User uploads application + manifest via web frontend 2. User uploads application performance model 27 configurations: 1 to 4 DFEs, 1GB and 2GB RAM, 10MB/s and 180MB/s storage performance, 1 to 3 CPU cores 3. User specifies objectives for run 4. Platform indicates whether objectives are satisfiable 5. User triggers execution 6. Application Manager selects resource configuration and executes process
  11. 11. http://www.harness-project.euhttp://www.harness-project.eu/ Demo: Provider view
  12. 12. http://www.harness-project.euhttp://www.harness-project.eu/ Cross-Resource Scheduler: CRS 1. CRS starts with no knowledge about resources 2. IRMs connect to CRS and announce their resources  IRM-NOVA => server machines  IRM-SHEPARD => accelerators  IRM-XtreemFS => storage 3. We issue 4 different reservation requests:  homogeneous compute resources  hardware accelerators  storage volume with performance guarantee  heterogeneous resources (1 VM + 1 DFE)
  13. 13. http://www.harness-project.euhttp://www.harness-project.eu/ Demo: RTM Application
  14. 14. http://www.harness-project.euhttp://www.harness-project.eu/ Conclusion Goal of HARNESS project: allow cloud providers and tenants take advantage of heterogeneous resources Key idea: heterogeneous resources as first-class entities FPGAs, GPGPUs, programmable routers, heterogeneous storage volumes Benefits: offer a wider context to make price/performance trade offs provide larger scope for resource allocation/optimization increase application performance reduce energy consumption and cost profiles

×