Your SlideShare is downloading. ×
Nephele pegasus
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Nephele pegasus

601
views

Published on

This talk is to give an overall overview related to Pegasus and Nephele framework

This talk is to give an overall overview related to Pegasus and Nephele framework

Published in: Technology, Education

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
601
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
51
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. An Overview of Nephele and Pegasus Presented  By:  Somnath  Mazumdar   somnath.mazumdar@ucdconnect.ie   h8ps://www.csi.ucd.ie/users/somnath-­‐mazumdar    
  • 2. Nephele: Efficient Parallel Data Processing in the Cloud By: Daniel Warneke and Odej Kao Technische Universität Berlin Berlin, GermanyPublished in: ProceedingMTAGS 09 Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers ACM New York, NY, USA ©2009
  • 3. Nephele: What for ???•  General trends: At the beginning of processing resource allocation is done (static), no scope to extend/remove resources (dynamic) ….•  Paper (Claims): “…first data processing framework to include the possibility of dynamically allocating/ deallocating different compute resources ….”
  • 4. Known Issues in Cloud•  Cloud resources are dynamic and heterogeneous•  Provisioning of resources on demand•  Cloud challenge: opaqueness
  • 5. Nephele Architectural Overview
  • 6. Jobs @ Nepheleq I. Steps to create a job(DAG):1.  Write own code for task.2.  Assign tasks to a vertex.3.  Define communication path for Job.q II. Add annotation to job description.q III. Transform Job Graph àExecution Graph  
  • 7. An Execution Graph Efficient Execution Graph creation depends on user input / job annotation description  
  • 8. Pegasus: A framework for mapping complex scientific workflows onto distributed systems By: Ewa Deelman, Gurmeet Singh, Mei-Hui Su, James Blythe, Yolanda Gil, CarlKesselman, Gaurang Mehta, Karan Vahia, G. Bruce Berriman, John Good, Anastasia Laity, Joseph C. Jacob and Daniel S. Katz University of Southern California Information Sciences Institute, CA, USA Infrared Processing and Analysis Center, Jet Propulsion Laboratory, CalTech USA Published in: Journal Scientific Programming archive Volume 13 Issue 3, July 2005 IOS Press Amsterdam, The Netherlands
  • 9. Before Starting….  ü  Workflow can capture the behavior of application.(abstract & Concrete)ü  Workflows are abstract in the application- level (describes application components and their dependencies)ü  Simplifies application development process(+)ü  Concrete workflow describe resources that would be used in execution of specific tasks.
  • 10. Pegasus: What for..??ü Describes: process of mapping from abstract to executable workflow can be automated.ü Assumption: 1. Application is already represented in an abstract workflow form 2. Data does not specify particularresources to be used.ü Scheduling horizon encompasses tasks that can be sent to execution system.ü Mapping horizon indicates how far into the workflow to map the tasks.
  • 11. Horizons & Costs Mapping depends on specific resources to execute specific tasks as well as data locality.
  • 12. Logical Overview of Pegasus
  • 13. Failure Recovery In Pegasus
  • 14. Thank  You