Building Computational
Grids with Apple’s Xgrid
            Middleware
                            Baden Hughes
       Dep...
Talk Overview
     Introduction
     Xgrid History
     Solution Architecture
     Apple Native vs 3rd Party Components
  ...
Introduction
     In June 2005 Apple released Xgrid 1.0, a framework
     for loosely coupled distributed computation,
   ...
Xgrid History
     1/2004 Xgrid Technical Preview 1
           Proof of concept only
           Limited developer release
...
Xgrid Architecture
     Client, Controller, Agent
     Client
           User interface; controller authentication; job
  ...
Configuration and Systems
Management
     Integrated local client configuration via System
     Preferences
     Remote cl...
Job Specification and
Management
     Job = 1 or more tasks
     Jobs described in Apple’s extensible plist format
     (e...
Scalability
     Published (cluster mode) benchmarks:
           128 Agents
           20,000 queued jobs
           100,0...
Apple Native Components
Overview




17 January 2006   Hughes @ AusGrid 2006   9
Apple Native Components
Detail
     GUI:
       Xgrid.app, simple user interface
     CLI
       Xgrid, batch oriented, su...
3rd Party Components
Overview




17 January 2006   Hughes @ AusGrid 2006   11
3rd Party Components Detail
     GridStuffer
           OS X GUI (Cocoa) application for multi-task job submission,
      ...
Production Experience
     Locations
           Universities: Stanford, Reed, Utah, Simon Fraser, Toronto,
           Melb...
Conclusion
     Significant advantage for Xgrid is ease of
     administration and use; contrasts with many other
     clu...
More Xgrid Info

     http://www.apple.com/acg/xgrid

     http://developer.apple.com/hardware/hpc

     http://www.macres...
Acknowledgements
     Research supported by Apple Computer Inc,
     through the Apple University Consortium and
     the ...
Upcoming SlideShare
Loading in …5
×

Building Computational Grids with Apple’s Xgrid Middleware

1,778 views

Published on

Paper at AusGrid2006 (January 2006, Hobart)

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,778
On SlideShare
0
From Embeds
0
Number of Embeds
16
Actions
Shares
0
Downloads
25
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Building Computational Grids with Apple’s Xgrid Middleware

  1. 1. Building Computational Grids with Apple’s Xgrid Middleware Baden Hughes Department of Computer Science and Software Engineering The University of Melbourne badenh@csse.unimelb.edu.au 17 January 2006 Hughes @ AusGrid 2006 1
  2. 2. Talk Overview Introduction Xgrid History Solution Architecture Apple Native vs 3rd Party Components Production Experience Conclusions 17 January 2006 Hughes @ AusGrid 2006 2
  3. 3. Introduction In June 2005 Apple released Xgrid 1.0, a framework for loosely coupled distributed computation, specifically targets embarrasingly parallel computation Xgrid has been positioned as being highly accessible advanced computing solution for less technically oriented user communities, particularly small to medium research groups without easy access to traditional HPC environments While technically simple, from a functional perspective Xgrid blurs the distinction between HPC user and maintainer, which could engender new grass-roots research through a self-sustaining model for HPC 17 January 2006 Hughes @ AusGrid 2006 3
  4. 4. Xgrid History 1/2004 Xgrid Technical Preview 1 Proof of concept only Limited developer release 11/2004 Xgrid Technical Preview 2 Robust CLI, immature GUI Wide developer release Reasonable adoption, 3rd party tools emerge 4/2005 Xgrid 1.0 Robust CLI, GUI, API Production oriented, vendor supported release Mac OS X Server centric (somewhat controversially) 17 January 2006 Hughes @ AusGrid 2006 4
  5. 5. Xgrid Architecture Client, Controller, Agent Client User interface; controller authentication; job submission/retrieval Controller Job submission; management (aka cluster head node + some brokering) Agent Compute node (dedicated or cycle stealing; local or remote) 17 January 2006 Hughes @ AusGrid 2006 5
  6. 6. Configuration and Systems Management Integrated local client configuration via System Preferences Remote client configuration via ssh, Apple RD; Netboot etc Controller based Server Admin tools allow ‘whole of grid’ configuration (authentication methods, agent type etc) Integrated user management and authentication (Kerberos, SSO, LDAP, Active Directory) Controller discovery either specified (FQDN) or dynamic (ZeroConf, Rendezvous, Bonjour) 17 January 2006 Hughes @ AusGrid 2006 6
  7. 7. Job Specification and Management Job = 1 or more tasks Jobs described in Apple’s extensible plist format (either native plist or XML) Submission to controller via client either in serial or batch mode Controller periodically probes all registered agents to determine availability (NB currently proprietary scheduling heuristic) Asynchronous client-controller interaction model: clients can disconnect and later reconnect to retrieve completed jobs 17 January 2006 Hughes @ AusGrid 2006 7
  8. 8. Scalability Published (cluster mode) benchmarks: 128 Agents 20,000 queued jobs 100,000 tasks per job 2Gb data per job 1Gb results per task 10Gb aggregate results per job Production sites with >200 agents in distributed mode Integration with existing OS X management tools such as ARD does allow for considerable scalability 17 January 2006 Hughes @ AusGrid 2006 8
  9. 9. Apple Native Components Overview 17 January 2006 Hughes @ AusGrid 2006 9
  10. 10. Apple Native Components Detail GUI: Xgrid.app, simple user interface CLI Xgrid, batch oriented, supports multitask submission Cocoa Framework XgridFoundation, ObjC core, with PyObj access, full featured Xgrid Controller client authentication, node management, job scheduling Tiger Agent Xgrid agent for Mac OS X 10.4 nodes Panther Agent Xgrid agent for Mac OS X 10.3 nodes Network Communications Based around BEEP (IETF RFC 3080) 17 January 2006 Hughes @ AusGrid 2006 10
  11. 11. 3rd Party Components Overview 17 January 2006 Hughes @ AusGrid 2006 11
  12. 12. 3rd Party Components Detail GridStuffer OS X GUI (Cocoa) application for multi-task job submission, monitoring, retrieval PyXG Python/PyObjC-based API for Xgrid client GridBus Data Broker: data-centric brokering service with (experimental) support for Xgrid client XgridLite: functionally reduced Xgrid controller for OS X workstation platforms Xgrid Linux Agent: Linux native Xgrid agent XgridAgent-Java: Platform-independent Java-based Xgrid agent 17 January 2006 Hughes @ AusGrid 2006 12
  13. 13. Production Experience Locations Universities: Stanford, Reed, Utah, Simon Fraser, Toronto, Melbourne Numerous industrial sites Application Domains Natural language processing Modelling biochemical receptors Nonlinear computation in epidemiology Rendering animations Low autocorrelation binary sequence search Biophytonic tissue analysis 17 January 2006 Hughes @ AusGrid 2006 13
  14. 14. Conclusion Significant advantage for Xgrid is ease of administration and use; contrasts with many other cluster/grid computing middleware Some disadvantages in area integration (particularly with other middleware suites) Active 3rd party development community and open relations with Apple’s Xgrid product group have made Xgrid more compelling Some shrink-wrapped applications have Xgrid support (eg GridMathematica) Other cluster/grid computing solutions for Mac OS X exist (eg MPI, Pooch) 17 January 2006 Hughes @ AusGrid 2006 14
  15. 15. More Xgrid Info http://www.apple.com/acg/xgrid http://developer.apple.com/hardware/hpc http://www.macresearch.org 17 January 2006 Hughes @ AusGrid 2006 15
  16. 16. Acknowledgements Research supported by Apple Computer Inc, through the Apple University Consortium and the Apple University Development Fund Thanks to Ernest Prabhakar (Xgrid Product Manager) and Richard Crandall (Advanced Computation Group) from Apple for comments on an earlier version of this paper 17 January 2006 Hughes @ AusGrid 2006 16

×