SlideShare a Scribd company logo
2006 JavaOneSM
Conference | Session TS-1109 |
TS-1109
The Sun Grid
Compute Utility
A Tutorial for Java Developers
Fay Salwen
Peter A. Murray
Uday Subbarayan
Staff Engineers, Sun Grid
Sun Microsystems, Inc.
Copyright © 2006, Sun Microsystems, Inc., All rights reserved.
2006 JavaOneSM
Conference | Session TS-1109 | 2
The Sun Grid Compute Utility
Learn how to leverage
1000s of CPUs on demand.
A Tutorial for Java Developers
2006 JavaOneSM
Conference | Session TS-1109 | 3
Agenda
Sun Grid Backgrounder
● What are Grid and Utility computing all about?
● Who are using them and for what?
What is the Sun Grid?
● What makes it so special and why would I want to use it?
How to use the Sun Grid
● Basic concepts
Developing Java Applications for Sun Grid
● How to leverage 1000s of CPUs
Example Applications for Sun Grid
● The best way to get started
Cool tools for Sun Grid
● To make make your life easier
2006 JavaOneSM
Conference | Session TS-1109 | 4
The Sun Grid Compute Utility
Sun Grid Backgrounder
What is the Sun Grid?
How to use the Sun Grid
Developing Java Applications for Sun Grid
Example Applications for Sun Grid
Cool tools for Sun Grid
Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 5
Definitions of a Grid
● No single definition
● Each company and individual have their own
● Wikipedia has no less than 7
● Some examples
● Major DB vendor defines it as—Pooling all your
resources into a central location for optimization
● Major HW manufacturer defines it as—The
virtualization of compute and storage resources
● One of the largest grid organizations says—The
grid will be defined when the question of what a
grid is, no longer exists
2006 JavaOneSM
Conference | Session TS-1109 | 6
Lowest Common Denominator
● A grid has lots of CPUs, storage, and
virtualization
● Everyone says it/they will save you $$$,
somehow
● Most vendors/users are afraid of multi tenancy,
except with “friendly” users (is there such a
thing?)
2006 JavaOneSM
Conference | Session TS-1109 | 7
What Is Grid Computing Used For?
Grid Computing is not new, it has been used
for years
Risk and portfolio analysis, Monte Carlo simulations
EDA, MCAD, fluid dynamics, crash-test simulations,
aerodynamics modeling
Digital content creation, animation,
digital asset management
Reservoir simulations, seismic processing
Weather analysis, nuclear yield simulation
Financial Services
Manufacturing
Entertainment/Media
Energy
Government/Edu
Industries Computing Tasks
2006 JavaOneSM
Conference | Session TS-1109 | 8
Benefits of Grid Computing
● Grid Computing allows organizations to:
● Pool systems, manage them
as a common resource
● Increasing utilization
● Reduce costs
● Increase agility through dynamic
resource allocation
2006 JavaOneSM
Conference | Session TS-1109 | 9
Financing
ServicesServices
UtilityUtility
ComputingComputing
TechnologyTechnology
The ability to intelligently match IT resources to
business demand on a pay-for-use basis.
Grid Computing != Utility Computing
Attributes
● Multi-tenancy
● Standardization
● Scale
● Automation
● Immediate Provisioning
● Granular Costing
Utility Computing (UC)—
2006 JavaOneSM
Conference | Session TS-1109 | 10
The Move From
Custom to Utility Models
Standardize/
Aggregate
Customize Utilize
$30/mo
$.05/kWh
2006 JavaOneSM
Conference | Session TS-1109 | 11
The Sun Grid Compute Utility
Background/Context
What is the Sun Grid?
How to use the Sun Grid
Developing Java Applications for Sun Grid
Example Applications for Sun Grid
Cool tools for Sun Grid
Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 12
The Move to Utility in Grid Computing
$1/CPUhr
Standardize/
Aggregate
Customize Utilize
2006 JavaOneSM
Conference | Session TS-1109 | 13
Grid Technology
Utility Model
What Is Sun Grid Compute Utility?
● Pay-per-use compute power
● Standard pricing—$1/cpu-hour
● No contract or minimum commitment
● Racks of compute nodes
● High-speed network interconnects
● Shared storage
● Distributed Resource Management (DRM)
Readily available in a…
2006 JavaOneSM
Conference | Session TS-1109 | 14
Sun Grid Access
● Web-based user interface
● Internet-accessible (www.network.com)
● Available to anybody (US only, at this time)
● No direct Internet access to compute nodes
2006 JavaOneSM
Conference | Session TS-1109 | 15
Access to What?
● Sun Fire V20z™ servers, each containing:
● Dual 2.4 GHz AMD Opteron™ processors with
HyperTransport technology for memory and I/O interface
● 8 GByte RAM
● Solaris™ 10 Operating System
● Storage
● Up to 10 GBytes per user in staging area
● NFS-mounted home directory in execution environment
● Sun N1™ Grid Engine 6 software
● Grid network infrastructure built on a Gbit/s
switched-Ethernet data network
2006 JavaOneSM
Conference | Session TS-1109 | 16
Types of Applications
● Batch/Grid
● Get job done faster
● Data Transformations, such as:
● Extract Transform Load (ETL)
● Document processing
● Computations and Simulations, e.g.:
● Digital media processing (renderings, animations, etc.)
● Financial calculations
● Scenario simulations
● Business Intelligence and Analytics
2006 JavaOneSM
Conference | Session TS-1109 | 17
Who Is Interested?
● Individuals/businesses:
● With variable loads
● Where Time = $
● That are small (e.g., start-ups)
● Anyone who wants to:
● Avoid having to purchase and manage HW/SW
● Avoid having to scale for the maximum
● Get cycles when they need them
● Pay a known cost
● Independent Software Vendors (ISVs)
Datacenter
Utilization
(Conceptual)
Peak
Std
2006 JavaOneSM
Conference | Session TS-1109 | 18
Sun Grid Usage and ISV Model
ISVs
Application/Data
Sun Grid
Compute Utility
Internet
Users
2006 JavaOneSM
Conference | Session TS-1109 | 19
The Sun Grid Compute Utility
Background/Context
What is the Sun Grid?
How to use the Sun Grid
Developing Java Applications for Sun Grid
Example Applications for Sun Grid
Cool tools for Sun Grid
Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 20
Login
2006 JavaOneSM
Conference | Session TS-1109 | 21
Help!
2006 JavaOneSM
Conference | Session TS-1109 | 22
The Code % vi HelloWorld.java
% javac HelloWorld.java
% vi helloworld.bash
% zip HelloWorld.zip HelloWorld.class helloworld.bash
/**
* This is the hello world application for JavaOne 2006
* The code is simple, to eliminate java as a point of confusion
*/
class HelloWorld
{
public static void main(String args[])
{
System.out.println("Hello JavaOne 2006 from the SunGrid ");
System.out.println("where you get a supercomputer for a $1.00");
}
}
#Script file to run the HelloWorld Java Application
#!/bin/sh
java HelloWorld
2006 JavaOneSM
Conference | Session TS-1109 | 23
Upload a Resource
2006 JavaOneSM
Conference | Session TS-1109 | 24
The Resource Exists!
2006 JavaOneSM
Conference | Session TS-1109 | 25
Create a Job
2006 JavaOneSM
Conference | Session TS-1109 | 26
Job Exists!
2006 JavaOneSM
Conference | Session TS-1109 | 27
Run It
2006 JavaOneSM
Conference | Session TS-1109 | 28
Run Status Pending/Started
2006 JavaOneSM
Conference | Session TS-1109 | 29
Run Finished
2006 JavaOneSM
Conference | Session TS-1109 | 30
Run Details
2006 JavaOneSM
Conference | Session TS-1109 | 31
Get the Output
2006 JavaOneSM
Conference | Session TS-1109 | 32
What Comes Back?
2006 JavaOneSM
Conference | Session TS-1109 | 33
StdOut
2006 JavaOneSM
Conference | Session TS-1109 | 34
The Sun Grid Compute Utility
Background/Context
What is the Sun Grid?
How to use the Sun Grid
Developing Java Applications for Sun Grid
Example Applications for Sun Grid
Cool tools for Sun Grid
Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 35
Java Applications for Sun Grid
● Parallel applications
● Numerous independent jobs (1000s)
● Run simultaneously
● Minimal cross-dependencies
● Types of parallel applications
● HPC applications
● Large data analysis
● These applications
● Are self-contained
● POJOs
2006 JavaOneSM
Conference | Session TS-1109 | 36
Creating a Java Grid Application
● Good candidates for parallel processing
● Compute intensive
● Time-consuming loops
● Need results faster
● How to parallelize an application
● Break loops down
● Create independent jobs to run simultaneously
● Partition data for each job
2006 JavaOneSM
Conference | Session TS-1109 | 37
Communication and Coordination
● Communication
● MPI (native libraries pre-installed)
● File system
● Applications read/write files
● NFS for sharing data
● RMI
● Coordination via N1 Grid Engine
● Jobs may have dependencies
● One job can wait for another
● Use qsub command with “hold_jid” option
2006 JavaOneSM
Conference | Session TS-1109 | 38
Develop and Pre-Test Locally
● Develop the application
● Write the code using your favorite IDE
● Include logs for debugging
● Use application-specific logging mechanism
● Write logs to the file system
● Package the Java application as JAR(s)
● Pre-Test the application (optional, but beneficial)
● Local grid with Solaris 10 and N1GE (open source)
● Develop and test the execution scripts
2006 JavaOneSM
Conference | Session TS-1109 | 39
The Sun Grid Compute Utility
● Background/Context
● What is the Sun Grid?
● How to use the Sun Grid
● Developing Java Applications for Sun Grid
● Example Applications for Sun Grid
● Cool tools for Sun Grid
● Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 40
File System Example Application
● Credit Card Processor
● Input file with transaction details for processing
● Partition processing and consolidate results
● Demonstrates typical processing model
● Available for download
● Sun Grid Community
● http://example-java-prog.developer.network.com/
● Source code and compiled binary
2006 JavaOneSM
Conference | Session TS-1109 | 41
CC Example—Flow of Execution
Input Text
Master Job
Parallel Job-1 Parallel Job-n
Output -1 Output -n
Consolidation Job
Final Result
Input Text-1 Input Text-n
2006 JavaOneSM
Conference | Session TS-1109 | 42
CC Example—Sun Grid Script
#!/usr/bin/bash
let NTASKS=$1
let COUNT=0
qsub -N step1 -b n java_step1.sh $NTASKS
while [ $COUNT -lt $NUMTASKS ] ; do
let COUNT=COUNT+1
qsub -N step2 -hold_jid step1 -b n java_step2.sh $COUNT
done
qsub -hold_jid step2 -b n final.sh CreditCardOutput $NTASKS
2006 JavaOneSM
Conference | Session TS-1109 | 43
CC Example—More Sun Grid Scripts
<java_step1.sh>
<java_step2.sh>
#!/usr/bin/bash
let NUMTASKS=$1
java -cp ccapp.jar com.sun.sungrid.sample.jog.CreditCardProcessor
CreditCardInfo.txt $NUMTASKS
#!/usr/bin/bash
let FNUM=$1
java -cp ccapp.jar com.sun.sungrid.sample.jog.CCParallelJobs
CreditCardInfo-$FNUM.txt CreditCardOutput-$FNUM.txt
2006 JavaOneSM
Conference | Session TS-1109 | 44
CC Example Java Code—Snippet 1
...
BufferedWriter out;
File file = new File("error.log");
PrintStream log = null;
log=new PrintStream(new FileOutputStream(file,true));
out = new BufferedWriter(new FileWriter("CCAppStatus.txt",true));
out.write("Welcome to SunGrid CreditCard Processor Sample App!");
out.write("This file contains the log");
out.write("Splitting the input file into multiple parts now... ");
out.close();
CCFileReader filereader=new CCFileReader();
filereader.divideFile(args[0], args[1]);
...
<CreditCardProcessor.java>
2006 JavaOneSM
Conference | Session TS-1109 | 45
CC Example Java Code—Snippet 2
...
PrintStream log=new PrintStream(new FileOutputStream(file,true));
BufferedWriter out = new BufferedWriter(new FileWriter(outfilename));
BufferedReader in = new BufferedReader(new FileReader(infilename));
while ((str = in.readLine()) != null) {
//process
out.write(“ ....”);
}
...
<CCParallelJobs.java>
2006 JavaOneSM
Conference | Session TS-1109 | 46
RMI Example Application
● RMISolver
● Server implements a mathematical solver
● Clients pass data object to server to be solved
● Demonstrates approach to locate the server
● Available for download
● Sun Grid Community
● http://examplermiprog.developer.network.com/
● Source code and compiled binary
2006 JavaOneSM
Conference | Session TS-1109 | 47
RMISolver Example—Classes
2006 JavaOneSM
Conference | Session TS-1109 | 48
RMISolver Example—Flow
2006 JavaOneSM
Conference | Session TS-1109 | 49
RMISolver Example—Scripts
2006 JavaOneSM
Conference | Session TS-1109 | 50
RMISolver Example—Main Script
svrResp=`qsub -N server startServer.ksh` # start the server
svrJobId=`echo "$svrResp" | awk '{print $3}'` # parse out the job id
status="not running"
until [ "$status" == "r" ]
do # loop until server job is running
status=`qstat | nawk '/'$svrJobId'/ {print $5}'`
sleep 10
done
filename="$HOME/serverhost"
until test -f $filename
do # loop until the hostname is there
sleep 10
done
servernode=`cat $filename` # fetch the server hostname
rm -f $filename # cleanup the file
qsub -N clients -t 1-$numClients startClient.ksh $servernode # start the clients
qsub -N stopServer -hold_jid clients stopServer.ksh $servernode # stop the server
2006 JavaOneSM
Conference | Session TS-1109 | 51
The Sun Grid Compute Utility
● Background/Context
● What is the Sun Grid?
● How to use the Sun Grid
● Developing Java Applications for Sun Grid
● Example Applications for Sun Grid
● Cool tools for Sun Grid
● Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 52
Why Tools for Sun Grid?
● Writing Sun Grid applications can get complex
● In addition to business logic, have to deal with:
● Application parallelization
● Infrastructure to distribute code and data
● Multi-threaded code (maybe)
● Programming model and application pattern(s)
● Scripts or DRMAA for N1 Grid Engine
● Remote debugging
2006 JavaOneSM
Conference | Session TS-1109 | 53
“Compute Server” Project Overview
● Java.net project that eases use of Sun Grid
● Programming model (Master/Worker pattern):
● Single-threaded tasks executed by workers
● Single-threaded task generator executed by master,
produces output and controls task generation
● Output processed off-grid
● Developer only has to write:
● The task class
● The task generator class
● The output processing code
● IDE integration
2006 JavaOneSM
Conference | Session TS-1109 | 54
Packaging and Execution Support
● Integrated Ant tasks for project functions:
● Build Project
● Debug Project
● Package Grid Resources
● Process Grid Output
● Properties
● Online help within Netbeans:
● “Compute Server Projects: Compute Server
Job Submission” subtopic
2006 JavaOneSM
Conference | Session TS-1109 | 55
Compute Server
Developer Experience
● Get SunGrid account
● Write the application-specific code
● Test locally to ensure correctness
● Specify key execution parameters
● Generate package for submission to Sun Grid
● Submit to Sun Grid and download results
● Executes output processing and displays results
IDE Supported Steps
2006 JavaOneSM
Conference | Session TS-1109 | 56
Additional Info
2006 JavaOneSM
Conference | Session TS-1109 | 57
More About Applications and Tools
● Structured approach for decomposition
● Patterns for examples shown here and more
● More on Compute Server, with a demo
● Unfortunately, no time here
● Will be covered in…
● TS-3117: Advanced Sun Grid—Creating
Applications for Horizontal Scale
● 4:30pm Today
1
2
2006 JavaOneSM
Conference | Session TS-1109 | 58
Sun Grid Futures
● Programmatic access
● Support for long-running services
● ISV models
● More...
Unfortunately, no time here
● Will be covered in…
● BOF-7995: What’s Next for Sun Grid?
● 10:30pm Today
ProgressionofServiceOfferings
Future
Today
2006 JavaOneSM
Conference | Session TS-1109 | 59
Q&A
2006 JavaOneSM
Conference | Session TS-1109 | 60
For More Information
JavaOne sessions
TS-3117: Advanced Sun Grid—Creating Applications
for Horizontal Scale
BOF-7995: What’s Next for Sun Grid?
Java.net Community Corner mini-sessions
Rapid Development of Sun Grid Applications using
Compute Server
Sun Grid as a Test-to-Scale facility
Sun Grid developer experience and API development
http://www.network.com
http://developer.network.com
http://computeserver.dev.java.net
http://gridengine.sunsource.net
http://www.sun.com/service/crs
2006 JavaOneSM
Conference | Session TS-1109 |
TS-1109
The Sun Grid
Compute Utility
A Tutorial for Java Developers
Fay Salwen
Peter A. Murray
Uday Subbarayan
Staff Engineers, Sun Grid
Sun Microsystems, Inc.

More Related Content

What's hot

Quantifying the Noisy Neighbor Problem in Openstack
Quantifying the Noisy Neighbor Problem in OpenstackQuantifying the Noisy Neighbor Problem in Openstack
Quantifying the Noisy Neighbor Problem in Openstack
Nodir Kodirov
 
Red Hat Cloud Infrastructure Conference 2013 - Presentation about OpenStack ...
Red Hat Cloud Infrastructure Conference 2013 -  Presentation about OpenStack ...Red Hat Cloud Infrastructure Conference 2013 -  Presentation about OpenStack ...
Red Hat Cloud Infrastructure Conference 2013 - Presentation about OpenStack ...
Elos Technologies s.r.o.
 
OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...
OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...
OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...
NETWAYS
 
Docker Workshop
Docker WorkshopDocker Workshop
Docker Workshop
Alex Baretto
 
Episode 2 Installation Triton Slides
Episode 2 Installation Triton SlidesEpisode 2 Installation Triton Slides
Episode 2 Installation Triton Slides
Laura Hood
 
What's New with Ceph - Ceph Day Silicon Valley
What's New with Ceph - Ceph Day Silicon ValleyWhat's New with Ceph - Ceph Day Silicon Valley
What's New with Ceph - Ceph Day Silicon Valley
Ceph Community
 
GIST AI-X Computing Cluster
GIST AI-X Computing ClusterGIST AI-X Computing Cluster
GIST AI-X Computing Cluster
Jax Jargalsaikhan
 
My personal journey through the World of Open Source! How What Was Old Beco...
My personal journey through  the World of Open Source!  How What Was Old Beco...My personal journey through  the World of Open Source!  How What Was Old Beco...
My personal journey through the World of Open Source! How What Was Old Beco...
Ceph Community
 
A Container Stack for Openstack - OpenStack Silicon Valley
A Container Stack for Openstack - OpenStack Silicon ValleyA Container Stack for Openstack - OpenStack Silicon Valley
A Container Stack for Openstack - OpenStack Silicon Valley
Stephen Gordon
 
Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...
Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...
Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...
Citrix
 
OpenNebula TechDay Boston 2015 - introduction and architecture
OpenNebula TechDay Boston 2015 - introduction and architectureOpenNebula TechDay Boston 2015 - introduction and architecture
OpenNebula TechDay Boston 2015 - introduction and architecture
OpenNebula Project
 
Part 2 Maximizing the utilization of GPU resources on-premise and in the cloud
Part 2   Maximizing the utilization of GPU resources on-premise and in the cloudPart 2   Maximizing the utilization of GPU resources on-premise and in the cloud
Part 2 Maximizing the utilization of GPU resources on-premise and in the cloud
Univa, an Altair Company
 
ceph openstack dream team
ceph openstack dream teamceph openstack dream team
ceph openstack dream team
Udo Seidel
 
NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...
NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...
NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...
Ceph Community
 
Apps software development with Vert.X
Apps software development with Vert.XApps software development with Vert.X
Apps software development with Vert.X
Jose Juan R. Zuñiga
 
Openstack platform -Red Hat Pizza and technology event - Israel
Openstack platform -Red Hat Pizza and technology event - IsraelOpenstack platform -Red Hat Pizza and technology event - Israel
Openstack platform -Red Hat Pizza and technology event - Israel
Arthur Berezin
 
OCDET Activity and Glusterfs
OCDET Activity and GlusterfsOCDET Activity and Glusterfs
OCDET Activity and Glusterfs
Masanori Itoh
 
OpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebula
OpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebulaOpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebula
OpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebula
OpenNebula Project
 
Introduction to CloudStack Storage Subsystem
Introduction to CloudStack Storage SubsystemIntroduction to CloudStack Storage Subsystem
Introduction to CloudStack Storage Subsystem
buildacloud
 
NantOmics
NantOmicsNantOmics
NantOmics
Ceph Community
 

What's hot (20)

Quantifying the Noisy Neighbor Problem in Openstack
Quantifying the Noisy Neighbor Problem in OpenstackQuantifying the Noisy Neighbor Problem in Openstack
Quantifying the Noisy Neighbor Problem in Openstack
 
Red Hat Cloud Infrastructure Conference 2013 - Presentation about OpenStack ...
Red Hat Cloud Infrastructure Conference 2013 -  Presentation about OpenStack ...Red Hat Cloud Infrastructure Conference 2013 -  Presentation about OpenStack ...
Red Hat Cloud Infrastructure Conference 2013 - Presentation about OpenStack ...
 
OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...
OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...
OpenNebula Conf 2014 | OpenNebula and MooseFS for disaster recovery: real clo...
 
Docker Workshop
Docker WorkshopDocker Workshop
Docker Workshop
 
Episode 2 Installation Triton Slides
Episode 2 Installation Triton SlidesEpisode 2 Installation Triton Slides
Episode 2 Installation Triton Slides
 
What's New with Ceph - Ceph Day Silicon Valley
What's New with Ceph - Ceph Day Silicon ValleyWhat's New with Ceph - Ceph Day Silicon Valley
What's New with Ceph - Ceph Day Silicon Valley
 
GIST AI-X Computing Cluster
GIST AI-X Computing ClusterGIST AI-X Computing Cluster
GIST AI-X Computing Cluster
 
My personal journey through the World of Open Source! How What Was Old Beco...
My personal journey through  the World of Open Source!  How What Was Old Beco...My personal journey through  the World of Open Source!  How What Was Old Beco...
My personal journey through the World of Open Source! How What Was Old Beco...
 
A Container Stack for Openstack - OpenStack Silicon Valley
A Container Stack for Openstack - OpenStack Silicon ValleyA Container Stack for Openstack - OpenStack Silicon Valley
A Container Stack for Openstack - OpenStack Silicon Valley
 
Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...
Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...
Citrix Synergy 2014 - Syn232 Building a Cloud Architecture and Self- Service ...
 
OpenNebula TechDay Boston 2015 - introduction and architecture
OpenNebula TechDay Boston 2015 - introduction and architectureOpenNebula TechDay Boston 2015 - introduction and architecture
OpenNebula TechDay Boston 2015 - introduction and architecture
 
Part 2 Maximizing the utilization of GPU resources on-premise and in the cloud
Part 2   Maximizing the utilization of GPU resources on-premise and in the cloudPart 2   Maximizing the utilization of GPU resources on-premise and in the cloud
Part 2 Maximizing the utilization of GPU resources on-premise and in the cloud
 
ceph openstack dream team
ceph openstack dream teamceph openstack dream team
ceph openstack dream team
 
NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...
NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...
NVMe over Fabrics and Composable Infrastructure - What Do They Mean for Softw...
 
Apps software development with Vert.X
Apps software development with Vert.XApps software development with Vert.X
Apps software development with Vert.X
 
Openstack platform -Red Hat Pizza and technology event - Israel
Openstack platform -Red Hat Pizza and technology event - IsraelOpenstack platform -Red Hat Pizza and technology event - Israel
Openstack platform -Red Hat Pizza and technology event - Israel
 
OCDET Activity and Glusterfs
OCDET Activity and GlusterfsOCDET Activity and Glusterfs
OCDET Activity and Glusterfs
 
OpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebula
OpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebulaOpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebula
OpenNebula TechDay Boston 2015 - Hyperconvergence and OpenNebula
 
Introduction to CloudStack Storage Subsystem
Introduction to CloudStack Storage SubsystemIntroduction to CloudStack Storage Subsystem
Introduction to CloudStack Storage Subsystem
 
NantOmics
NantOmicsNantOmics
NantOmics
 

Similar to SunGrid: Cloud Computing

The TIMES Cloud Service & the TIMES/MIRO App
The TIMES Cloud Service & the TIMES/MIRO AppThe TIMES Cloud Service & the TIMES/MIRO App
The TIMES Cloud Service & the TIMES/MIRO App
IEA-ETSAP
 
Node.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleNode.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scale
Dmytro Semenov
 
Automated Deployment and Management of Edge Clouds
Automated Deployment and Management of Edge CloudsAutomated Deployment and Management of Edge Clouds
Automated Deployment and Management of Edge Clouds
Jay Bryant
 
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
In-Memory Computing Summit
 
Red Hat Java Update and Quarkus Introduction
Red Hat Java Update and Quarkus IntroductionRed Hat Java Update and Quarkus Introduction
Red Hat Java Update and Quarkus Introduction
John Archer
 
JITServerTalk JCON World 2023.pdf
JITServerTalk JCON World 2023.pdfJITServerTalk JCON World 2023.pdf
JITServerTalk JCON World 2023.pdf
RichHagarty
 
20141111_SOS3_Gallo
20141111_SOS3_Gallo20141111_SOS3_Gallo
20141111_SOS3_Gallo
Andrea Gallo
 
SolarWinds Scalability for the Enterprise
SolarWinds Scalability for the EnterpriseSolarWinds Scalability for the Enterprise
SolarWinds Scalability for the Enterprise
SolarWinds
 
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageWebinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
MayaData Inc
 
Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Source Investments in Mainframe Through the Next Generation - Showcasing...Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Mainframe Project
 
Academy PRO: Node.js platform. Lecture 1
Academy PRO: Node.js platform. Lecture 1  Academy PRO: Node.js platform. Lecture 1
Academy PRO: Node.js platform. Lecture 1
Binary Studio
 
DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...
DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...
DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...
Haggai Philip Zagury
 
SemeruRuntimesUnderTheCover .pptx
SemeruRuntimesUnderTheCover .pptxSemeruRuntimesUnderTheCover .pptx
SemeruRuntimesUnderTheCover .pptx
SumanMitra22
 
Node.js Test
Node.js TestNode.js Test
Node.js Test
Maksym Kovalko
 
Nodejs
NodejsNodejs
New Product Release Roundup: SAM, NPM, & VNQM
New Product Release Roundup: SAM, NPM, & VNQMNew Product Release Roundup: SAM, NPM, & VNQM
New Product Release Roundup: SAM, NPM, & VNQM
SolarWinds
 
CON6423: Scalable JavaScript applications with Project Nashorn
CON6423: Scalable JavaScript applications with Project NashornCON6423: Scalable JavaScript applications with Project Nashorn
CON6423: Scalable JavaScript applications with Project Nashorn
Michel Graciano
 
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Jakarta_EE
 
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
The Eclipse Foundation
 
Splunking the JVM
Splunking the JVMSplunking the JVM
Splunking the JVM
Damien Dallimore
 

Similar to SunGrid: Cloud Computing (20)

The TIMES Cloud Service & the TIMES/MIRO App
The TIMES Cloud Service & the TIMES/MIRO AppThe TIMES Cloud Service & the TIMES/MIRO App
The TIMES Cloud Service & the TIMES/MIRO App
 
Node.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scaleNode.js Web Apps @ ebay scale
Node.js Web Apps @ ebay scale
 
Automated Deployment and Management of Edge Clouds
Automated Deployment and Management of Edge CloudsAutomated Deployment and Management of Edge Clouds
Automated Deployment and Management of Edge Clouds
 
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
IMC Summit 2016 Breakout - Brian Bulkowski - NVMe, Storage Class Memory and O...
 
Red Hat Java Update and Quarkus Introduction
Red Hat Java Update and Quarkus IntroductionRed Hat Java Update and Quarkus Introduction
Red Hat Java Update and Quarkus Introduction
 
JITServerTalk JCON World 2023.pdf
JITServerTalk JCON World 2023.pdfJITServerTalk JCON World 2023.pdf
JITServerTalk JCON World 2023.pdf
 
20141111_SOS3_Gallo
20141111_SOS3_Gallo20141111_SOS3_Gallo
20141111_SOS3_Gallo
 
SolarWinds Scalability for the Enterprise
SolarWinds Scalability for the EnterpriseSolarWinds Scalability for the Enterprise
SolarWinds Scalability for the Enterprise
 
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storageWebinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
Webinar: OpenEBS - Still Free and now FASTEST Kubernetes storage
 
Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Source Investments in Mainframe Through the Next Generation - Showcasing...Open Source Investments in Mainframe Through the Next Generation - Showcasing...
Open Source Investments in Mainframe Through the Next Generation - Showcasing...
 
Academy PRO: Node.js platform. Lecture 1
Academy PRO: Node.js platform. Lecture 1  Academy PRO: Node.js platform. Lecture 1
Academy PRO: Node.js platform. Lecture 1
 
DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...
DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...
DevOpsDays Tel Aviv DEC 2022 | Building A Cloud-Native Platform Brick by Bric...
 
SemeruRuntimesUnderTheCover .pptx
SemeruRuntimesUnderTheCover .pptxSemeruRuntimesUnderTheCover .pptx
SemeruRuntimesUnderTheCover .pptx
 
Node.js Test
Node.js TestNode.js Test
Node.js Test
 
Nodejs
NodejsNodejs
Nodejs
 
New Product Release Roundup: SAM, NPM, & VNQM
New Product Release Roundup: SAM, NPM, & VNQMNew Product Release Roundup: SAM, NPM, & VNQM
New Product Release Roundup: SAM, NPM, & VNQM
 
CON6423: Scalable JavaScript applications with Project Nashorn
CON6423: Scalable JavaScript applications with Project NashornCON6423: Scalable JavaScript applications with Project Nashorn
CON6423: Scalable JavaScript applications with Project Nashorn
 
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
 
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
Kubernetes Native Java and Eclipse MicroProfile | EclipseCon Europe 2019
 
Splunking the JVM
Splunking the JVMSplunking the JVM
Splunking the JVM
 

Recently uploaded

GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Green Software Development
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
ICS
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
Aftab Hussain
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
Peter Muessig
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Łukasz Chruściel
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Crescat
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Julian Hyde
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
Grant Fritchey
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
brainerhub1
 
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise EditionWhy Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Envertis Software Solutions
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
lorraineandreiamcidl
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
rodomar2
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Neo4j
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
Ayan Halder
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
Remote DBA Services
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 

Recently uploaded (20)

GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
 
OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024OpenMetadata Community Meeting - 5th June 2024
OpenMetadata Community Meeting - 5th June 2024
 
Webinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for EmbeddedWebinar On-Demand: Using Flutter for Embedded
Webinar On-Demand: Using Flutter for Embedded
 
Graspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code AnalysisGraspan: A Big Data System for Big Code Analysis
Graspan: A Big Data System for Big Code Analysis
 
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling ExtensionsUI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
UI5con 2024 - Boost Your Development Experience with UI5 Tooling Extensions
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
Vitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdfVitthal Shirke Java Microservices Resume.pdf
Vitthal Shirke Java Microservices Resume.pdf
 
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Need for Speed: Removing speed bumps from your Symfony projects ⚡️
Need for Speed: Removing speed bumps from your Symfony projects ⚡️
 
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
Introducing Crescat - Event Management Software for Venues, Festivals and Eve...
 
Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)Measures in SQL (SIGMOD 2024, Santiago, Chile)
Measures in SQL (SIGMOD 2024, Santiago, Chile)
 
Using Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query PerformanceUsing Query Store in Azure PostgreSQL to Understand Query Performance
Using Query Store in Azure PostgreSQL to Understand Query Performance
 
Unveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdfUnveiling the Advantages of Agile Software Development.pdf
Unveiling the Advantages of Agile Software Development.pdf
 
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise EditionWhy Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
Why Choose Odoo 17 Community & How it differs from Odoo 17 Enterprise Edition
 
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOMLORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM
 
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CDKuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
KuberTENes Birthday Bash Guadalajara - Introducción a Argo CD
 
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissancesAtelier - Innover avec l’IA Générative et les graphes de connaissances
Atelier - Innover avec l’IA Générative et les graphes de connaissances
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
Using Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional SafetyUsing Xen Hypervisor for Functional Safety
Using Xen Hypervisor for Functional Safety
 
Oracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptxOracle Database 19c New Features for DBAs and Developers.pptx
Oracle Database 19c New Features for DBAs and Developers.pptx
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 

SunGrid: Cloud Computing

  • 1. 2006 JavaOneSM Conference | Session TS-1109 | TS-1109 The Sun Grid Compute Utility A Tutorial for Java Developers Fay Salwen Peter A. Murray Uday Subbarayan Staff Engineers, Sun Grid Sun Microsystems, Inc. Copyright © 2006, Sun Microsystems, Inc., All rights reserved.
  • 2. 2006 JavaOneSM Conference | Session TS-1109 | 2 The Sun Grid Compute Utility Learn how to leverage 1000s of CPUs on demand. A Tutorial for Java Developers
  • 3. 2006 JavaOneSM Conference | Session TS-1109 | 3 Agenda Sun Grid Backgrounder ● What are Grid and Utility computing all about? ● Who are using them and for what? What is the Sun Grid? ● What makes it so special and why would I want to use it? How to use the Sun Grid ● Basic concepts Developing Java Applications for Sun Grid ● How to leverage 1000s of CPUs Example Applications for Sun Grid ● The best way to get started Cool tools for Sun Grid ● To make make your life easier
  • 4. 2006 JavaOneSM Conference | Session TS-1109 | 4 The Sun Grid Compute Utility Sun Grid Backgrounder What is the Sun Grid? How to use the Sun Grid Developing Java Applications for Sun Grid Example Applications for Sun Grid Cool tools for Sun Grid Q&A
  • 5. 2006 JavaOneSM Conference | Session TS-1109 | 5 Definitions of a Grid ● No single definition ● Each company and individual have their own ● Wikipedia has no less than 7 ● Some examples ● Major DB vendor defines it as—Pooling all your resources into a central location for optimization ● Major HW manufacturer defines it as—The virtualization of compute and storage resources ● One of the largest grid organizations says—The grid will be defined when the question of what a grid is, no longer exists
  • 6. 2006 JavaOneSM Conference | Session TS-1109 | 6 Lowest Common Denominator ● A grid has lots of CPUs, storage, and virtualization ● Everyone says it/they will save you $$$, somehow ● Most vendors/users are afraid of multi tenancy, except with “friendly” users (is there such a thing?)
  • 7. 2006 JavaOneSM Conference | Session TS-1109 | 7 What Is Grid Computing Used For? Grid Computing is not new, it has been used for years Risk and portfolio analysis, Monte Carlo simulations EDA, MCAD, fluid dynamics, crash-test simulations, aerodynamics modeling Digital content creation, animation, digital asset management Reservoir simulations, seismic processing Weather analysis, nuclear yield simulation Financial Services Manufacturing Entertainment/Media Energy Government/Edu Industries Computing Tasks
  • 8. 2006 JavaOneSM Conference | Session TS-1109 | 8 Benefits of Grid Computing ● Grid Computing allows organizations to: ● Pool systems, manage them as a common resource ● Increasing utilization ● Reduce costs ● Increase agility through dynamic resource allocation
  • 9. 2006 JavaOneSM Conference | Session TS-1109 | 9 Financing ServicesServices UtilityUtility ComputingComputing TechnologyTechnology The ability to intelligently match IT resources to business demand on a pay-for-use basis. Grid Computing != Utility Computing Attributes ● Multi-tenancy ● Standardization ● Scale ● Automation ● Immediate Provisioning ● Granular Costing Utility Computing (UC)—
  • 10. 2006 JavaOneSM Conference | Session TS-1109 | 10 The Move From Custom to Utility Models Standardize/ Aggregate Customize Utilize $30/mo $.05/kWh
  • 11. 2006 JavaOneSM Conference | Session TS-1109 | 11 The Sun Grid Compute Utility Background/Context What is the Sun Grid? How to use the Sun Grid Developing Java Applications for Sun Grid Example Applications for Sun Grid Cool tools for Sun Grid Q&A
  • 12. 2006 JavaOneSM Conference | Session TS-1109 | 12 The Move to Utility in Grid Computing $1/CPUhr Standardize/ Aggregate Customize Utilize
  • 13. 2006 JavaOneSM Conference | Session TS-1109 | 13 Grid Technology Utility Model What Is Sun Grid Compute Utility? ● Pay-per-use compute power ● Standard pricing—$1/cpu-hour ● No contract or minimum commitment ● Racks of compute nodes ● High-speed network interconnects ● Shared storage ● Distributed Resource Management (DRM) Readily available in a…
  • 14. 2006 JavaOneSM Conference | Session TS-1109 | 14 Sun Grid Access ● Web-based user interface ● Internet-accessible (www.network.com) ● Available to anybody (US only, at this time) ● No direct Internet access to compute nodes
  • 15. 2006 JavaOneSM Conference | Session TS-1109 | 15 Access to What? ● Sun Fire V20z™ servers, each containing: ● Dual 2.4 GHz AMD Opteron™ processors with HyperTransport technology for memory and I/O interface ● 8 GByte RAM ● Solaris™ 10 Operating System ● Storage ● Up to 10 GBytes per user in staging area ● NFS-mounted home directory in execution environment ● Sun N1™ Grid Engine 6 software ● Grid network infrastructure built on a Gbit/s switched-Ethernet data network
  • 16. 2006 JavaOneSM Conference | Session TS-1109 | 16 Types of Applications ● Batch/Grid ● Get job done faster ● Data Transformations, such as: ● Extract Transform Load (ETL) ● Document processing ● Computations and Simulations, e.g.: ● Digital media processing (renderings, animations, etc.) ● Financial calculations ● Scenario simulations ● Business Intelligence and Analytics
  • 17. 2006 JavaOneSM Conference | Session TS-1109 | 17 Who Is Interested? ● Individuals/businesses: ● With variable loads ● Where Time = $ ● That are small (e.g., start-ups) ● Anyone who wants to: ● Avoid having to purchase and manage HW/SW ● Avoid having to scale for the maximum ● Get cycles when they need them ● Pay a known cost ● Independent Software Vendors (ISVs) Datacenter Utilization (Conceptual) Peak Std
  • 18. 2006 JavaOneSM Conference | Session TS-1109 | 18 Sun Grid Usage and ISV Model ISVs Application/Data Sun Grid Compute Utility Internet Users
  • 19. 2006 JavaOneSM Conference | Session TS-1109 | 19 The Sun Grid Compute Utility Background/Context What is the Sun Grid? How to use the Sun Grid Developing Java Applications for Sun Grid Example Applications for Sun Grid Cool tools for Sun Grid Q&A
  • 20. 2006 JavaOneSM Conference | Session TS-1109 | 20 Login
  • 21. 2006 JavaOneSM Conference | Session TS-1109 | 21 Help!
  • 22. 2006 JavaOneSM Conference | Session TS-1109 | 22 The Code % vi HelloWorld.java % javac HelloWorld.java % vi helloworld.bash % zip HelloWorld.zip HelloWorld.class helloworld.bash /** * This is the hello world application for JavaOne 2006 * The code is simple, to eliminate java as a point of confusion */ class HelloWorld { public static void main(String args[]) { System.out.println("Hello JavaOne 2006 from the SunGrid "); System.out.println("where you get a supercomputer for a $1.00"); } } #Script file to run the HelloWorld Java Application #!/bin/sh java HelloWorld
  • 23. 2006 JavaOneSM Conference | Session TS-1109 | 23 Upload a Resource
  • 24. 2006 JavaOneSM Conference | Session TS-1109 | 24 The Resource Exists!
  • 25. 2006 JavaOneSM Conference | Session TS-1109 | 25 Create a Job
  • 26. 2006 JavaOneSM Conference | Session TS-1109 | 26 Job Exists!
  • 27. 2006 JavaOneSM Conference | Session TS-1109 | 27 Run It
  • 28. 2006 JavaOneSM Conference | Session TS-1109 | 28 Run Status Pending/Started
  • 29. 2006 JavaOneSM Conference | Session TS-1109 | 29 Run Finished
  • 30. 2006 JavaOneSM Conference | Session TS-1109 | 30 Run Details
  • 31. 2006 JavaOneSM Conference | Session TS-1109 | 31 Get the Output
  • 32. 2006 JavaOneSM Conference | Session TS-1109 | 32 What Comes Back?
  • 33. 2006 JavaOneSM Conference | Session TS-1109 | 33 StdOut
  • 34. 2006 JavaOneSM Conference | Session TS-1109 | 34 The Sun Grid Compute Utility Background/Context What is the Sun Grid? How to use the Sun Grid Developing Java Applications for Sun Grid Example Applications for Sun Grid Cool tools for Sun Grid Q&A
  • 35. 2006 JavaOneSM Conference | Session TS-1109 | 35 Java Applications for Sun Grid ● Parallel applications ● Numerous independent jobs (1000s) ● Run simultaneously ● Minimal cross-dependencies ● Types of parallel applications ● HPC applications ● Large data analysis ● These applications ● Are self-contained ● POJOs
  • 36. 2006 JavaOneSM Conference | Session TS-1109 | 36 Creating a Java Grid Application ● Good candidates for parallel processing ● Compute intensive ● Time-consuming loops ● Need results faster ● How to parallelize an application ● Break loops down ● Create independent jobs to run simultaneously ● Partition data for each job
  • 37. 2006 JavaOneSM Conference | Session TS-1109 | 37 Communication and Coordination ● Communication ● MPI (native libraries pre-installed) ● File system ● Applications read/write files ● NFS for sharing data ● RMI ● Coordination via N1 Grid Engine ● Jobs may have dependencies ● One job can wait for another ● Use qsub command with “hold_jid” option
  • 38. 2006 JavaOneSM Conference | Session TS-1109 | 38 Develop and Pre-Test Locally ● Develop the application ● Write the code using your favorite IDE ● Include logs for debugging ● Use application-specific logging mechanism ● Write logs to the file system ● Package the Java application as JAR(s) ● Pre-Test the application (optional, but beneficial) ● Local grid with Solaris 10 and N1GE (open source) ● Develop and test the execution scripts
  • 39. 2006 JavaOneSM Conference | Session TS-1109 | 39 The Sun Grid Compute Utility ● Background/Context ● What is the Sun Grid? ● How to use the Sun Grid ● Developing Java Applications for Sun Grid ● Example Applications for Sun Grid ● Cool tools for Sun Grid ● Q&A
  • 40. 2006 JavaOneSM Conference | Session TS-1109 | 40 File System Example Application ● Credit Card Processor ● Input file with transaction details for processing ● Partition processing and consolidate results ● Demonstrates typical processing model ● Available for download ● Sun Grid Community ● http://example-java-prog.developer.network.com/ ● Source code and compiled binary
  • 41. 2006 JavaOneSM Conference | Session TS-1109 | 41 CC Example—Flow of Execution Input Text Master Job Parallel Job-1 Parallel Job-n Output -1 Output -n Consolidation Job Final Result Input Text-1 Input Text-n
  • 42. 2006 JavaOneSM Conference | Session TS-1109 | 42 CC Example—Sun Grid Script #!/usr/bin/bash let NTASKS=$1 let COUNT=0 qsub -N step1 -b n java_step1.sh $NTASKS while [ $COUNT -lt $NUMTASKS ] ; do let COUNT=COUNT+1 qsub -N step2 -hold_jid step1 -b n java_step2.sh $COUNT done qsub -hold_jid step2 -b n final.sh CreditCardOutput $NTASKS
  • 43. 2006 JavaOneSM Conference | Session TS-1109 | 43 CC Example—More Sun Grid Scripts <java_step1.sh> <java_step2.sh> #!/usr/bin/bash let NUMTASKS=$1 java -cp ccapp.jar com.sun.sungrid.sample.jog.CreditCardProcessor CreditCardInfo.txt $NUMTASKS #!/usr/bin/bash let FNUM=$1 java -cp ccapp.jar com.sun.sungrid.sample.jog.CCParallelJobs CreditCardInfo-$FNUM.txt CreditCardOutput-$FNUM.txt
  • 44. 2006 JavaOneSM Conference | Session TS-1109 | 44 CC Example Java Code—Snippet 1 ... BufferedWriter out; File file = new File("error.log"); PrintStream log = null; log=new PrintStream(new FileOutputStream(file,true)); out = new BufferedWriter(new FileWriter("CCAppStatus.txt",true)); out.write("Welcome to SunGrid CreditCard Processor Sample App!"); out.write("This file contains the log"); out.write("Splitting the input file into multiple parts now... "); out.close(); CCFileReader filereader=new CCFileReader(); filereader.divideFile(args[0], args[1]); ... <CreditCardProcessor.java>
  • 45. 2006 JavaOneSM Conference | Session TS-1109 | 45 CC Example Java Code—Snippet 2 ... PrintStream log=new PrintStream(new FileOutputStream(file,true)); BufferedWriter out = new BufferedWriter(new FileWriter(outfilename)); BufferedReader in = new BufferedReader(new FileReader(infilename)); while ((str = in.readLine()) != null) { //process out.write(“ ....”); } ... <CCParallelJobs.java>
  • 46. 2006 JavaOneSM Conference | Session TS-1109 | 46 RMI Example Application ● RMISolver ● Server implements a mathematical solver ● Clients pass data object to server to be solved ● Demonstrates approach to locate the server ● Available for download ● Sun Grid Community ● http://examplermiprog.developer.network.com/ ● Source code and compiled binary
  • 47. 2006 JavaOneSM Conference | Session TS-1109 | 47 RMISolver Example—Classes
  • 48. 2006 JavaOneSM Conference | Session TS-1109 | 48 RMISolver Example—Flow
  • 49. 2006 JavaOneSM Conference | Session TS-1109 | 49 RMISolver Example—Scripts
  • 50. 2006 JavaOneSM Conference | Session TS-1109 | 50 RMISolver Example—Main Script svrResp=`qsub -N server startServer.ksh` # start the server svrJobId=`echo "$svrResp" | awk '{print $3}'` # parse out the job id status="not running" until [ "$status" == "r" ] do # loop until server job is running status=`qstat | nawk '/'$svrJobId'/ {print $5}'` sleep 10 done filename="$HOME/serverhost" until test -f $filename do # loop until the hostname is there sleep 10 done servernode=`cat $filename` # fetch the server hostname rm -f $filename # cleanup the file qsub -N clients -t 1-$numClients startClient.ksh $servernode # start the clients qsub -N stopServer -hold_jid clients stopServer.ksh $servernode # stop the server
  • 51. 2006 JavaOneSM Conference | Session TS-1109 | 51 The Sun Grid Compute Utility ● Background/Context ● What is the Sun Grid? ● How to use the Sun Grid ● Developing Java Applications for Sun Grid ● Example Applications for Sun Grid ● Cool tools for Sun Grid ● Q&A
  • 52. 2006 JavaOneSM Conference | Session TS-1109 | 52 Why Tools for Sun Grid? ● Writing Sun Grid applications can get complex ● In addition to business logic, have to deal with: ● Application parallelization ● Infrastructure to distribute code and data ● Multi-threaded code (maybe) ● Programming model and application pattern(s) ● Scripts or DRMAA for N1 Grid Engine ● Remote debugging
  • 53. 2006 JavaOneSM Conference | Session TS-1109 | 53 “Compute Server” Project Overview ● Java.net project that eases use of Sun Grid ● Programming model (Master/Worker pattern): ● Single-threaded tasks executed by workers ● Single-threaded task generator executed by master, produces output and controls task generation ● Output processed off-grid ● Developer only has to write: ● The task class ● The task generator class ● The output processing code ● IDE integration
  • 54. 2006 JavaOneSM Conference | Session TS-1109 | 54 Packaging and Execution Support ● Integrated Ant tasks for project functions: ● Build Project ● Debug Project ● Package Grid Resources ● Process Grid Output ● Properties ● Online help within Netbeans: ● “Compute Server Projects: Compute Server Job Submission” subtopic
  • 55. 2006 JavaOneSM Conference | Session TS-1109 | 55 Compute Server Developer Experience ● Get SunGrid account ● Write the application-specific code ● Test locally to ensure correctness ● Specify key execution parameters ● Generate package for submission to Sun Grid ● Submit to Sun Grid and download results ● Executes output processing and displays results IDE Supported Steps
  • 56. 2006 JavaOneSM Conference | Session TS-1109 | 56 Additional Info
  • 57. 2006 JavaOneSM Conference | Session TS-1109 | 57 More About Applications and Tools ● Structured approach for decomposition ● Patterns for examples shown here and more ● More on Compute Server, with a demo ● Unfortunately, no time here ● Will be covered in… ● TS-3117: Advanced Sun Grid—Creating Applications for Horizontal Scale ● 4:30pm Today 1 2
  • 58. 2006 JavaOneSM Conference | Session TS-1109 | 58 Sun Grid Futures ● Programmatic access ● Support for long-running services ● ISV models ● More... Unfortunately, no time here ● Will be covered in… ● BOF-7995: What’s Next for Sun Grid? ● 10:30pm Today ProgressionofServiceOfferings Future Today
  • 59. 2006 JavaOneSM Conference | Session TS-1109 | 59 Q&A
  • 60. 2006 JavaOneSM Conference | Session TS-1109 | 60 For More Information JavaOne sessions TS-3117: Advanced Sun Grid—Creating Applications for Horizontal Scale BOF-7995: What’s Next for Sun Grid? Java.net Community Corner mini-sessions Rapid Development of Sun Grid Applications using Compute Server Sun Grid as a Test-to-Scale facility Sun Grid developer experience and API development http://www.network.com http://developer.network.com http://computeserver.dev.java.net http://gridengine.sunsource.net http://www.sun.com/service/crs
  • 61. 2006 JavaOneSM Conference | Session TS-1109 | TS-1109 The Sun Grid Compute Utility A Tutorial for Java Developers Fay Salwen Peter A. Murray Uday Subbarayan Staff Engineers, Sun Grid Sun Microsystems, Inc.