View - VMworld.com: Virtualization: Virtual Conference

Workshop progetto CyberSAR, december 4/6 th 2007
Using VMware Windows virtual machines in a CERN GRID environment at CyberSAR (Sardinia, Italy)
michelangelo puliga ( [email_address] )
dipartimento di Matematica ed Informatica,Università di Cagliari

Summary
Virtualization technologies
Task automation on Windows
VMware on the CERN grid of CyberSAR infrastructure
Conclusions

Virtualization technology
Definitions
Logical rings: (hardware = ring 0 wrapped by kernel (ring 1) wrapped by shell (ring 2)
Hypervisor: (virtual machine monitor process):
the main process that control the operations from and to the virtual machine
A brief History:
First tools running directly on a special hardware (IBM s/370)
First running example of software virtualization in a x86 env: VMware virtual platform (1999)‏
ParaVirtualization XEN (first example of virtualization with hypervisor running at kernel level (ring 1)‏
New perspectives with VT (Intel) and Pacifica (Amd) virtual machine software dedicated processor technology

VMware, Xen, VirtualBox solutions
Paravirtualization: XEN.
Advantages: Performances. Open Source, Optimal support to Live migration
Disadvantages: Needs a modified Linux kernel, support for 64 bit architectures is currently poor.
VMware
Advantages: Reliability, easy management. Desktop products with nice and powerful interfaces. Development kit
Disadvantages: Almost completely closed source. Networking not flexible in basic products
VirtualBox
Advantages : Open Source, simple and powerful networking structure. Desktop ready
Disadvantages: No high level API

Task automation on Unix and Windows
Unix automation.
Operating system built for automation (The power is in the shell)‏
Several languages for system management and automation (perl and, more recently, python)‏
Example problem:
How to rename thousands of files adding to each file name the date string and a progressive index ?
PERL or SHELL solution:
10 lines of code, and 2 minutes to write the code
Windows native solution: is there one ?
DOS (#!>%&) NO
consider the ”infamous” jscript or vbscript or 3 rd part software
#!/usr/bin/perl # @list_files = ` ls `; $date = ` date +"%m%d%Y"`; $num_files = @list_files; for ($i = 0; $i < $num_files; $i++)‏ { $fname = $list_files[$i]; $fdest = $fname.$date.$i; ` mv $fname $fdest` }

Task automation on Windows.
Until 2008 (longhorn) no complete powershell : in all Windows operating systems some administrative tasks are possible only from the graphic interface
On WinXP and Vista is present the windows scripting host (WSH) tool, with Jscript or VB syntax.
Advantages: WSH can explore and interact with the entire OS with little restrictions (i.e.manipulating windows registry keys).
Disadvantages: security issues. Wsh is the ideal tool for virus coders.

Software automation on Windows. High level software automation with COM, OLE and ActiveX Modern tools: DCOM and .NET Advantages: Very powerful graphic interfaces, excellent development tools (Visual studio etc.) Perfect integration with MS software (like Excel, Word). Disadvantages: Not practical for simple tasks, documentation too detailed (at windows dev center) and not to the point. Open source alternatives: Consider Unix tools in windows (python or Perl, natives or running in unix-like env: CygWin) Workshop progetto CyberSAR, december 4/6 th 2007

Basic examples:
Visual Basic Script (WSH)‏
Aim: Controlling the presence of a Drive
(network or disk) attached to the computer
Set WshShell = WScript . CreateObject ("WScript.Shell")‏
Set WshNetwork = WScript . CreateObject ("WScript.Network")‏
Set AllDrives = WshNetwork. EnumNetworkDrives ()‏
DriveLetter = "N:" 'must be capitalized
RemotePath = "avec"
AlreadyConnected = False
For i = 0 To AllDrives. Count - 1 Step 2
If AllDrives. Item (i) = DriveLetter Then AlreadyConnected = True0
Next
If AlreadyConnected = False then
WShNetwork. MapNetworkDrive DriveLetter, RemotePath
WshShell. PopUp "Drive " & DriveLetter & " connected successfully."
Else
WShNetwork. RemoveNetworkDrive DriveLetter
WshShell. PopUp "Drive " & DriveLetter & " disconnected."
End if

Basic examples (cont.):
MATLAB automation with Perl ( note the possibility
to run this code on different OS )
AIM: executing a matlab script
#!c:erlinperl.exe use Math::Matlab::Local; $matlab = Math:: Matlab :: Local -> new ({ cmd => 'C:MATLAB6p5inmatlab.bat -nosplash ', root_mwd => 'C:ScriptS' }); $fin = 'C:ScriptSEqsim.m'; open (IP,"<$fin"); $code = ""; @v = <IP>; foreach $ar (@v) { $code .= $ar; } close (IP); if ( $matlab-> execute ($code) ) { $mm = $matlab-> fetch_resul t; print $mm; } else { print $matlab-> err_msg ; }

Basic examples (cont.):
COM Objects
AIM: automating Excel (filling a table with values)
Set objExcel = CreateObject ("Excel.Application")‏ objExcel. Visible = True Set objWorkbook = objExcel. Workbooks . Add ()‏ Set objWorksheet = objWorkbook. Worksheets (1)‏ objWorksheet. Cells (1,1) = "Operating System" objWorksheet. Cells (2,1) = "Windows Server 2003" objWorksheet. Cells (3,1) = "Windows XP" objWorksheet. Cells (4,1) = "Windows 2000" objWorksheet. Cells (5,1) = "Windows NT 4.0" objWorksheet. Cells (6,1) = "Other" objWorksheet. Cells (1,2) = "Number of Computers" objWorksheet. Cells (2,2) = 145 objWorksheet. Cells (3,2) = 987 objWorksheet. Cells (4,2) = 611 objWorksheet. Cells (5,2) = 41 objWorksheet. Cells (6,2) = 56 Set objRange = objWorksheet. UsedRange objRange. Select Set colCharts = objExcel. Charts colCharts. Add ()‏

VMware and Windows at the CyberSAR GRID
Aims:
Executing Windows software in a Linux GRID environment with no software emulation (WINE).
The Linux GRID environment should be able to run Windows software with no reference to GRID software
(GRID software ignores Windows)
Requirements:
Linux 64 bit architecture with kernel optimized for GRID and high performance computing software
Mandatory Linux distribution: CENTOS4.5 (RHEL 4) or Scientific Linux 5 (this is a Cern GRID requirement)‏
Packaged Software for GRID
No remote computing with servers outside the GRID infrastructure
Solutions actually tried:
VMware Server (freeware) : (1.04 e 2.0 Beta)‏
VMware Workstation 6.0 (not freeware but with a demo of 30days)‏
Other solutions:
VirtualBox with internal bridge network (virtual interfaces to simulate remote computing)‏

ENVIRONMENT DEVELOPMENT
Network tools for development
Linux virtual interfaces (tun or tap) and network bridge to build a VPN with a specific GRID server.
Public VPN concentrator server (openvpn with private key),
needed to secure the path between developers servers and grid servers.
VMware server console (not running on VMware Workstation)
VncServer (available with VMware Workstation) and a vnc viewer
Linux iproute tool and iptables firewall (DNAT e MASQUERADING needed for many network operations)
VMware libraries: Vix-API e Vix-API Perl versione (1.12)
Windows simulation software and OS
MATLAB 6.5 for windows (need a special licence to permit terminal server operations)
WinXP home edition SP2 (the most used Windows OS)

Steps for testing environment building
VPN (OpenVPN in trunk mode - concentrator)
1) create a virtual interface tap for each client to connect
2) put the virtual interfaces tap into a bridge without assigning an IP to the bridge interface ( br0 )
3) create the vpn key and start openVPN listening on a specific tap and a different TCP port for each
planned client (the same key must be shared between client and server)
server vpn trunk file for a client vpn client file
dev tap1
proto tcp-server
secret /etc/openvpn/CONFS/key
ping 10
verb 1
mute 10
lport 500 1
remote 80.x.x.x rport 5001 nobind proto tcp-client dev tap ifconfig 10.0.1.3 255.255.255.0 secret /etc/openvpn/CONF/key ping 10 verb 1 mute 10

Installing VMware Server and VMware Workstation
Download the software and install it in the standard RH Linux way ( rpm -i ).
Configure the server (vmware-config.pl) using NAT (note that Bridge mode needs physical addresses on the network)‏
NB. VMware Server 1.04 works on X86_64 but the VIX API Perl/C/COM are not completely working
Installing ViX-API and ViX-API-Perl
API perl needs to be compiled (issuing #perl Makefile.PL)
Note: Only version 1.12 (or above) contains the libraries for x86_64 ( libvix.so ).

Workshop progetto CyberSAR, december 4/6 th 2007 CERN grid details The GRID infrastructure is composed by a geographical distributed filesystem and a middleware software (gLite) acting as a intelligent queue system. The middleware gLite is a software agent (called broker) that chooses the best server ( worker node ) to run the simulation. The submitted job is composed by a shell script and some input files. All software paths needs to be fully specified and the worker node must contain all software needed by the simulation. To run windows applications in the Grid environment a worker node is equipped with a VMware Windows VM. The shell script calls a little Perl (or C) code written with the Vix API and runs applications or scripts inside the Windows VM (VMware tools needs to be installed inside the guest VM). The simulation result is then returned to the Linux host: Queue system user linux host Windows VM application software win/linux shared folder script (Vix API)‏ linux host Queue system user copy data run take control run submit select and transfer input data copy data to remote filesystem run

Application code
VMware Perl API example
notes:
1) The handle created could be
remote or local
2) The std. error is available for
the Handle
#!/usr/bin/perl ### carico i moduli per vmware use VMware::Vix::Simple; use VMware::Vix::API::Constants; ## constants my $err = VIX_OK; my $hostHandle = VIX_INVALID_HANDLE; ..... ## connessione al server vmware ($err, $hostHandle ) = HostConnect (VIX_API_VERSION, VIX_SERVICEPROVIDER_VMWARE_WORKSTATION, undef, # hostName 0, # hostPort undef, # userName undef, # password 0, # options VIX_INVALID_HANDLE); # propertyListHandle die "HostConnect() failed, $err ", GetErrorText ($err), " " if $err != VIX_OK; ### apri la macchina virtuale: restituisce l'handle alla macchina virtuale ($err, $vmHandle) = VMOpen ( $hostHandle , " $pathvm "); die "VMOpen() failed, $err ", GetErrorText($err), " " if $err != VIX_OK;

Application code
VMware Perl API (cont.)
Notes.
1) log into the windows system
2) VMRunScriptInGuest runs the script
into the windows system
$err = VMLoginInGuest ($vmHandle, "$user", # userName "$passw", # password 0); # options die "VMLoginInGuest() failed, $err ", GetErrorText ($err), " " if $err != VIX_OK; # avvia lo script (inserire il percorso completo) ‏ $err = VMRunScriptInGuest ($vmHandle, "$script", "", VIX_RUNPROGRAM_ACTIVATE_WINDOW, # options VIX_INVALID_HANDLE); die "VMRunProgramInGuest() failed, $err ", GetErrorText ($err), " " if $err != VIX_OK;

Application code
A simple dos batch file is needed to move
a script file from the Linux shared folder
(host) to the windows running
directory.
C:INDOWSsystem32 cscript.exe "C:ScriptSMA.vbs" copy "c:ocuments and SettingsmickDesktopcalcdoc.sxw" " hostShared FoldersScambiocalcdoc.sxw "

Application code
SCRIPT VBS (Openoffice) ‏.
Notes:
1) Need the sun com interface
Set oServiceManager = CreateObject ("com.sun.star.ServiceManager") ‏ ' Get the Desktop object. Set oDesktop = oServiceManager. createInstance ("com.sun.star.frame.Desktop") ‏ ' Use this empty array when no arguments are needed. Dim aNoArgs() ‏ ' Create a new empty spreadsheet. Set oCalcDoc = oDesktop . loadComponentFromURL (" private:factory/scalc ", "_blank", 0, array()) ‏ Set oSheet = oCalcDoc.getSheets().getByIndex(0) ‏ Call oSheet.getCellByPosition(0, 0). SetFormula ("Month") ‏ Call oSheet.getCellByPosition(1, 3). SetValue (4103) ‏ ' Save the spreadsheet. Call oCalcDoc. storeToURL ("file:///c:/documents and settings/mick/desktop /calcdoc.sxw", array()) ‏ ' Now save it as an Excel file. Call oSheet. Close (True) ‏ Call oDesktop. Close (True) ‏

CONCLUSIONS
VMware Perl API is a powerful tool to integrate virtual machines and remote executing tasks avoiding the intensive use of dedicated network services
VMware permits the code reuse and heterogeneous simulation between different OS and softwares
Some problems with VMware Perl API in a 64 bit environment (API needs to be improved)
Future projects
Using vmware infrastructure in the Grid middleware

Workshop progetto CyberSAR, december 4/6 th 2007 Acknowledgements: The present work was presented to the CyberSAR (cyber infrastructure for Sardinia, Italy) workshop (4/6 dec 2007 ) for a non specialized audience. The main goal was to show how to integrate heterogeneous operating systems and windows software in a special hardware infrastructure dedicated to GRID and High Performance Computing. Thanks to the Linkalab (complex network research laboratory) for support and suggestions in the research. LINKS www.cybersar.com www.cern.ch/grid www.linkalab.it www.unica.it

View - VMworld.com: Virtualization: Virtual Conference

by Cameroon45

Accessibility

Categories

Upload Details

Usage Rights

Statistics

View - VMworld.com: Virtualization: Virtual Conference Presentation Transcript