• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
View - VMworld.com: Virtualization: Virtual Conference

View - VMworld.com: Virtualization: Virtual Conference






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    View - VMworld.com: Virtualization: Virtual Conference View - VMworld.com: Virtualization: Virtual Conference Presentation Transcript

    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Using VMware Windows virtual machines in a CERN GRID environment at CyberSAR (Sardinia, Italy)
      • michelangelo puliga ( [email_address] )
      • dipartimento di Matematica ed Informatica,Università di Cagliari
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Summary
      • Virtualization technologies
      • Task automation on Windows
      • VMware on the CERN grid of CyberSAR infrastructure
      • Conclusions
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Virtualization technology
      • Definitions
      • Logical rings: (hardware = ring 0 wrapped by kernel (ring 1) wrapped by shell (ring 2)
      • Hypervisor: (virtual machine monitor process):
      • the main process that control the operations from and to the virtual machine
      • A brief History:
      • First tools running directly on a special hardware (IBM s/370)
      • First running example of software virtualization in a x86 env: VMware virtual platform (1999)‏
      • ParaVirtualization XEN (first example of virtualization with hypervisor running at kernel level (ring 1)‏
      • New perspectives with VT (Intel) and Pacifica (Amd) virtual machine software dedicated processor technology
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • VMware, Xen, VirtualBox solutions
      • Paravirtualization: XEN.
      • Advantages: Performances. Open Source, Optimal support to Live migration
      • Disadvantages: Needs a modified Linux kernel, support for 64 bit architectures is currently poor.
      • VMware
      • Advantages: Reliability, easy management. Desktop products with nice and powerful interfaces. Development kit
      • Disadvantages: Almost completely closed source. Networking not flexible in basic products
      • VirtualBox
      • Advantages : Open Source, simple and powerful networking structure. Desktop ready
      • Disadvantages: No high level API
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Task automation on Unix and Windows
      • Unix automation.
      • Operating system built for automation (The power is in the shell)‏
      • Several languages for system management and automation (perl and, more recently, python)‏
      • Example problem:
      • How to rename thousands of files adding to each file name the date string and a progressive index ?
      • PERL or SHELL solution:
      • 10 lines of code, and 2 minutes to write the code
      • Windows native solution: is there one ?
      • DOS (#!>%&) NO
      • consider the ”infamous” jscript or vbscript or 3 rd part software
      #!/usr/bin/perl # @list_files = ` ls `; $date = ` date +&quot;%m%d%Y&quot;`; $num_files = @list_files; for ($i = 0; $i < $num_files; $i++)‏ { $fname = $list_files[$i]; $fdest = $fname.$date.$i; ` mv $fname $fdest` }
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Task automation on Windows.
      • Until 2008 (longhorn) no complete powershell : in all Windows operating systems some administrative tasks are possible only from the graphic interface
      • On WinXP and Vista is present the windows scripting host (WSH) tool, with Jscript or VB syntax.
      • Advantages: WSH can explore and interact with the entire OS with little restrictions (i.e.manipulating windows registry keys).
      • Disadvantages: security issues. Wsh is the ideal tool for virus coders.
    • Software automation on Windows. High level software automation with COM, OLE and ActiveX Modern tools: DCOM and .NET Advantages: Very powerful graphic interfaces, excellent development tools (Visual studio etc.) Perfect integration with MS software (like Excel, Word). Disadvantages: Not practical for simple tasks, documentation too detailed (at windows dev center) and not to the point. Open source alternatives: Consider Unix tools in windows (python or Perl, natives or running in unix-like env: CygWin) Workshop progetto CyberSAR, december 4/6 th 2007
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Basic examples:
      • Visual Basic Script (WSH)‏
      • Aim: Controlling the presence of a Drive
      • (network or disk) attached to the computer
        • Set WshShell = WScript . CreateObject (&quot;WScript.Shell&quot;)‏
        • Set WshNetwork = WScript . CreateObject (&quot;WScript.Network&quot;)‏
        • Set AllDrives = WshNetwork. EnumNetworkDrives ()‏
        • DriveLetter = &quot;N:&quot; 'must be capitalized
        • RemotePath = &quot;avec&quot;
        • AlreadyConnected = False
        • For i = 0 To AllDrives. Count - 1 Step 2
        • If AllDrives. Item (i) = DriveLetter Then AlreadyConnected = True0
        • Next
        • If AlreadyConnected = False then
        • WShNetwork. MapNetworkDrive DriveLetter, RemotePath
        • WshShell. PopUp &quot;Drive &quot; & DriveLetter & &quot; connected successfully.&quot;
        • Else
        • WShNetwork. RemoveNetworkDrive DriveLetter
        • WshShell. PopUp &quot;Drive &quot; & DriveLetter & &quot; disconnected.&quot;
        • End if
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Basic examples (cont.):
      • MATLAB automation with Perl ( note the possibility
      • to run this code on different OS )
      • AIM: executing a matlab script
      #!c:erlinperl.exe use Math::Matlab::Local; $matlab = Math:: Matlab :: Local -> new ({ cmd => 'C:MATLAB6p5inmatlab.bat -nosplash ', root_mwd => 'C:ScriptS' }); $fin = 'C:ScriptSEqsim.m'; open (IP,&quot;<$fin&quot;); $code = &quot;&quot;; @v = <IP>; foreach $ar (@v) { $code .= $ar; } close (IP); if ( $matlab-> execute ($code) ) { $mm = $matlab-> fetch_resul t; print $mm; } else { print $matlab-> err_msg ; }
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Basic examples (cont.):
      • COM Objects
      • AIM: automating Excel (filling a table with values)
      Set objExcel = CreateObject (&quot;Excel.Application&quot;)‏ objExcel. Visible = True Set objWorkbook = objExcel. Workbooks . Add ()‏ Set objWorksheet = objWorkbook. Worksheets (1)‏ objWorksheet. Cells (1,1) = &quot;Operating System&quot; objWorksheet. Cells (2,1) = &quot;Windows Server 2003&quot; objWorksheet. Cells (3,1) = &quot;Windows XP&quot; objWorksheet. Cells (4,1) = &quot;Windows 2000&quot; objWorksheet. Cells (5,1) = &quot;Windows NT 4.0&quot; objWorksheet. Cells (6,1) = &quot;Other&quot; objWorksheet. Cells (1,2) = &quot;Number of Computers&quot; objWorksheet. Cells (2,2) = 145 objWorksheet. Cells (3,2) = 987 objWorksheet. Cells (4,2) = 611 objWorksheet. Cells (5,2) = 41 objWorksheet. Cells (6,2) = 56 Set objRange = objWorksheet. UsedRange objRange. Select Set colCharts = objExcel. Charts colCharts. Add ()‏
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • VMware and Windows at the CyberSAR GRID
      • Aims:
      • Executing Windows software in a Linux GRID environment with no software emulation (WINE).
      • The Linux GRID environment should be able to run Windows software with no reference to GRID software
      • (GRID software ignores Windows)
      • Requirements:
      • Linux 64 bit architecture with kernel optimized for GRID and high performance computing software
      • Mandatory Linux distribution: CENTOS4.5 (RHEL 4) or Scientific Linux 5 (this is a Cern GRID requirement)‏
      • Packaged Software for GRID
      • No remote computing with servers outside the GRID infrastructure
      • Solutions actually tried:
      • VMware Server (freeware) : (1.04 e 2.0 Beta)‏
      • VMware Workstation 6.0 (not freeware but with a demo of 30days)‏
      • Other solutions:
      • VirtualBox with internal bridge network (virtual interfaces to simulate remote computing)‏
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Network tools for development
      • Linux virtual interfaces (tun or tap) and network bridge to build a VPN with a specific GRID server.
      • Public VPN concentrator server (openvpn with private key),
      • needed to secure the path between developers servers and grid servers.
      • VMware server console (not running on VMware Workstation)
      • VncServer (available with VMware Workstation) and a vnc viewer
      • Linux iproute tool and iptables firewall (DNAT e MASQUERADING needed for many network operations)
      • VMware libraries: Vix-API e Vix-API Perl versione (1.12)
      • Windows simulation software and OS
      • MATLAB 6.5 for windows (need a special licence to permit terminal server operations)
      • WinXP home edition SP2 (the most used Windows OS)
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Steps for testing environment building
      • VPN (OpenVPN in trunk mode - concentrator)
      • 1) create a virtual interface tap for each client to connect
      • 2) put the virtual interfaces tap into a bridge without assigning an IP to the bridge interface ( br0 )
      • 3) create the vpn key and start openVPN listening on a specific tap and a different TCP port for each
      • planned client (the same key must be shared between client and server)
      • server vpn trunk file for a client vpn client file
        • dev tap1
        • proto tcp-server
        • secret /etc/openvpn/CONFS/key
        • ping 10
        • verb 1
        • mute 10
        • lport 500 1
      remote 80.x.x.x rport 5001 nobind proto tcp-client dev tap ifconfig secret /etc/openvpn/CONF/key ping 10 verb 1 mute 10
        • Installing VMware Server and VMware Workstation
        • Download the software and install it in the standard RH Linux way ( rpm -i ).
        • Configure the server (vmware-config.pl) using NAT (note that Bridge mode needs physical addresses on the network)‏
        • NB. VMware Server 1.04 works on X86_64 but the VIX API Perl/C/COM are not completely working
        • Installing ViX-API and ViX-API-Perl
        • API perl needs to be compiled (issuing #perl Makefile.PL)
        • Note: Only version 1.12 (or above) contains the libraries for x86_64 ( libvix.so ).
      Workshop progetto CyberSAR, december 4/6 th 2007
    • Workshop progetto CyberSAR, december 4/6 th 2007 CERN grid details The GRID infrastructure is composed by a geographical distributed filesystem and a middleware software (gLite) acting as a intelligent queue system. The middleware gLite is a software agent (called broker) that chooses the best server ( worker node ) to run the simulation. The submitted job is composed by a shell script and some input files. All software paths needs to be fully specified and the worker node must contain all software needed by the simulation. To run windows applications in the Grid environment a worker node is equipped with a VMware Windows VM. The shell script calls a little Perl (or C) code written with the Vix API and runs applications or scripts inside the Windows VM (VMware tools needs to be installed inside the guest VM). The simulation result is then returned to the Linux host: Queue system user linux host Windows VM application software win/linux shared folder script (Vix API)‏ linux host Queue system user copy data run take control run submit select and transfer input data copy data to remote filesystem run
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Application code
      • VMware Perl API example
      • notes:
      • 1) The handle created could be
      • remote or local
      • 2) The std. error is available for
      • the Handle
      #!/usr/bin/perl ### carico i moduli per vmware use VMware::Vix::Simple; use VMware::Vix::API::Constants; ## constants my $err = VIX_OK; my $hostHandle = VIX_INVALID_HANDLE; ..... ## connessione al server vmware ($err, $hostHandle ) = HostConnect (VIX_API_VERSION, VIX_SERVICEPROVIDER_VMWARE_WORKSTATION, undef, # hostName 0, # hostPort undef, # userName undef, # password 0, # options VIX_INVALID_HANDLE); # propertyListHandle die &quot;HostConnect() failed, $err &quot;, GetErrorText ($err), &quot; &quot; if $err != VIX_OK; ### apri la macchina virtuale: restituisce l'handle alla macchina virtuale ($err, $vmHandle) = VMOpen ( $hostHandle , &quot; $pathvm &quot;); die &quot;VMOpen() failed, $err &quot;, GetErrorText($err), &quot; &quot; if $err != VIX_OK;
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Application code
      • VMware Perl API (cont.)
      • Notes.
      • 1) log into the windows system
      • 2) VMRunScriptInGuest runs the script
      • into the windows system
      $err = VMLoginInGuest ($vmHandle, &quot;$user&quot;, # userName &quot;$passw&quot;, # password 0); # options die &quot;VMLoginInGuest() failed, $err &quot;, GetErrorText ($err), &quot; &quot; if $err != VIX_OK; # avvia lo script (inserire il percorso completo) ‏ $err = VMRunScriptInGuest ($vmHandle, &quot;$script&quot;, &quot;&quot;, VIX_RUNPROGRAM_ACTIVATE_WINDOW, # options VIX_INVALID_HANDLE); die &quot;VMRunProgramInGuest() failed, $err &quot;, GetErrorText ($err), &quot; &quot; if $err != VIX_OK;
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Application code
      • A simple dos batch file is needed to move
      • a script file from the Linux shared folder
      • (host) to the windows running
      • directory.
      C:INDOWSsystem32 cscript.exe &quot;C:ScriptSMA.vbs&quot; copy &quot;c:ocuments and SettingsmickDesktopcalcdoc.sxw&quot; &quot; hostShared FoldersScambiocalcdoc.sxw &quot;
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • Application code
      • SCRIPT VBS (Openoffice) ‏.
      • Notes:
      • 1) Need the sun com interface
      Set oServiceManager = CreateObject (&quot;com.sun.star.ServiceManager&quot;) ‏ ' Get the Desktop object. Set oDesktop = oServiceManager. createInstance (&quot;com.sun.star.frame.Desktop&quot;) ‏ ' Use this empty array when no arguments are needed. Dim aNoArgs() ‏ ' Create a new empty spreadsheet. Set oCalcDoc = oDesktop . loadComponentFromURL (&quot; private:factory/scalc &quot;, &quot;_blank&quot;, 0, array()) ‏ Set oSheet = oCalcDoc.getSheets().getByIndex(0) ‏ Call oSheet.getCellByPosition(0, 0). SetFormula (&quot;Month&quot;) ‏ Call oSheet.getCellByPosition(1, 3). SetValue (4103) ‏ ' Save the spreadsheet. Call oCalcDoc. storeToURL (&quot;file:///c:/documents and settings/mick/desktop /calcdoc.sxw&quot;, array()) ‏ ' Now save it as an Excel file. Call oSheet. Close (True) ‏ Call oDesktop. Close (True) ‏
    • Workshop progetto CyberSAR, december 4/6 th 2007
      • VMware Perl API is a powerful tool to integrate virtual machines and remote executing tasks avoiding the intensive use of dedicated network services
      • VMware permits the code reuse and heterogeneous simulation between different OS and softwares
      • Some problems with VMware Perl API in a 64 bit environment (API needs to be improved)
      • Future projects
      • Using vmware infrastructure in the Grid middleware
    • Workshop progetto CyberSAR, december 4/6 th 2007 Acknowledgements: The present work was presented to the CyberSAR (cyber infrastructure for Sardinia, Italy) workshop (4/6 dec 2007 ) for a non specialized audience. The main goal was to show how to integrate heterogeneous operating systems and windows software in a special hardware infrastructure dedicated to GRID and High Performance Computing. Thanks to the Linkalab (complex network research laboratory) for support and suggestions in the research. LINKS www.cybersar.com www.cern.ch/grid www.linkalab.it www.unica.it