Oracle Solaris 11 platform for ECI Telecom private cloud infrastructure
OverviewMark Markman2013Oracle Solaris 11 platform forECI Telecom private cloudinfrastructure
Project Definition1. Deliver and deploy the ECI Management Networkinfrastructure as a private cloud based on built-in OracleSolaris 11 technologies without purchasing anddeployment additional 3rd party software.2. Provide high economical indices for the solution (ROI,TCO) by effective HW capacity utilization and downtimeminimization (RAS).3. Minimize development and QA cost by reusing of existingsolutions where possible (reduce or eliminate theapplication SW code changes).
Principles & Concepto Cloud composed of Solaris 10 Branded and Solaris 11 Nativezones (referred below as VMs).o Global Solaris 11 zone running exclusively VMs and resourcessetup/control facilities.o Fast and simple VM deployment.o Scalability and load balancing by migrating the workloadbetween HW resources attached to the cloud.o Every VM runs a separate single application instance.o Both shared and local (directly attached) storages aresupported.
Benefits:1. Adding of HW abstraction extends life span of existingsolutions, enabling future HW upgrades transparent.2. Minimizing /eliminating of application SW changes toadopt the new environment saves the development andQA effort.3. Solaris 11 OS installation in global zone is minimized tosupport the basic VM and Cloud facilities only (fastdeployment, higher security, etc…).4. Faster and reliable infrastructure and application UpgradeProcedures (decreased downtime).5. Most of the Solaris 11 innovation can be used in brandedzones as well as in native ones.
Solaris 11 technologies used:o Zoneso ZFSo AI Distro Constructoro Network Virtualization (“Crossbow”)o Boot Environmentso COMSTARo IPSo zonestat, DTrace, etc…All these technologies work together – the wholeis greater then the sum of its parts!
Solution Components:o Customized Operating System (Oracle Solaris 11.1).o OS Post-Installation setup and tuning facilities.o Application installation/upgrade tools.o Setup, service and maintenance utilities.o Virtual Machine (zone) for Standard Production OEM Run-Time environment (Branded and Native).o Cloud Management and Monitoring tool.o Zone Migration tool (Solaris10 -> Solaris11).
Solaris 10 Branded Zoneo VM (Solaris 10 Branded Zone) created from zone image file.o Zone image is compressed snapshot datastream file.o Zone creation general steps:1. Creating prototype datasets from the image file2. Cloning prototype datasets into zone dataset3. Setup standard (empty) zone configuration within cloned dataset4. Attaching new zone5. Performing post-install configurationNew branded zone creation time (on x86 system):first zone on ZFS pool ~ 40 secsecond zone (prototype dataset already exists) ~ 8 sec (!)
Zone Administration toolDeveloped to manage all zone (VM) tasks needed:o Create zone of a specific type: Branded zone from prototype image file Native Solaris 11 zone from scratch Native Solaris 11 zone from prototype image file By cloning the existing zoneo Move zone between ZFS pools on the same server.o Transfer zone to another physical server.o Attach Migrated zone from Solaris 10.o Setup and tune zone parameters .o Zone start / stop / destroy / display status
VM Deployment Workflow(shared storage iSCSI case)1. create ZFS volume2. create iSCSI target 1. Discover iSCSI target2. Assign vdev for iSCSI target3. Configure ZFS pool on vdev4. Create datasets for zone deployment5. Copy (receive) zone image to dataset6. Configure new zone7. Attach new zone8. Post-install setup/tuning9. Boot the new zoneStorageServerExample (share storage is Solaris11 box):pkg install group/feature/storage-serversvcadm enable stmfsvccfg export -a stmfzfs create -V 10g rpool/iscsi_volstmfadm create-lu /dev/zvol/rdsk/rpool/iscsi_volstmfadm list-lustmfadm add-view 600144F0563C8C0000005149E7320001svcadm enable -r svc:/network/iscsi/target:defaultitadm create-targetDiscover iSCSI target:iscsiadm add static-configiqn.1986-03.com.sun.iqn…,18.104.22.168iscsiadm add discovery-address 22.214.171.124:3260iscsiadm modify discovery --static enabledevfsadm -i iscsiCreate ZFS pool/datasets:zpool create iscsi-pool c0t600144F0563C8C0000005149E7320001d0zfs create zone_datasetsCopy zone image to dataset:gzcat prototype | zfs recv -u -d iscsi_poolCreate zone:zfs clone -p proto_clone iscsi_pool/zone_ds… create zone cfg file…zonecfg -z zone_name -f zone_cfg_filezoneadm -z zone_name attachzoneadm -z zone_name boot
VM Transfer(shared storage iSCSI case)1. Shutdown zone2. Export iSCSI ZFS pool3. Disconnect iSCSI target4. Transfer zone metadata toserver B1. Discover the iSCSI target2. Assign vdev for iSCSI target3. Import the ZFS pool4. Mount datasets5. Tune received zone metadata6. Attach transferred zone7. Boot the transferred zoneStorageServer BServer A
VM Transfer(shared storage iSCSI case)StorageServer BServer A
Additional Features & Tools:1. Migration ToolTransforms Solaris 10 sparse-root zone intobranded zone for Solaris 112. Zone MonitorPerformance monitoring tool, based on Solaris 11zonestat utility3. Cloud ManagerTool for the whole cloud management andmonitoring1. Migration ToolTransforms Solaris 10 sparse-root zone intobranded zone for Solaris 112. Zone MonitorPerformance monitoring tool, based on Solaris 11zonestat utility3. Cloud ManagerTool for the whole cloud management andmonitoring
Implementation: current statusManagement Systems R&D private cloud:• Number of phys. servers: 30 (x86, SPARC)• Number of VMs: ~400 (350 branded + 50 native)• Central storage: 22TB (Sun ZFS Appliance 7120)• Storage access: iSCSI (via 1GbE and 10GbE)• User access: PC-X emulators• New VM deployment time: 8sec. (average)• VM migration time: 5sec.Use of Solaris 11 cloud features made HW utilization and TCOparameters much higher.