Vmbkp is an online backup tool for VMware vSphere that performs full, differential, and incremental backups of virtual machines. It uses efficient archive formats and sequential I/O to backup virtual disk (VMDK) files. Key features include multi-generation backup management, command-line interface, and support for backup scheduling via Cron. The tool utilizes the VDDK and VI Java libraries to interface with vSphere and perform tasks like snapshots and VMDK access during the backup process.
As time goes on more OSes are getting Dom0 support, so there's a growing need to provide a platform independent set of tools from which to operate Xen. This talk will expose the different mechanisms used on NetBSD that diverge from the Linux approach, and how Xen is improving its userspace tools to provide a more platform independent support.
The talk also touches upon various features that BSD provides or plans to provide with Xen, thus presenting a coherent roadmap view of where we've come from, and what lies ahead.
What's in this talk:
Xen and BSD
Status updates from the world of BSD
Ecosystem/userbase
As part of the Google Summer of Code, we tried to add support for SeaBIOS in order to allow guest OSes to be booted directly from PV disk devices rather than from the emulated disk device. SeaBIOS is the BIOS implementation that upstream qemu uses. When the virtual machine is created, SeaBIOS upon initialization uses a generic Xenstore client to communicate with the back end and initialize the front-end block device that will connect to the back end. After the connection is established I/O requests are made via the BIOS int 0x13 interface, guest OSes use the int 0x13 without needing to be aware that PV drivers were used.
As time goes on more OSes are getting Dom0 support, so there's a growing need to provide a platform independent set of tools from which to operate Xen. This talk will expose the different mechanisms used on NetBSD that diverge from the Linux approach, and how Xen is improving its userspace tools to provide a more platform independent support.
The talk also touches upon various features that BSD provides or plans to provide with Xen, thus presenting a coherent roadmap view of where we've come from, and what lies ahead.
What's in this talk:
Xen and BSD
Status updates from the world of BSD
Ecosystem/userbase
As part of the Google Summer of Code, we tried to add support for SeaBIOS in order to allow guest OSes to be booted directly from PV disk devices rather than from the emulated disk device. SeaBIOS is the BIOS implementation that upstream qemu uses. When the virtual machine is created, SeaBIOS upon initialization uses a generic Xenstore client to communicate with the back end and initialize the front-end block device that will connect to the back end. After the connection is established I/O requests are made via the BIOS int 0x13 interface, guest OSes use the int 0x13 without needing to be aware that PV drivers were used.
Configuring and Using the New Virtualization Features in Windows Server 2012Lai Yoong Seng
During Windows Server 2012 Roadshow event at Singapore and Malaysia, i have presented on how to configure Hyper-V High Availability and Hyper-V Replica.
Dealing with Hardware Heterogeneity Using EmbeddedXEN, a Virtualization Frame...The Linux Foundation
EmbeddedXEN is a particularly efficient virtualization framework tailored to ARM-based core embedded systems.
While security and OS isolation are key features of conventional virtualizuation frameworks, the main concerns for EmbeddedXEN are device heterogeneity and realtime aspects, which are particularly important in the embedded world.
EmbeddedXEN mainly relies on the original XEN architecture but with major differences in the way guest OS are handled: the hypervisor has been simplified, and only two guest OS (dom0 and domU) can run simultaneously; while dom0 is used to manage the native OS with drivers (original and backend splitted drivers), a paravirtualized OS (domU) can be cross-compiled on a different ARM device, and user applications can run seamlessly on the (virtualized) host device. Another important difference is that no user space tools are required to manage the VMs; the framework produces a compact single binary image containing both dom0 and domU guests, which can be easily deployed. The Xenbus architecture has been adapted to that context.
EmbeddedXEN therefore allows the porting of an OS and its applications from an ARM embedded device to last generation ARM hardware, such as HTC Smartphone for example.
In a traditional Xen configuration domain 0 is used for a large number of different functions including running the toolstack(s), backends for network and disk I/O, running the QEMU device model instances, driving the physical devices in the system, handling guest console/framebuffer I/O and miscellaneous monitoring and management functions. Having all these functions in one domain produces a complex environment which is susceptible to shared fate on the failure of any one function, has complex interactions between functions (including resource contention) which makes it difficult to predict performance, and has limited flexibility (such as requiring the same kernel for all device drivers).
""Domain 0 disaggregation"" has been discussed for some time as a way to break out domain 0's functions into separate domains. Doing this enables each domain to be tailored to its function such as using a different kernel or operating system to drive different physical devices. Splitting functions into separate domains removes some of the unintentional interactions such as in-domain resource contention and reduces the system impact of the failure of a single function such as a device driver crash.
Although domain 0 disaggregation is not new it is seldom used in practise and much of its use is focussed on providing enhanced security. Citrix XenServer will be moving towards a disaggregated domain 0 in order to provide better security, scalability, performance, reliability, supportability and flexibility. This talk will describe XenServer's “Windsor” architecture and explain how it will provide the above benefits to customers and users. We will present an overview of the architecture and some early experimental measurements showing the benefits.
XCP: The Art of Open Virtualization for the Enterprise and the CloudThe Linux Foundation
XCP is a free and open source self-contained virtualization solution for servers, built on top of the Xen hypervisor. It is easily installable in a few minutes from a single image file, yet powerful and scalable enough to be useful to power users, enterprise environments and cloud deployments. Created from the open-source components of XenServer, it supports the virtualization of a range of operating systems, including Linux, Solaris, BSDs and Windows. This talk will introduce XCP and explain its relationship with Xen and Linux. We will quickly demonstrate how to use XCP via the command-line and using opensource graphical interfaces, and describe some interesting features that set XCP apart from other virtualization platforms.
Configuring and Using the New Virtualization Features in Windows Server 2012Lai Yoong Seng
During Windows Server 2012 Roadshow event at Singapore and Malaysia, i have presented on how to configure Hyper-V High Availability and Hyper-V Replica.
Dealing with Hardware Heterogeneity Using EmbeddedXEN, a Virtualization Frame...The Linux Foundation
EmbeddedXEN is a particularly efficient virtualization framework tailored to ARM-based core embedded systems.
While security and OS isolation are key features of conventional virtualizuation frameworks, the main concerns for EmbeddedXEN are device heterogeneity and realtime aspects, which are particularly important in the embedded world.
EmbeddedXEN mainly relies on the original XEN architecture but with major differences in the way guest OS are handled: the hypervisor has been simplified, and only two guest OS (dom0 and domU) can run simultaneously; while dom0 is used to manage the native OS with drivers (original and backend splitted drivers), a paravirtualized OS (domU) can be cross-compiled on a different ARM device, and user applications can run seamlessly on the (virtualized) host device. Another important difference is that no user space tools are required to manage the VMs; the framework produces a compact single binary image containing both dom0 and domU guests, which can be easily deployed. The Xenbus architecture has been adapted to that context.
EmbeddedXEN therefore allows the porting of an OS and its applications from an ARM embedded device to last generation ARM hardware, such as HTC Smartphone for example.
In a traditional Xen configuration domain 0 is used for a large number of different functions including running the toolstack(s), backends for network and disk I/O, running the QEMU device model instances, driving the physical devices in the system, handling guest console/framebuffer I/O and miscellaneous monitoring and management functions. Having all these functions in one domain produces a complex environment which is susceptible to shared fate on the failure of any one function, has complex interactions between functions (including resource contention) which makes it difficult to predict performance, and has limited flexibility (such as requiring the same kernel for all device drivers).
""Domain 0 disaggregation"" has been discussed for some time as a way to break out domain 0's functions into separate domains. Doing this enables each domain to be tailored to its function such as using a different kernel or operating system to drive different physical devices. Splitting functions into separate domains removes some of the unintentional interactions such as in-domain resource contention and reduces the system impact of the failure of a single function such as a device driver crash.
Although domain 0 disaggregation is not new it is seldom used in practise and much of its use is focussed on providing enhanced security. Citrix XenServer will be moving towards a disaggregated domain 0 in order to provide better security, scalability, performance, reliability, supportability and flexibility. This talk will describe XenServer's “Windsor” architecture and explain how it will provide the above benefits to customers and users. We will present an overview of the architecture and some early experimental measurements showing the benefits.
XCP: The Art of Open Virtualization for the Enterprise and the CloudThe Linux Foundation
XCP is a free and open source self-contained virtualization solution for servers, built on top of the Xen hypervisor. It is easily installable in a few minutes from a single image file, yet powerful and scalable enough to be useful to power users, enterprise environments and cloud deployments. Created from the open-source components of XenServer, it supports the virtualization of a range of operating systems, including Linux, Solaris, BSDs and Windows. This talk will introduce XCP and explain its relationship with Xen and Linux. We will quickly demonstrate how to use XCP via the command-line and using opensource graphical interfaces, and describe some interesting features that set XCP apart from other virtualization platforms.
VMware vSphere Version Comparison 4.0 to 6.5Sabir Hussain
VMware vSphere leverages the power of virtualization to transform datacenters into simplified cloud computing infrastructures and enables IT organizations to deliver flexible and reliable IT services VMware vSphere virtualizes and aggregates the underlying physical hardware resources across multiple system and provides pools off virtual resources to the datacenter.
VM Virtualization
VMGate.com
What’s New in VMware vCenter Site Recovery Manager v5.0Eric Sloof
Summary of SRM v5.0 New Features
New user interface
Planned migration – with replication update
Failback
vSphere Replication
Faster IP customization
Shadow VM icons
In guest scripts
VM dependency
VMware Backups That Work—Lessons Learned From VADP Performance Benchmark TestingSymantec
We’ve pushed the backup performance envelope so that you don’t have to! Industry leaders Cisco, NetApp, VMware and Symantec teamed up to develop a best practice framework and performance benchmark based on the vStorage APIs for Data Protection (VADP). The test configuration uses the popular NetApp FlexPod environment. The result that proves that you can easily protect over 4 TB of virtual machine data an hour. Most think that to provide this sort of performance, a mountain of hardware is required. This is not the case. We show you how these performance numbers can be easily obtained with minimal hardware and a small budget. Improving backup performance also creates more reliable backups, shorter backup windows and less impact on the vSphere infrastructure. Whether or not you are using NetBackup, we invite you to attend this hangout where we'll dig deep into VMware VADP and its performance characteristics. We'll share lessons learned from extensive lab benchmarks simulating real production workloads. This will help you design and deploy a backup solution for your VMware vSphere environment that meets your business SLAs.
Bonus exclusively at this hangout: In this hangout, we will provide a sneak preview some of NetBackup for VMware features coming in NetBackup 7.6. We have exciting results to share on how NetBackup is pushing the performance envelope further!
Panel Members:
• Abdul Rasheed, Product Marketing Manager, vExpert
• George Winter, Technical Product Manager, vExpert
• Alex Sakaguchi, Product Marketing Manager
View Hangout: http://bit.ly/1efz3zB
Turning OpenStack Swift into a VM storage platformwim_provoost
OpenStack Swift is the Object Storage project within OpenStack. Alas, due to technical hurdles (eventual consistency, blocks <> objects, …) it is impossible to run Virtual Machines directly on Swift. You need a layer in between Swift and the hypervisor which can overcome these hurdles. This is where Open vStorage comes in.
Open vStorage is an open-source VM storage router. It is a software layer (called the VM Storage Router) in between Virtual Machines and storage backends. It allows to abstract the backend from the Virtual Machine and creates a uniform, single namespace across multiple hosts. These VM Storage Routers (VSRs) operate like a grid leveraging local flash memory or SSDs and any storage back-end (S3 compatible object store, (distributed) filesystem, NAS) to provide an extremely high performance and reliable storage system.
One of the supported Object Stores is OpenStack Swift. Open vStorage is the only solution to turn OpenStack Swift into block storage for Virtual Machines. Through a Cinder Plugin it allows to create and manage volumes directly on top of OpenStack Swift. Combining Open vStorage with OpenStack Swift allows to create a scale-out, performing, VM-centric storage platform which neatly integrates with OpenStack.
HYPER-V upcoming version 3.0 features overview. This presentation covers the information about new enhancement about Netwokring, Storage and VHDx format.
主に論文 "Weak Consistency: A Generalized Theory and Optimistic Implementations for Distributed Transactions" の紹介。
https://pmg.csail.mit.edu/pubs/adya99__weak_consis-abstract.html
論文の紹介
Suffix trees for inputs larger than main memory
Marina Barsky, Ulrike Stege and Alex Thomo
Information Systems archive Volume 36 Issue 3, May, 2011
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
From Siloed Products to Connected Ecosystem: Building a Sustainable and Scala...
Vmbkp: VMware vSphere Incremental Backup Tool
1. Vmbkp: An Online Backup Tool
for VMware vSphere
Oct 15, 2010
HOSHINO Takashi
Cybozu Labs, Inc.
1
2. What is Vmbkp?
• Backup software for Virtual Machines
in VMware vSphere environment
– Online full/differential/incremental backup
– Multi-generation backup management
– Efficient archive access with sequential IO and reverse diff.
– Command-line I/F for scheduling by Cron
2
3. Supported platform
• VMware vSphere 4
– vCenter server managing several ESX(i)s
– Single ESX(i) (not tested)
– Free ESXi is not supported (snapshot fails)
• Backup server
– Linux on x86_64 host.
– CentOS 5.5 64bit is confirmed
3
4. Hardware Architecture
Control/GetInfo with
vSphere Soap Protocol VMware vSphere
vCenter Server
Vmbkp LAN
Server
VMware VMware
ESX(i) Host ESX(i) Host
SAN
Data Transfer via SAN
with VDDK Protocol
Backup VM VM VM
Storage Storage Storage Storage
You can use NBD transfer without SAN. 4
5. Commands
• Update:
– Get and save information of all available VMs
• Backup:
– Execute backup of the specified vm/group or all
• Restore:
– Execute restore of the specified archived generation as a new VM
• Check:
– Check backup archives are valid
• Status:
– Show status of backup archives
5
6. Commands –cont.
• Destroy:
– Remove a virtual machine from vSphere environment
• Clean:
– Delete archives of virtual machines
• List:
– Get a list of virtual machines satisfying specified conditions
• Help:
– Show usage
6
7. Workflow
Backup Restore
Prepare config Prepare config
(Register to cron)
Read config/profiles Read config/profiles
Get vSphere information Restore target VMs
Backup target VMs Import ovf
Export ovf (without disks) Add disks to new VM
Create snapshot Restore vmdk files
(Get changed block info)
Backup vmdk files
Delete snapshot
(Delete previous dump)
User task
Update profiles
Vmbkp task
7
8. Configuration files
• Global (required)
– Global configuration
• Backup directory
• Number of generations to keep
• Vmdkbkp path to backup/restore vmdk files
• vSphere authentication information
• Group (optional)
– Group configuration for convenient use
8
9. Layout of Archive Files
• <backup dir>
– AllVM profile
• <backup dir>/<vm>/
– VM profile
• <backup dir>/<vm>/<generation>/
– Generation profile
– Ovf file for VM configuration
– Dump/digest/rdiff/bmp files for each vmdk
9
10. Profiles
• Allvm
– Information/status of all VMs in the target vSphere environment
– Updated by update command
• Vm
– Information/status of archives of a VM
– Created/updated by backup command and referred by restore
command
• Generation
– Information/status of each generation of backup of a VM
– Created by backup command and referred by restore command
10
11. Software Architecture
Cron User
Command-line Interface
Backup/Restore Controller
Utility Soap Wrapper Vmdkbkp Wrapper
Library Snapshot
Vmdkbkp: Vmdk
Ovf Backup/Restore
Bitmap Changed blocks
XML (Ovf) Tool/Library (C++)
Config/Profile VI Java Library VDDK C Library
VMware vSphere VMware ESX(i) SAN
vCenter Server Host Storage
11
12. Required Tools and Libraries
• Java SE 1.6
– Java, Javac, Jar comands
• VI-Java 2.1GA
– soap wrapper
• G++ 4.4
• Boost 1.43
– shared_ptr, scoped_array, thread, and iostreams
• VDDK 1.2.0
– Virtual disk development kit by Vmware
12
15. What is VmdkBkp?
• Online backup software
for remote/local vmdk files
in VMware vSphere environments.
– Currently support vSphere version 4.
• Written in C++
• Uses VDDK Library by Vmware
• Used by Vmbkp (java) tool
16. Archive Files
• Dump/Rdiff
– VMDK metadata and blocks archive
without zero-blocks
– Dump is full archive,
Rdiff is reverse differential one
– Dump + Rdiff = Previous dump
• Digest
– MD5 digest data for all blocks of VMDK
– Used to check equality of blocks,
and validate corresponding dump/rdiff files
17. Supported Commands
• Dump
– Execute full/differential/incremental dump
• Restore
– Execute restore with dump/rdiff
• Check
– Validate dump/rdiff with digest data
• Print
– Print dump/rdiff/digest for human read
• Digest
– Make digest from dump
• Merge
– Make past dump from current dump and past rdiff(s)
18. How to Backup Remote Vmdk
• Command line:
– vmdkbkp dump [connect options] --mode [full/diff/incr]
--vm [vm moref] --snapshot [snapshot moref]
--remote [disk path]
--dumpin [previous dump] --dumpout [current dump]
--digestin [previous digest] --digestout [current digest]
--bmpin [changed block bitmap]
--rdiffout [current-previous rdiff]
• Inputs/Outputs:
– Full: Just --dumpout and --digestout are required
– Diff: All options except --bmpin are required
– Incr: All options are required
19. Full Backup
VM Virtual Disk
Configuration (vmdk) • Ovf
All blocks – VM configuration data
(without disk information)
Vmbkp Tool
• Dump
Non-zero blocks – Full data of vmdk
(without zero-blocks)
Backup files
• Digest
Dump – Digest data of all blocks
Ovf Digest
19
20. Differential Backup
VM Virtual Disk • Rdiff
Configuration (vmdk)
– Reverse difference
All blocks data of vmdk
– Dump’ + Rdiff’ = Dump
Vmbkp Tool
• You can delete dump of previous
generation after current backup
Non-zero blocks
Backup files of Backup files of
previous generation current generation
Dump Dump’ Rdiff’
Ovf Digest Ovf’ Digest’
20
21. Incremental Backup
VM Virtual Disk Changed Block
Configuration (vmdk) Information
Changed blocks
• Changed Block Information
Vmbkp Tool – The set of address of changed
blocks after previous backup
Non-zero blocks
Backup files of Backup files of
previous generation current generation
Dump Dump’ Rdiff’
Ovf Digest Ovf’ Digest’
21
22. Vmdk Archives Relationships
Write some data on the 1st vm.
0.vmdk 1.vmdk
Full Full
dump dump
Diff
dump
0.dump 1.dump
0.digest 1.digest
Incr
dump 1-0.rdiff
Check the all dump/digest files rdiff2bmp
from all possible paths are the same
using check_dump_and_dump and 1.bitmap
check_digest_and_digest.
23. Vmdk Archives Relationships –cont.
Write some data on the 1st vm.
0.vmdk 1.vmdk
Restore Restore
Merge
0.dump 1.dump
0.digest 1.digest
Restore to 0.dump
1-0.rdiff
Digest Full dump 0.vmdk to 0r.dump
Check 0.dump and 0r.dump are the same.
Merge 1.dump and 1-0.rdiff to 0m.dump
Digest 0m.dump to 0m.digest
Check 0.{dump,digest} and 0m.{dump.digest} are the same.
24. Software Architecture of vmdkbkp
Command Command executor
Util Header Manager Specific components
Exception Serialize Bitmap General components
• Command • Manager
– Parse command-line and execute it – Manage (1) VDDK connection,
• Util (2) vmdk file access, and (3)
dump/rdiff/digest file access
– Configuration, Time, etc.
• Serialize
• Header – StringMap/Integers data serializer
– Manage header/blocks of
dump/rdiff/digest files • Bitmap
• Exception – Bitmap data serializer
– Exceptions and related macros.
25. VDDK Control with Fork
• Solves the problem that VDDK re-initialization
for SAN transfer due to SCSI reservation
conflict error inevitably fails and falls back to
NBD transfer.
25
26. VDDK Control with Fork –cont.
Main process
Provide the same interface
VddkController with Vddk/Vmdk Manager
VddkWorker(parent) Manage processes and
communicate with child
Forked process
Wrapper of Vddk/Vmdk
VddkWorker(child) manager and communicate
with parent
VddkManager VmdkManager
27. Multi-threaded Archive Manager
• Improves performance with gziped multi-
stream dump/restore/check/merge
operations
Archive Managers Interface of archive accesses
specialized for each command
Archive IO Managers Multi-threaded/Single-threaded
stream access for each archive file
DataReader, DataWriter Worker thread and its controller for
Gzip compresson/decompression
Queue Thread-safe FIFO
27
29. Restore with SAN
• Problem in restore with SAN
– Failed auto-allocation for thin vmdk
– Auto-allocation is too slow for thick vmdk
– There is no efficient allocation API.
• If zero-block restore with NBD is faster, use it
as allocation method
– not fast…
30. Future Work
• Improve parallelism
– Solving SCSI reservation conflict problem
– Multi-threaded compression
• Restore with SAN
– Depends on VDDK’s efficient block allocation API
30