The document discusses virtualization system security testing. It summarizes the work of the 360 Marvel Team, which focuses on attack and defense techniques in virtualized systems. The team has developed a fuzzing framework to test for vulnerabilities in hardware virtualization components like Qemu and Vmware. The framework analyzes data flowing to components, changes it to trigger abnormalities, and records results. Using this framework, the team has found over 20 vulnerabilities. The document outlines the framework's functions, workflow, and how it was used to analyze vulnerabilities in the Qemu e1000 network device and Vmware e1000 network device through a case study.

1. 1
Speaker: Qinghao Tang
Title：360 Marvel Team Leader
Virtualization Device
Emulator Testing Technology

2. 2
360 Marvel Team
Established in May 2015, the first professional could
computing and virtualization security team in China.
Focusing on attack and defense techniques in virtualization
system.
● fuzzing framework
● guest machine escape technology
● Hypervisor risk defense technology

3. 3
Agenda
• Virtualization System Attack Surface
• Fuzzing framework
• Case study

4. 4
Virtualization System Attack Surface

5. 5
Cloud Computing

6. 6
Hypervisor
Types
Xen Kvm Vmware
function
quantizing
distribution
flexible
scheduling

7. 7
Distinctions
OS
Physical Devices
Guest OS
Hardware
emulator
Hypervisor
Physical Devices
Guest OS
Hardware
emulator
Normal Server Virtualization Server

8. 8
Attacking Processes in cloud computing
1. Enter VM via web or other devices
2. Exploit virtualization system vulnerabilities to escape VM
3. lateral movements to others VMs on host
4. Access to host network

9. 9
Operation Principles of device emulators

10. 10
The attack surface
• Hardware virtualization components’ diversity
Qemu： 30+
Vmware：20+
• Bridge between inside-outside
VM os -> device emulators -> Host os
• Related Vulnerabilities result big dangers
• Limitation

11. 11
Fuzzing Framework

12. 12
Basic intro
Attack surface ： hardware virtual components
Environment ： qemu ， vmware
Testing results ： more than 20 vulnerabilities
Challenges ： lower layers hard to predict；
Trends
• more attack surfaces
• more kinds of virtualization systems

13. 13
• Hardware virtualization focus on lower layers
• Testing data totally different
Compare to traditional targets
System Kernel
Hypervisor

14. 14
1. Analyze data which flowed to components
2. Change flowed-in data’s contents and timing
3. Recording all of tiny abnormal activities
4. Analyze abnormal activities, takes and optimize fuzz
framework.
Methods for testing hardware virtual components

15. 15
Other factors of fuzz framework
1.Flexibility (other OS)
• vm in Linux
• coding in C and Python
2. Deeply understand VM system
• language for coding
• development environment
• coding style

16. 16
os
Control Center
Fuzz framework structure
HostHost
os os os osos

17. 17
Fuzz framework working flow (part 1)
1. Set up network and hardware environment, launch server, client
and monitor.
2. Load system hook module, get all of machines’ device emulators
3. Client ask server for testing data of emulators, server send out
required data.
4. Client received and loaded testing data, launch test.
5. Monitor continually monitor hypervisor, and record logs.

18. 18
Fuzz framework working flow (part 2)
6. Notify the server after client testing finished
7. Server get logs from client and monitor, save it.
8. Server launch log analyze module, determine if anything
wrong happened, and notify admins.
9. Analyze program exceptions, optimize fuzz framework

19. 19
Functions of controlling center

20. 20
Get target components info

21. 21
• Device access ports
• Device deal with structures used by data.
• Device data processing
Testing data

22. • User space: generate testing dat,
send request to client kernel
• Kernel space: apply for memory, fill
memory, send info to ports
• Device emulator：testing data flow
inside，trigger exceptions
22
Testing data attacks
user space
Kernel space
Device controller

23. 23
Monitor
VM management
• Snapshot
• Reboot
• VM device editing
Dynamic debugging
• Debugging Mode on Start
• Load Debugging Plugin
VM processing log
• User space
• Kernel space

24. 24
Exceptions occur in device emulator
• VM os crash
• Hypervisor crash
• Invisible results

25. 25
Advanced monitoring skills
• Dynamic
• Static

26. 26
Optimize fuzz framework by using log data
• Client log
Decrease invalid combinations
• Monitor log
Promote coverage
• Server log

27. 27
Vulnerabilities found by us

28. 28
Case Study

29. 29
• Initialization
Port Allocation ， Address Mapping
Device Status Setting, Resource Allocation
• Data Transfer
'Write Command' to device TDT register
process of descriptor
3 types descriptor ： context ， data ， legacy
data xfer
set status ， wait for next instruction
• Processing Details
Circular Memory
TSO ： tcp segmentation/flow control.
Principle of e1000 Network Device

30. 30
• Qemu e1000 Network Device
• Vmware e1000 Network Device
E1000 vulnerability analysis

31. 31
Pcnet network card emulator working processes
Io port write
Control and
Status
Registers
write
Receive Send
Virtual Network Interface

32. 32
• Qemu pcnet Network Device
Pcnet vulnerability analysis

33. 33
Summary
Stay tuned for more achievements by
360 Marvel Team

34. 34
Q & A
Email：tangqinghao@360.cn
QQ：702108451