SlideShare a Scribd company logo

Don't Tell Joanna the Virtualized Rootkit is Dead (Blackhat 2007)

Nate Lawson
Nate Lawson
Nate LawsonEngineer at Root Labs

Analysis of virtualized rootkit detection methods. Introduces "Samsara", our framework for detecting virtualization and an implementation of data/instruction TLB sizing, HPET timer, and VT errata tests. We predict the future will be cat-and-mouse, where each side analyzes and responds to the behavior of their opponent, ad infinitum. Joint talk given with Thomas Ptacek and Peter Ferrie.

Don't Tell Joanna the Virtualized Rootkit is Dead (Blackhat 2007)

1 of 40
Download to read offline
Don’t Tell Joanna
The Virtualized Rootkit Is Dead

 Nate Lawson     @   Root Labs
 Peter Ferrie    @   Symantec
 Thomas Ptacek   @   Matasano
Introduction
What We’re Going To Talk About

★ HVM Malware Recap

★ Nothing Is 100% Undetectable

★ Samsara: A Framework For HVM Malware Detection

★ Conclusion: A Cat And Mouse Game
HVM Malware
A Remedial Course
We’ve Seen This Movie Before


 1984      ‘90s       1994 - 1996                 1998-              2006-
                                                 IAT
                               libkvm            Rootkit             firmware



thompson   dark      hidesrc      amodload   Back          SSDT      virtualized
compiler   avenger                           Orifice       Rootkit
backdoor
Hyperjacking vs. Virtualization

                        idealized
   vmware                rootkit                hyperjacker

guest A   guest B
                            native OS
                                               native OS   vmm
      vmm
                              vmm
    hardware                hardware               hardware

                  small                 direct guest
                               =
                footprint               hardware io
Ad

Recommended

[Ruxcon 2011] Post Memory Corruption Memory Analysis
[Ruxcon 2011] Post Memory Corruption Memory Analysis[Ruxcon 2011] Post Memory Corruption Memory Analysis
[Ruxcon 2011] Post Memory Corruption Memory AnalysisMoabi.com
 
BSides Denver: Stealthy, hypervisor-based malware analysis
BSides Denver: Stealthy, hypervisor-based malware analysisBSides Denver: Stealthy, hypervisor-based malware analysis
BSides Denver: Stealthy, hypervisor-based malware analysisTamas K Lengyel
 
Hardware backdooring is practical : slides
Hardware backdooring is practical : slidesHardware backdooring is practical : slides
Hardware backdooring is practical : slidesMoabi.com
 
[Defcon] Hardware backdooring is practical
[Defcon] Hardware backdooring is practical[Defcon] Hardware backdooring is practical
[Defcon] Hardware backdooring is practicalMoabi.com
 
[Hackito2012] Hardware backdooring is practical
[Hackito2012] Hardware backdooring is practical[Hackito2012] Hardware backdooring is practical
[Hackito2012] Hardware backdooring is practicalMoabi.com
 
CrySys guest-lecture: Virtual machine introspection on modern hardware
CrySys guest-lecture: Virtual machine introspection on modern hardwareCrySys guest-lecture: Virtual machine introspection on modern hardware
CrySys guest-lecture: Virtual machine introspection on modern hardwareTamas K Lengyel
 
Pitfalls and limits of dynamic malware analysis
Pitfalls and limits of dynamic malware analysisPitfalls and limits of dynamic malware analysis
Pitfalls and limits of dynamic malware analysisTamas K Lengyel
 
Virtual Machine Introspection with Xen
Virtual Machine Introspection with XenVirtual Machine Introspection with Xen
Virtual Machine Introspection with XenTamas K Lengyel
 

More Related Content

What's hot

VM Forking and Hypervisor-based Fuzzing with Xen
VM Forking and Hypervisor-based Fuzzing with XenVM Forking and Hypervisor-based Fuzzing with Xen
VM Forking and Hypervisor-based Fuzzing with XenTamas K Lengyel
 
Hacktivity 2016: Stealthy, hypervisor based malware analysis
Hacktivity 2016: Stealthy, hypervisor based malware analysisHacktivity 2016: Stealthy, hypervisor based malware analysis
Hacktivity 2016: Stealthy, hypervisor based malware analysisTamas K Lengyel
 
VM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzingVM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzingTamas K Lengyel
 
OffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with XenOffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with XenTamas K Lengyel
 
Pitfalls of virtual machine introspection on modern hardware
Pitfalls of virtual machine introspection on modern hardwarePitfalls of virtual machine introspection on modern hardware
Pitfalls of virtual machine introspection on modern hardwareTamas K Lengyel
 
DefCon 2012 - Hardware Backdooring (Slides)
DefCon 2012 - Hardware Backdooring (Slides)DefCon 2012 - Hardware Backdooring (Slides)
DefCon 2012 - Hardware Backdooring (Slides)Michael Smith
 
Comparative Analysis of Personal Firewalls
Comparative Analysis of Personal FirewallsComparative Analysis of Personal Firewalls
Comparative Analysis of Personal FirewallsAndrej Šimko
 
Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!
Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!
Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!Anne Nicolas
 
Scalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis System
Scalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis SystemScalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis System
Scalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis SystemTamas K Lengyel
 
Stealthy, Hypervisor-based Malware Analysis
Stealthy, Hypervisor-based Malware AnalysisStealthy, Hypervisor-based Malware Analysis
Stealthy, Hypervisor-based Malware AnalysisTamas K Lengyel
 
D1 t2 jonathan brossard - breaking virtualization by switching to virtual 8...
D1 t2   jonathan brossard - breaking virtualization by switching to virtual 8...D1 t2   jonathan brossard - breaking virtualization by switching to virtual 8...
D1 t2 jonathan brossard - breaking virtualization by switching to virtual 8...kbour23
 
Exploitation and State Machines
Exploitation and State MachinesExploitation and State Machines
Exploitation and State MachinesMichael Scovetta
 
How to Root 10 Million Phones with One Exploit
How to Root 10 Million Phones with One ExploitHow to Root 10 Million Phones with One Exploit
How to Root 10 Million Phones with One ExploitJiahong Fang
 
ARMvisor @ Linux Symposium 2012
ARMvisor @ Linux Symposium 2012ARMvisor @ Linux Symposium 2012
ARMvisor @ Linux Symposium 2012Peter Chang
 
Malicious Hypervisor - Virtualization in Shellcodes by Adhokshaj Mishra
Malicious Hypervisor - Virtualization in Shellcodes by Adhokshaj MishraMalicious Hypervisor - Virtualization in Shellcodes by Adhokshaj Mishra
Malicious Hypervisor - Virtualization in Shellcodes by Adhokshaj MishraOWASP Delhi
 
[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory Analysis[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory AnalysisMoabi.com
 
ARMvisor, more details
ARMvisor, more detailsARMvisor, more details
ARMvisor, more detailsPeter Chang
 
sponsorAVAST-VB2014
sponsorAVAST-VB2014sponsorAVAST-VB2014
sponsorAVAST-VB2014Martin Hron
 
You didnt see it’s coming? "Dawn of hardened Windows Kernel"
You didnt see it’s coming? "Dawn of hardened Windows Kernel" You didnt see it’s coming? "Dawn of hardened Windows Kernel"
You didnt see it’s coming? "Dawn of hardened Windows Kernel" Peter Hlavaty
 
Kernel_Crash_Dump_Analysis
Kernel_Crash_Dump_AnalysisKernel_Crash_Dump_Analysis
Kernel_Crash_Dump_AnalysisBuland Singh
 

What's hot (20)

VM Forking and Hypervisor-based Fuzzing with Xen
VM Forking and Hypervisor-based Fuzzing with XenVM Forking and Hypervisor-based Fuzzing with Xen
VM Forking and Hypervisor-based Fuzzing with Xen
 
Hacktivity 2016: Stealthy, hypervisor based malware analysis
Hacktivity 2016: Stealthy, hypervisor based malware analysisHacktivity 2016: Stealthy, hypervisor based malware analysis
Hacktivity 2016: Stealthy, hypervisor based malware analysis
 
VM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzingVM Forking and Hypervisor-based fuzzing
VM Forking and Hypervisor-based fuzzing
 
OffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with XenOffensiveCon2022: Case Studies of Fuzzing with Xen
OffensiveCon2022: Case Studies of Fuzzing with Xen
 
Pitfalls of virtual machine introspection on modern hardware
Pitfalls of virtual machine introspection on modern hardwarePitfalls of virtual machine introspection on modern hardware
Pitfalls of virtual machine introspection on modern hardware
 
DefCon 2012 - Hardware Backdooring (Slides)
DefCon 2012 - Hardware Backdooring (Slides)DefCon 2012 - Hardware Backdooring (Slides)
DefCon 2012 - Hardware Backdooring (Slides)
 
Comparative Analysis of Personal Firewalls
Comparative Analysis of Personal FirewallsComparative Analysis of Personal Firewalls
Comparative Analysis of Personal Firewalls
 
Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!
Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!
Kernel Recipes 2014 - Writing Code: Keep It Short, Stupid!
 
Scalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis System
Scalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis SystemScalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis System
Scalability, Fidelity and Stealth in the DRAKVUF Dynamic Malware Analysis System
 
Stealthy, Hypervisor-based Malware Analysis
Stealthy, Hypervisor-based Malware AnalysisStealthy, Hypervisor-based Malware Analysis
Stealthy, Hypervisor-based Malware Analysis
 
D1 t2 jonathan brossard - breaking virtualization by switching to virtual 8...
D1 t2   jonathan brossard - breaking virtualization by switching to virtual 8...D1 t2   jonathan brossard - breaking virtualization by switching to virtual 8...
D1 t2 jonathan brossard - breaking virtualization by switching to virtual 8...
 
Exploitation and State Machines
Exploitation and State MachinesExploitation and State Machines
Exploitation and State Machines
 
How to Root 10 Million Phones with One Exploit
How to Root 10 Million Phones with One ExploitHow to Root 10 Million Phones with One Exploit
How to Root 10 Million Phones with One Exploit
 
ARMvisor @ Linux Symposium 2012
ARMvisor @ Linux Symposium 2012ARMvisor @ Linux Symposium 2012
ARMvisor @ Linux Symposium 2012
 
Malicious Hypervisor - Virtualization in Shellcodes by Adhokshaj Mishra
Malicious Hypervisor - Virtualization in Shellcodes by Adhokshaj MishraMalicious Hypervisor - Virtualization in Shellcodes by Adhokshaj Mishra
Malicious Hypervisor - Virtualization in Shellcodes by Adhokshaj Mishra
 
[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory Analysis[CCC-28c3] Post Memory Corruption Memory Analysis
[CCC-28c3] Post Memory Corruption Memory Analysis
 
ARMvisor, more details
ARMvisor, more detailsARMvisor, more details
ARMvisor, more details
 
sponsorAVAST-VB2014
sponsorAVAST-VB2014sponsorAVAST-VB2014
sponsorAVAST-VB2014
 
You didnt see it’s coming? "Dawn of hardened Windows Kernel"
You didnt see it’s coming? "Dawn of hardened Windows Kernel" You didnt see it’s coming? "Dawn of hardened Windows Kernel"
You didnt see it’s coming? "Dawn of hardened Windows Kernel"
 
Kernel_Crash_Dump_Analysis
Kernel_Crash_Dump_AnalysisKernel_Crash_Dump_Analysis
Kernel_Crash_Dump_Analysis
 

Viewers also liked

Crypto Strikes Back! (Google 2009)
Crypto Strikes Back! (Google 2009)Crypto Strikes Back! (Google 2009)
Crypto Strikes Back! (Google 2009)Nate Lawson
 
Foundations of Platform Security
Foundations of Platform SecurityFoundations of Platform Security
Foundations of Platform SecurityNate Lawson
 
ACPI and FreeBSD (Part 2)
ACPI and FreeBSD (Part 2)ACPI and FreeBSD (Part 2)
ACPI and FreeBSD (Part 2)Nate Lawson
 
When Crypto Attacks! (Yahoo 2009)
When Crypto Attacks! (Yahoo 2009)When Crypto Attacks! (Yahoo 2009)
When Crypto Attacks! (Yahoo 2009)Nate Lawson
 
Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)
Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)
Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)Nate Lawson
 
Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)
Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)
Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)Nate Lawson
 
ACPI and FreeBSD (Part 1)
ACPI and FreeBSD (Part 1)ACPI and FreeBSD (Part 1)
ACPI and FreeBSD (Part 1)Nate Lawson
 
Highway to Hell: Hacking Toll Systems (Blackhat 2008)
Highway to Hell: Hacking Toll Systems (Blackhat 2008)Highway to Hell: Hacking Toll Systems (Blackhat 2008)
Highway to Hell: Hacking Toll Systems (Blackhat 2008)Nate Lawson
 
Designing and Attacking DRM (RSA 2008)
Designing and Attacking DRM (RSA 2008)Designing and Attacking DRM (RSA 2008)
Designing and Attacking DRM (RSA 2008)Nate Lawson
 
TLS Optimization
TLS OptimizationTLS Optimization
TLS OptimizationNate Lawson
 
TLS/SSL MAC security flaw
TLS/SSL MAC security flawTLS/SSL MAC security flaw
TLS/SSL MAC security flawNate Lawson
 
TLS/SSL Protocol Design 201006
TLS/SSL Protocol Design 201006TLS/SSL Protocol Design 201006
TLS/SSL Protocol Design 201006Nate Lawson
 
TLS/SSL Protocol Design
TLS/SSL Protocol DesignTLS/SSL Protocol Design
TLS/SSL Protocol DesignNate Lawson
 
Presentation On Cross Cultural Communication
Presentation On Cross Cultural CommunicationPresentation On Cross Cultural Communication
Presentation On Cross Cultural Communicationshubham1306
 
INTRODUCTION TO C PROGRAMMING
INTRODUCTION TO C PROGRAMMINGINTRODUCTION TO C PROGRAMMING
INTRODUCTION TO C PROGRAMMINGAbhishek Dwivedi
 

Viewers also liked (16)

Crypto Strikes Back! (Google 2009)
Crypto Strikes Back! (Google 2009)Crypto Strikes Back! (Google 2009)
Crypto Strikes Back! (Google 2009)
 
Foundations of Platform Security
Foundations of Platform SecurityFoundations of Platform Security
Foundations of Platform Security
 
ACPI and FreeBSD (Part 2)
ACPI and FreeBSD (Part 2)ACPI and FreeBSD (Part 2)
ACPI and FreeBSD (Part 2)
 
When Crypto Attacks! (Yahoo 2009)
When Crypto Attacks! (Yahoo 2009)When Crypto Attacks! (Yahoo 2009)
When Crypto Attacks! (Yahoo 2009)
 
Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)
Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)
Copy Protection Wars: Analyzing Retro and Modern Schemes (RSA 2007)
 
Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)
Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)
Using FreeBSD to Design a Secure Digital Cinema Server (Usenix 2004)
 
ACPI and FreeBSD (Part 1)
ACPI and FreeBSD (Part 1)ACPI and FreeBSD (Part 1)
ACPI and FreeBSD (Part 1)
 
Highway to Hell: Hacking Toll Systems (Blackhat 2008)
Highway to Hell: Hacking Toll Systems (Blackhat 2008)Highway to Hell: Hacking Toll Systems (Blackhat 2008)
Highway to Hell: Hacking Toll Systems (Blackhat 2008)
 
Designing and Attacking DRM (RSA 2008)
Designing and Attacking DRM (RSA 2008)Designing and Attacking DRM (RSA 2008)
Designing and Attacking DRM (RSA 2008)
 
TLS Optimization
TLS OptimizationTLS Optimization
TLS Optimization
 
TLS/SSL MAC security flaw
TLS/SSL MAC security flawTLS/SSL MAC security flaw
TLS/SSL MAC security flaw
 
TLS/SSL Protocol Design 201006
TLS/SSL Protocol Design 201006TLS/SSL Protocol Design 201006
TLS/SSL Protocol Design 201006
 
TLS/SSL Protocol Design
TLS/SSL Protocol DesignTLS/SSL Protocol Design
TLS/SSL Protocol Design
 
Presentation On Cross Cultural Communication
Presentation On Cross Cultural CommunicationPresentation On Cross Cultural Communication
Presentation On Cross Cultural Communication
 
INTRODUCTION TO C PROGRAMMING
INTRODUCTION TO C PROGRAMMINGINTRODUCTION TO C PROGRAMMING
INTRODUCTION TO C PROGRAMMING
 
Slideshare ppt
Slideshare pptSlideshare ppt
Slideshare ppt
 

Similar to Don't Tell Joanna the Virtualized Rootkit is Dead (Blackhat 2007)

IoT exploitation: from memory corruption to code execution by Marco Romano
IoT exploitation: from memory corruption to code execution by Marco RomanoIoT exploitation: from memory corruption to code execution by Marco Romano
IoT exploitation: from memory corruption to code execution by Marco RomanoCodemotion
 
IoT exploitation: from memory corruption to code execution - Marco Romano - C...
IoT exploitation: from memory corruption to code execution - Marco Romano - C...IoT exploitation: from memory corruption to code execution - Marco Romano - C...
IoT exploitation: from memory corruption to code execution - Marco Romano - C...Codemotion
 
Larson Macaulay apt_malware_past_present_future_out_of_band_techniques
Larson Macaulay apt_malware_past_present_future_out_of_band_techniquesLarson Macaulay apt_malware_past_present_future_out_of_band_techniques
Larson Macaulay apt_malware_past_present_future_out_of_band_techniquesScott K. Larson
 
White Lightning Sept 2014
White Lightning Sept 2014White Lightning Sept 2014
White Lightning Sept 2014Bryce Kunz
 
[HackInTheBox] Breaking virtualization by any means
[HackInTheBox] Breaking virtualization by any means[HackInTheBox] Breaking virtualization by any means
[HackInTheBox] Breaking virtualization by any meansMoabi.com
 
Playing CTFs for Fun & Profit
Playing CTFs for Fun & ProfitPlaying CTFs for Fun & Profit
Playing CTFs for Fun & Profitimpdefined
 
XS Boston 2008 Paravirt Ops in Linux IA64
XS Boston 2008 Paravirt Ops in Linux IA64XS Boston 2008 Paravirt Ops in Linux IA64
XS Boston 2008 Paravirt Ops in Linux IA64The Linux Foundation
 
[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode
[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode
[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 modeMoabi.com
 
Inside the Matrix,How to Build Transparent Sandbox for Malware Analysis
Inside the Matrix,How to Build Transparent Sandbox for Malware AnalysisInside the Matrix,How to Build Transparent Sandbox for Malware Analysis
Inside the Matrix,How to Build Transparent Sandbox for Malware AnalysisChong-Kuan Chen
 
Software Defined Data Centers - June 2012
Software Defined Data Centers - June 2012Software Defined Data Centers - June 2012
Software Defined Data Centers - June 2012Brent Salisbury
 
OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...
OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...
OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...The Linux Foundation
 
The Potential Impact of Software Defined Networking SDN on Security
The Potential Impact of Software Defined Networking SDN on SecurityThe Potential Impact of Software Defined Networking SDN on Security
The Potential Impact of Software Defined Networking SDN on SecurityBrent Salisbury
 
Writing Metasploit Plugins
Writing Metasploit PluginsWriting Metasploit Plugins
Writing Metasploit Pluginsamiable_indian
 
Initiation concrète-à-la-virtualisation-devoxx-fr-2021
Initiation concrète-à-la-virtualisation-devoxx-fr-2021Initiation concrète-à-la-virtualisation-devoxx-fr-2021
Initiation concrète-à-la-virtualisation-devoxx-fr-2021Pierre-Antoine Grégoire
 
Buffer overflow – Smashing The Stack
Buffer overflow – Smashing The StackBuffer overflow – Smashing The Stack
Buffer overflow – Smashing The StackTomer Zait
 
How Triton can help to reverse virtual machine based software protections
How Triton can help to reverse virtual machine based software protectionsHow Triton can help to reverse virtual machine based software protections
How Triton can help to reverse virtual machine based software protectionsJonathan Salwan
 
BlueHat v18 || A mitigation for kernel toctou vulnerabilities
BlueHat v18 || A mitigation for kernel toctou vulnerabilitiesBlueHat v18 || A mitigation for kernel toctou vulnerabilities
BlueHat v18 || A mitigation for kernel toctou vulnerabilitiesBlueHat Security Conference
 
Buffer Overflow - Smashing the Stack
Buffer Overflow - Smashing the StackBuffer Overflow - Smashing the Stack
Buffer Overflow - Smashing the StackironSource
 

Similar to Don't Tell Joanna the Virtualized Rootkit is Dead (Blackhat 2007) (20)

IoT exploitation: from memory corruption to code execution by Marco Romano
IoT exploitation: from memory corruption to code execution by Marco RomanoIoT exploitation: from memory corruption to code execution by Marco Romano
IoT exploitation: from memory corruption to code execution by Marco Romano
 
IoT exploitation: from memory corruption to code execution - Marco Romano - C...
IoT exploitation: from memory corruption to code execution - Marco Romano - C...IoT exploitation: from memory corruption to code execution - Marco Romano - C...
IoT exploitation: from memory corruption to code execution - Marco Romano - C...
 
Larson Macaulay apt_malware_past_present_future_out_of_band_techniques
Larson Macaulay apt_malware_past_present_future_out_of_band_techniquesLarson Macaulay apt_malware_past_present_future_out_of_band_techniques
Larson Macaulay apt_malware_past_present_future_out_of_band_techniques
 
White Lightning Sept 2014
White Lightning Sept 2014White Lightning Sept 2014
White Lightning Sept 2014
 
[HackInTheBox] Breaking virtualization by any means
[HackInTheBox] Breaking virtualization by any means[HackInTheBox] Breaking virtualization by any means
[HackInTheBox] Breaking virtualization by any means
 
Playing CTFs for Fun & Profit
Playing CTFs for Fun & ProfitPlaying CTFs for Fun & Profit
Playing CTFs for Fun & Profit
 
XS Boston 2008 Paravirt Ops in Linux IA64
XS Boston 2008 Paravirt Ops in Linux IA64XS Boston 2008 Paravirt Ops in Linux IA64
XS Boston 2008 Paravirt Ops in Linux IA64
 
[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode
[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode
[Ruxcon] Breaking virtualization by switching the cpu to virtual 8086 mode
 
You suck at Memory Analysis
You suck at Memory AnalysisYou suck at Memory Analysis
You suck at Memory Analysis
 
Inside the Matrix,How to Build Transparent Sandbox for Malware Analysis
Inside the Matrix,How to Build Transparent Sandbox for Malware AnalysisInside the Matrix,How to Build Transparent Sandbox for Malware Analysis
Inside the Matrix,How to Build Transparent Sandbox for Malware Analysis
 
Software Defined Data Centers - June 2012
Software Defined Data Centers - June 2012Software Defined Data Centers - June 2012
Software Defined Data Centers - June 2012
 
OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...
OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...
OSSEU17: How Open Source Project Xen Puts Security Software Vendors Ahead of ...
 
The Potential Impact of Software Defined Networking SDN on Security
The Potential Impact of Software Defined Networking SDN on SecurityThe Potential Impact of Software Defined Networking SDN on Security
The Potential Impact of Software Defined Networking SDN on Security
 
Writing Metasploit Plugins
Writing Metasploit PluginsWriting Metasploit Plugins
Writing Metasploit Plugins
 
VXCON 2017
VXCON 2017VXCON 2017
VXCON 2017
 
Initiation concrète-à-la-virtualisation-devoxx-fr-2021
Initiation concrète-à-la-virtualisation-devoxx-fr-2021Initiation concrète-à-la-virtualisation-devoxx-fr-2021
Initiation concrète-à-la-virtualisation-devoxx-fr-2021
 
Buffer overflow – Smashing The Stack
Buffer overflow – Smashing The StackBuffer overflow – Smashing The Stack
Buffer overflow – Smashing The Stack
 
How Triton can help to reverse virtual machine based software protections
How Triton can help to reverse virtual machine based software protectionsHow Triton can help to reverse virtual machine based software protections
How Triton can help to reverse virtual machine based software protections
 
BlueHat v18 || A mitigation for kernel toctou vulnerabilities
BlueHat v18 || A mitigation for kernel toctou vulnerabilitiesBlueHat v18 || A mitigation for kernel toctou vulnerabilities
BlueHat v18 || A mitigation for kernel toctou vulnerabilities
 
Buffer Overflow - Smashing the Stack
Buffer Overflow - Smashing the StackBuffer Overflow - Smashing the Stack
Buffer Overflow - Smashing the Stack
 

Recently uploaded

Importance of magazines in education ppt
Importance of magazines in education pptImportance of magazines in education ppt
Importance of magazines in education pptsafnarafeek2002
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersThousandEyes
 
2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch TuesdayIvanti
 
Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.Mateusz Kwasniewski
 
Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Daniel Toomey
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
Q1 Memory Fabric Forum: XConn CXL Switches for AI
Q1 Memory Fabric Forum: XConn CXL Switches for AIQ1 Memory Fabric Forum: XConn CXL Switches for AI
Q1 Memory Fabric Forum: XConn CXL Switches for AIMemory Fabric Forum
 
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptxssuser796efb
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfDomotica daVinci
 
Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellencePrecisely
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfDomotica daVinci
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!XfilesPro
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!KivenRaySarsaba
 
Manual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveManual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveDomotica daVinci
 
Journey of Television in World & in India
Journey of Television in World & in IndiaJourney of Television in World & in India
Journey of Television in World & in IndiaAdarshAgarwal66
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS Chicago
 
From eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the ManufacturingFrom eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the ManufacturingSoracom Global, Inc.
 
Navigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersNavigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersOnePlan Solutions
 
Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxMaarten Balliauw
 
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfZ-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfDomotica daVinci
 

Recently uploaded (20)

Importance of magazines in education ppt
Importance of magazines in education pptImportance of magazines in education ppt
Importance of magazines in education ppt
 
Enhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for PartnersEnhancing SaaS Performance: A Hands-on Workshop for Partners
Enhancing SaaS Performance: A Hands-on Workshop for Partners
 
2024 February Patch Tuesday
2024 February Patch Tuesday2024 February Patch Tuesday
2024 February Patch Tuesday
 
Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.Put a flag on it. A busy developer's guide to feature toggles.
Put a flag on it. A busy developer's guide to feature toggles.
 
Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024Microsoft Azure News - Feb 2024
Microsoft Azure News - Feb 2024
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
Q1 Memory Fabric Forum: XConn CXL Switches for AI
Q1 Memory Fabric Forum: XConn CXL Switches for AIQ1 Memory Fabric Forum: XConn CXL Switches for AI
Q1 Memory Fabric Forum: XConn CXL Switches for AI
 
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
2) Presentation_Overview_ISO_16140-3_Method_verification_20210322.pptx
 
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdfTete thermostatique Zigbee MOES BRT-100 V2.pdf
Tete thermostatique Zigbee MOES BRT-100 V2.pdf
 
Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center Excellence
 
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdfQuinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
Quinto Z-Wave Heltun_HE-RS01_User_Manual_B9AH.pdf
 
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
5 Things You Shouldn’t Do at Salesforce World Tour Sydney 2024!
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!
 
Manual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-WaveManual Eurotronic Thermostatic Valve Comry Z-Wave
Manual Eurotronic Thermostatic Valve Comry Z-Wave
 
Journey of Television in World & in India
Journey of Television in World & in IndiaJourney of Television in World & in India
Journey of Television in World & in India
 
AWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user groupAWS reInvent 2023 recaps from Chicago AWS user group
AWS reInvent 2023 recaps from Chicago AWS user group
 
From eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the ManufacturingFrom eSIMs to iSIMs: It’s Inside the Manufacturing
From eSIMs to iSIMs: It’s Inside the Manufacturing
 
Navigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio LeadersNavigating the Never Normal Strategies for Portfolio Leaders
Navigating the Never Normal Strategies for Portfolio Leaders
 
Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptx
 
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdfZ-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
Z-Wave Fan coil Thermostat Heltun_HE-HT01_User_Manual.pdf
 

Don't Tell Joanna the Virtualized Rootkit is Dead (Blackhat 2007)

  • 1. Don’t Tell Joanna The Virtualized Rootkit Is Dead Nate Lawson @ Root Labs Peter Ferrie @ Symantec Thomas Ptacek @ Matasano
  • 3. What We’re Going To Talk About ★ HVM Malware Recap ★ Nothing Is 100% Undetectable ★ Samsara: A Framework For HVM Malware Detection ★ Conclusion: A Cat And Mouse Game
  • 5. We’ve Seen This Movie Before 1984 ‘90s 1994 - 1996 1998- 2006- IAT libkvm Rootkit firmware thompson dark hidesrc amodload Back SSDT virtualized compiler avenger Orifice Rootkit backdoor
  • 6. Hyperjacking vs. Virtualization idealized vmware rootkit hyperjacker guest A guest B native OS native OS vmm vmm vmm hardware hardware hardware small direct guest = footprint hardware io
  • 7. Hyperjacker Analogy: WebScarab browser web proxy web server Almost all hardware functionality is left untouched. VMM picks and chooses (via trap handler) what to manipulate. kernel VMM CPU Vitriol 1 is less than 2000 lines of C code.
  • 8. X86 System Hierarchy software instructions ISA CPU microarchitecture VT cache sched exec tlb btb chipset peripheral bus hardware
  • 9. Vitriol: ddz’s HVM Hyperjacker ★ Dino Dai Zovi’s 2006 Matasano Black Hat Talk ring 0 (OS X) ring -1 (Vitriol) cpu (Core Duo) vmx_fork() initialize vmm rdmsr 1ch vmexit rdmsr vmenter int 80h int
  • 10. Blue Pill: Joanna’s HVM Rootkit ★ Joanna Rutkowska’s 2006 COSEINC Black Hat Talk ★ Just Like Vitriol, but: – uses AMD SVM, not Intel VT-x – Vista, not OS X – loads self via Vista (beta) swap bug – implements network IO with debug registers – loads LWIP stack into the kernel – apparently implements nested virtual machines ★ Claim: 100% Undetectable Malware
  • 12. Detecting VMWare Is Easy ★ Unrealistic outdated device hardware – ISA ethernet controller? – 440BX chipset? ★ Holes in virtualization – SIDT – Microsoft-manufactured motherboard – Registry keys – “VMware” in video and SMBIOS strings ★ Guest-to-host communication channels – VMware: inb/outb to magic port – VirtualPC: illegal instructions ★ Wide timing variances – Hard to trap timer reads (RDTSC), accuracy suffers
  • 13. Cross Section A measure of stealth, quantified by intrusiveness. kernel virtualized malware malware osx x86 kernel hardware
  • 14. Cross Section for Virtualization cross section ● noun: amount of the original system that rootkit must emulate to remain hidden ● etymology: radar, stealth planes ★ Varies by layer chosen for rootkit ★ Dictates complexity of rootkit – fails to trap and emulate a feature: detectable – emulation too complex: big target, detectable ★ “Entire x86 hardware platform” is a huge cross-section
  • 15. Fundamental Problem This instruction: cpuid should take 200 cycles, not 5000, is unprivileged, and should have no impact on cache, BTB, or TLB.
  • 16. Three Detection Strategies ★ Strategy 1: Side-Channel Attacks VM overhead creates detectable “trails” through microarchitecture that are prohibitively hard to conceal. ★ Strategy 2: Vantage-Point Attacks VM cross-section forces it to recognize and emulate many obscure hardware features. ★ Strategy 3: Vulnerability Analysis The more features a rootkit implements to hide itself, the more bugs it exposes.
  • 17. What Is A Side Channel Attack? ★ Any resource consumed in a logic-dependent way leaks information. For example: if (strcmp(guessPassword, realPassword)) return LOGIN_FAILED; ‘83: Tenex ‘95: Timing Breaks ‘98: DPA Breaks ‘04: Remote Timing ‘06: BTB Timing Password (Lampson) RSA (Kocher) Smartcards (Kocher) Breaks SSL (Boneh) Breaks RSA (Aciicmez)
  • 18. But Real Systems Are Too Noisy! BZZZZZT! ★ Astounding: “Opportunities and Limits of Remote Timing Attacks”; Crosby, Riedi, Wallach – WAN timing: 15-100 microseconds resolution (!) – LAN: 100 nanoseconds ★ If noisy, take more samples and average – Decouples noise from true resource consumption – Local access = higher resolution ★ We monopolize ring 0 for several microseconds – Less than an AV scanner ★ Data-dependent side-channels are hard to eliminate.
  • 19. Finding Side-Channel Attacks ★ How to look for it: – Enumerate all resources your opponent has to use • say, “executing instructions” – Identify how to measure that resource from your vantage point • Branch target buffer state can be read by timing branches in your own thread – Take as many measurements as possible – Eliminate jitter with Stat 101 ★ Secret: identifying whether or not any code (i.e., hypervisor) executed much easier than extracting a key from that code
  • 20. Vantage Point Attacks ★ Dilemma: either let me talk directly to the hardware, which will betray you, or emulate the hardware, with perfect fidelity. ★ HPET: alternative high-precision timers (supercedes but does not eliminate the RTC). ★ Performance Event Counters: instructions retired (in/ out of VMM), cache misses, branch mispredictions, model-specific events. ★ GART: scatter-gather memory map for graphics devices
  • 21. Finding Vantage Point Attacks ★ Embedded timers (exposed directly, via CSR, or indirectly via behavior). Force Blue Pill to emulate every mainstream peripheral. ★ Model-specific MSRs and CSRs (particularly scattered in sensitive functionality). Not all MSRs and CSRs are documented. ★ “Bounceable” Memory Access through devices (DMA rings, etc). Force Blue Pill to emulate every mainstream peripheral.
  • 22. Finding Hypervisor Bugs ★ Sources of Bugs – nested/VTX talk about – errata vmware – vtx loading errata interrupt bug ★ Where To Look – Get all datasheets and errata – Descriptions give microarchitectural behavior (i.e., priority of faults in various error cases) – Focus on items that require a full simulator to emulate correctly – Or, behavior that can’t be trapped/emulated – Special credit: “won’t fix” errata
  • 23. Samsara a framework for detecting virtualized malware
  • 24. Implementation overview ★ three tests implemented – Instruction and data TLB – HPET – VT Errata ★ test framework ★ rantipole: HVM rootkit simulator ★ futures
  • 25. What’s a TLB? bffff430h Translation Lookaside Buffer 1 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 1 0 0 0 0 1 1 0 0 0 0 –Remembers the translated addresses of memory you touched •Like a cache for page tables (Virtual memory CR3 101) •Not directly visible from software but you can pde+X pde+Y affect it –Flush it completely (MOV CR3/CR4) –Flush an individual page (INVLPG) pte+N pte+K pte+J –Indirectly by R/W/X from a page When a VMEXIT occurs, hypervisor execution and memory access leave trails through the TLBs
  • 26. General Approach To Snooping should be identical to This sequence: this sequence: desync tlb desync tlb color memory color memory cpuid nop read memory read memory
  • 27. TLB Snooping (0) RAM 1 (1) PTE pte, tlb synced, (2) TLB TLB (3) RAM “random” pte, tlb desynced, 2 RAM colored post vmexit: 3(a) basic VT-x post vmexit: AMD SVM 3(b)
  • 28. Aside: ASID’s Don’t Help, Joanna ★ “But guest TLB usage can be partitioned with ASID” – Application Space ID, tag that says which context owns a TLB entry ★ Made for performance, not transparency! ★ Hypervisor still needs to read exit condition from VM ★ VMEXIT trap handler needs to run – Hence, at least one data and instruction TLB entry will be lost ★ Hypervisor has to now flush TLB manually for guest – Observable timing difference = side channel
  • 29. Data TLB Test implementation ★ Allocate N+1 pages ★ Fill N pages with “A”, one page with “B” ★ Touch the pages in order to flush TLB and fill with those mappings ★ Remap N pages to one page without invalidating the TLB ★ Walk backwards through all N pages, reading data – Keep a counter – Once you see “B”, you’ve flushed the TLB ★ Repeat test but add probe to trap to hypervisor ★ If count varies with trapping instruction, you’re virtualized
  • 30. Instructions Have Their Own TLB ★ Instruction TLB test mostly same – Pattern A = mov eax, 1; ret – Pattern B = mov eax, 2; ret – “Reading” pages by call $PAGE, check value in eax afterwards ★ Same design pattern works for other tests ★ Branch target buffer (BTB): – Modern x86 caches branch targets. Saturate the BTB cache and evictions will make certain branches take observably longer. ★ Cache invalidation (INVD): – INVD flushes the cache but discards queued writes. Saturate the cache, cause VMEXIT, and then INVD out the whole cache; evictions will make certain writes persistent.
  • 31. Vantage Point Example: HPET ★ Clock VM exits via the HPET instead of the TSC. ★ Kernel: – export sysctl “hpet_attach” – map HPET IO range to requesting processes ★ Userland: – (1) sample hpet – (2) sample tsc – (3) cpuid (or other vmexit) – (4) elapsed tsc – (5) elapsed hpet – (6) reconcile – (7) repeat
  • 32. Other Timing Sources ★ The PIT (16 bit counter/interrupt timer) ★ The RTC (battery-backed realtime counter [1 sec granular] and interrupt timer) ★ The LAPIC (32 bit, device-dependent frequency) ★ The ACPI timer (24 bit PITx3 timer) ★ VGA blanking – Fun drinking game: “count the exposed embedded timers in peripherals”.
  • 33. Bugcheck Example: VMCS Fuzzer ★ VMCS stores host (hypervisor) state, guest (OS X) state, execution controls, and status. ★ VT-x dox specify 50+ checks on fields, checked in well-defined model-specific order, generating different exceptions (GPF, SIGILL, VMLAUNCH error, etc). ★ Break each check, verify expected exception/signal/ result.
  • 34. Extending Samsara ★ Plug-in interface for adding new tests – Kernel or userland – Tests use configurable probe function ★ Userland support programs – Query and generate traces – Analyze results of traces ★ Profiles/”signatures” for well-known hardware ★ Runs on MacOS and FreeBSD – Easy to port to new archs
  • 35. Rantipole: An HVM Testbed ★ Rantipole is a (crippled) HVM detection testbed – only works on OS X – only works on Core 1 Duo – only works in native 32 bit mode – loudly advertises itself – stripped of any SMP sync code (UP-only) – no “backdoor” or “malicious” capabilities – self-destructs in 10 minutes ★ Malware authors: you are better off reading Xen
  • 36. What Rantipole Does ★ (1) check cpuid, feature msr for VMX ★ (2) allocate vmx and vmcs from IOMallocContiguous ★ (3) initialize vmcs, call vmclear ★ (4) copy segments, stack, cr3 to vmcs host and guest ★ (5) set host/root/hypervisor eip to trap handler ★ (6) set exec controls to pick events we want ★ (7) vmptrld to add vmcs ★ (8) (a) vmlaunch (b) spin
  • 37. Futures ★ Implement more tests – VGA blanking interval timer – BTB – INVD – Multi-core cooperating threads ★ Improve simulator to validate tests ★ Port to new archs tom: BAT SIGNAL ★ None of this is really important anyway DIAGRAM HERE
  • 38. Conclusion a cat and mouse game
  • 39. The Cat And Mouse Process Diagram: “research”, “implement”, “deploy”, “repeat” (circles, arrows, huge cliche, boring!): research---------. | | V implement ----. | V deploy `___________/ Yes, you’ve seen it before!
  • 40. Questions are your way of proving to Joanna that you paid attention.