Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Metasploit Framework Executable Encoding


Published on

A school project. How well does Metasploit encode its executables? I'm putting it to the test against 10 different antivirus programs. This is just the proposal; the finished version should be done in a month or so.

Published in: Technology
  • Be the first to comment

Metasploit Framework Executable Encoding

  1. 1. Metasploit Payload Encoding and Antivirus Detection
  2. 2. Research Question <ul><li>How well does Metasploit’s executable encoding prevent detection by antivirus software? </li></ul>
  3. 3. Hypothesis & Null Hypothesis <ul><li>Shikata na Gai encoding scheme will result in the lowest number of detections by antivirus software, because it utilizes a polymorphic engine.
  4. 4. There is no statistical difference between antivirus detection of executables that have been encoded using Metasploit </li></ul>
  5. 5. Introduction <ul><li>Rationale </li><ul><li>War between antivirus software and malware programmers
  6. 6. What is Metasploit?
  7. 7. How well are we protected from malware?
  8. 8. Negative affects of malware
  9. 9. Can a simple encoding scheme render AV useless?
  10. 10. What about more advanced encoders?
  11. 11. How well can AV software defend against easily accessable encoders provided by the Metasploit framwork? </li></ul></ul>
  12. 12. Terms <ul><li>Malware </li><ul><li>Any file that does things to a computer that a view does not want it to, or is not aware o </li></ul><li>MD5 </li><ul><li>Method of creating a ”fingprint” for a file. Unique to EVERY file (3.4 * 10^38 combinations possible) </li></ul><li>Binary file </li><ul><li>An executable program's file </li></ul><li>Sandbox </li><ul><li>Virtual area that is completely separated from the host computer </li></ul></ul>
  13. 13. Terms Continued <ul><li>Compile </li><ul><li>To turn readable code into an executable file </li></ul><li>Encryption + Decryption </li><ul><li>Encryption turn information into seemingly random information, using a key (like a password). Decryption reverses this </li></ul><li>Exploit </li><ul><li>To take advantage of a programer's mistakes. Can make a computer execute programs. Aka- vulnerability, 0day </li></ul><li>TCP/IP </li><ul><li>Protocol that the Internet mainly works on </li></ul><li>Virtual Machine </li><ul><li>Program used to emulate (simulate) an entire computer </li></ul></ul>
  14. 14. Background: How AV Works <ul><li>2 main methods of detection: </li><ul><li>File signature
  15. 15. Heuristics </li></ul></ul>
  16. 16. File Signatures <ul><li>Most common method
  17. 17. Algorithm or static signature </li><ul><li>Ex/ MD5 or look for suspitious behavior </li></ul></ul>Person using AV to scan a file Compare signature of file to a database with known signatures Does it match the signature of a known virus? Compute a signature from the unknown file Unkown file Database of file signatures
  18. 18. File Signature Scanning <ul><li>Benefits: </li><ul><li>Easy to implement
  19. 19. Not resource intensive </li></ul><li>Limitations </li><ul><li>Useless against new threats </li></ul></ul>
  20. 20. Heuristic Scanning Scan the file Person using AV Run it in a sandbox Monitor system calls & activity and relay info to a risk analysis engine Does it look suspicious? Unknown file Unknown file Risk analysis engine
  21. 21. Heuristic Scanning <ul><li>Benefits: </li><ul><li>Can detect new malware
  22. 22. Doesn't rely on a signature database </li></ul><li>Limitations: </li><ul><li>False positives
  23. 23. Resource intensive </li></ul></ul>
  24. 24. How To Avoid Detection <ul><li>Polymorphism </li><ul><li>Functional portion of code is encrypted
  25. 25. Decrypted on runtime
  26. 26. Change encryption/decryption key each time run </li></ul><li>Metamorphism </li><ul><li>Each time malware replicates, change itself </li></ul><li>Oligomorphism </li><ul><li>Similar to polymorphism
  27. 27. Chooses decryptor from set of key combinations </li></ul></ul>
  28. 28. Polymorphism hwfeidedwefef dfewfewfewfe grvervklmwefwe welkfimj eifcjm cwif jioregio mg wwoijmgeirojg r Encrypted portion Encryption + decryption engine with key Bla bla key=10 If (decrypted) { EvilStuff(); } Bla bla key=10 Decrypt the main code run Do evil things to the computer If (decrypted) { EvilStuff(); } Bla bla key=10 If (decrypted) { EvilStuff(); } Bla bla key=10 Change the encrytion + decryption engine, And change the key Kjlkmdckldklcm Sdclknmewnge Sdklmroivnslkw Kmewvionjrewg Wenmgrerjnkng spowgnjrekjwe Bah bah key=11 Encrypt the main code with new key and engine
  29. 29. Metamorphism 0x74 0x68 0x69 0x73 0x20 0x69 0x73 0x20 0x70 0x77 0x6e 0x7a 0x6f 0x72 0x20 0x63 0x6f 0x64 Virus (hex view) Runs and does evil stuff void main() { EvilFunction(); } void main() { EvilFunction(); UselessFunction(); } Takes its own source code and adds stuff A useless piece is added (like a NOP slide) Recompiled with new code 0x7a 0x6f 0x6d 0x67 0x20 0x64 0x69 0x73 0x20 0x69 0x00 0x00 0x00 0x00 0x00 0x73 0x20 0x6e A new binary is produced
  30. 30. Oligomorphism welkfimj eifcjm cwif jioregio mg wwoijmgeirojg r [engine] key=??? Encrypted portion Decryptor/Key Pieces If (decrypted) { EvilStuff(123); } key=a+c/b If (decrypted) { EvilStuff(321); } key=b+c^d If (decrypted) { EvilStuff(213); } key=b%2 +c A B C D
  31. 31. Metasploit Itself <ul><li>Exploit framwork
  32. 32. Ruby
  33. 33. Exploits + payloads
  34. 34. >1mil annual downloads
  35. 35. Constantly updated with wild exploits
  36. 36. Can produce standalone trojan binaries </li></ul>
  37. 37. In this study... <ul><li>Reverse TCP Payload </li><ul><li>Reverse= victim connects to attacker
  38. 38. TCP= done over Internet </li></ul><li>All information for connection hard encoded in executable </li></ul>Connection! Unknowing victim Evil hacker
  39. 39. Variables <ul><li>Independent: </li><ul><li>Type of encoder used </li></ul><li>Dependent: </li><ul><li>Whether or not the malware is detected by AV </li></ul></ul>
  40. 40. Controls <ul><li>Positive: </li><ul><li>Unencoded executable </li></ul><li>Negative Control: </li><ul><li>A file known to be benign
  41. 41. ” hello world” program in C compiled on clean Ubuntu install </li></ul></ul><ul><li>Constants: </li><ul><li>Environment scan is performed is identical
  42. 42. Snapshot feature on Vmware </li></ul><li>Same version of AV </li><ul><li>All 2011 </li></ul><li>All AV updated to most recent virus definitions
  43. 43. Same version & installation of Metasploit used </li></ul>
  44. 44. Controls and More <ul><li>Same exploit & payload
  45. 45. Same reverse TCP information (IP, port) used
  46. 46. All encodings were done on the same original executable </li></ul><ul><li>Replicates </li><ul><li>Scans repeated 5 times each trial to ensure accuracy </li></ul></ul>
  47. 47. Data Collection
  48. 48. Analysis <ul><li>Is there statistical difference between encoders?
  49. 49. Chi Square Test </li></ul>
  50. 50. References Instruction set reference manual. (1999). Intel architecture software developer's manual . Retrieved February 3, 2011, from Glossary - securelist . (2011). Retrieved from Metasploit express user guide. (2010). Rapid7 , (3.5.1), Retrieved from Metasploit framework . (2010). Retrieved from Munro, J. (2002, July 10). Antivirus research and detection techniques . Retrieved from,2845,1154648,00.asp Static application data. (2008). Uninformed, 9 . Retrieved from