A New Detection Framework<br />What would you do with a pointer and a size?<br />
Define the problem<br />Why do we need a new detection framework?<br />
The Challenge is Different<br />Attacks have switched from server attacks to client attacks<br />Common attack vectors are...
Network Systems…<br />Inline systems must emulate the processing of 1000s of desktops<br />Detection of many backchannels ...
Coverage Gap<br />Broadly speaking, IDS systems deal with packet-by-packet inspection with some level of reassembly<br />B...
Fill the Gap<br />A system is needed that can handle varied detection needs<br />A system is needed that extensible, open ...
NRT Framework<br />Near-Realtime Detection Framework or:<br />“Anything is Possible”<br />
The Dispatcher<br />The heart of the NRT system<br />APIs to handle:<br />Deep Inspection Nugget registration<br />Data Ha...
The Dispatcher<br /><ul><li>  Implements a database to provide a  centralized set of file information and
 Handles incoming queries for Data Handlers that have failed local cache hits
 Handles detection requests from both Data Handlers and DINs
 Handles incoming results from Deep Inspection Nuggets
  Handles database updates based on DIN data
 Writes out verbose logging based on DIN data
 Provides alerting to Data Handlers</li></li></ul><li>Data Handler<br />Capture data and metadata<br />Contact dispatcher ...
The Data Handler<br /><ul><li>  Data (in this case a file) is captured
  Metadata is captured (in this case URL and filename)
  A local cache of MD5 sums and URLs of files previously collected
 A library to handle managing the initial file evaluation, cache checks and communication with the Dispatcher</li></li></u...
The Deep Inspection Nugget<br /><ul><li>  Registers with the Dispatcher
 Processes data provided by the Data Handlers, as instructed by the Dispatcher
 Handles incoming queries for Data Handlers that have failed local cache hits
 Handles detection requests from both Data Handlers and DINs
 Handles incoming results from Deep Inspection Nuggets
  Handles database updates based on DIN data
 Writes out verbose logging based on DIN data
 Provides alerting to Data Handlers</li></li></ul><li>Framework Goals<br />Provide entry to the system for any arbitrary d...
The Design<br />An implementation of the NRT goals on a Snort platform<br />Target:  Malicious pdf files <br />
More info about the nuggets<br />Let’s pretend that the PDF nugget already has the data…<br />
The Dispatcher<br />Why are we passing back files?<br />
We Like Data<br />MD5 is stored for files and subcomponents both bad and good<br />Primarily this is used to avoid reproce...
Why Taint known good?<br />After an update to detection, previously analyzed files may be found to be bad<br />We don’t re...
We Like to Provide Data<br />When a subcomponent alerts, it is stored for logging in its fully normalized state.<br />If a...
We Really Like to Provide Data<br />Verbose data back to Data Handler should also be as verbose as possible<br />In this c...
Nugget Brain Storm…<br />Seriously, what would you do with a pointer and a size?<br />
The basic idea<br />Create file format templates which parse our elements and provide you a datastructure<br />Provide a f...
Demo One<br />JBIG, ASCII Hex Decoding & Inflation<br />
What just happened?<br />04/21-11:17:58.1271873878 [**] [300:3221225473:1] URL:/wrl/first.pdf Hostname:wrl Alert Info:Prob...
Upcoming SlideShare
Loading in …5
×

A New Framework for Detection

639 views
559 views

Published on

Project Razorback(tm) is an undertaking by the Sourcefire VRT. This is the initial presentation of the new framework for detection.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
639
On SlideShare
0
From Embeds
0
Number of Embeds
47
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

A New Framework for Detection

  1. 1. A New Detection Framework<br />What would you do with a pointer and a size?<br />
  2. 2. Define the problem<br />Why do we need a new detection framework?<br />
  3. 3. The Challenge is Different<br />Attacks have switched from server attacks to client attacks<br />Common attack vectors are easily obfuscated<br />JavaScript<br />Compression<br />File formats are made by insane people<br />Looking at you Flash and OLE guy…<br />Back-channel systems are increasingly difficult to detect<br />
  4. 4. Network Systems…<br />Inline systems must emulate the processing of 1000s of desktops<br />Detection of many backchannels is most successful with statistical evaluation of network traffic<br />
  5. 5. Coverage Gap<br />Broadly speaking, IDS systems deal with packet-by-packet inspection with some level of reassembly<br />Broadly speaking, AV systems typically target indicators of known bad files or system states<br />“…the argument put forward that there's something wrong with anti-virus products that don't detect metasploit output is fallacious on 2 counts: 1) the output isn't necessarily malware (usually only greyware), and 2) anti-virus products are not the proper defense against known exploits (patching is).”<br /> -- Kurt Wismer<br />
  6. 6. Fill the Gap<br />A system is needed that can handle varied detection needs<br />A system is needed that extensible, open and scalable<br />A system is needed that facilitates incident response, not just triggers it<br />So……<br />
  7. 7. NRT Framework<br />Near-Realtime Detection Framework or:<br />“Anything is Possible”<br />
  8. 8. The Dispatcher<br />The heart of the NRT system<br />APIs to handle:<br />Deep Inspection Nugget registration<br />Data Handler registration<br />Detection requests<br />Alerting<br />Full analysis logging<br />Output to API compliant systems<br />Database driven<br />
  9. 9. The Dispatcher<br /><ul><li> Implements a database to provide a centralized set of file information and
  10. 10. Handles incoming queries for Data Handlers that have failed local cache hits
  11. 11. Handles detection requests from both Data Handlers and DINs
  12. 12. Handles incoming results from Deep Inspection Nuggets
  13. 13. Handles database updates based on DIN data
  14. 14. Writes out verbose logging based on DIN data
  15. 15. Provides alerting to Data Handlers</li></li></ul><li>Data Handler<br />Capture data and metadata<br />Contact dispatcher for handling<br />Has this file been evaluated before?<br />Where should I send it?<br />Pass that data set to a Deep Inspection Nugget<br />Accept feedback from the Dispatcher for detection request<br />Asynchronous alerting<br />Local cache of detection outcome<br />
  16. 16. The Data Handler<br /><ul><li> Data (in this case a file) is captured
  17. 17. Metadata is captured (in this case URL and filename)
  18. 18. A local cache of MD5 sums and URLs of files previously collected
  19. 19. A library to handle managing the initial file evaluation, cache checks and communication with the Dispatcher</li></li></ul><li>Deep Inspection Nugget (DIN)<br />Must handle data transfer from Data Handlers<br />Must communicate with Dispatcher<br />Register detection capability<br />Request for additional processing of subcomponents<br />Provide alerting feedback to Dispatcher <br />
  20. 20. The Deep Inspection Nugget<br /><ul><li> Registers with the Dispatcher
  21. 21. Processes data provided by the Data Handlers, as instructed by the Dispatcher
  22. 22. Handles incoming queries for Data Handlers that have failed local cache hits
  23. 23. Handles detection requests from both Data Handlers and DINs
  24. 24. Handles incoming results from Deep Inspection Nuggets
  25. 25. Handles database updates based on DIN data
  26. 26. Writes out verbose logging based on DIN data
  27. 27. Provides alerting to Data Handlers</li></li></ul><li>Framework Goals<br />Provide entry to the system for any arbitrary data type<br />Determine and manage detection based on a registered DIN<br />Provide alerting to any framework capable system<br />Provide verbose, detailed logging on the findings of the Nugget Farm<br />Make intelligent use of all data discovered during the evaluation process<br />
  28. 28. The Design<br />An implementation of the NRT goals on a Snort platform<br />Target: Malicious pdf files <br />
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41. More info about the nuggets<br />Let’s pretend that the PDF nugget already has the data…<br />
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61. The Dispatcher<br />Why are we passing back files?<br />
  62. 62. We Like Data<br />MD5 is stored for files and subcomponents both bad and good<br />Primarily this is used to avoid reprocessing files we’ve already looked at<br />But after a update to any DIN, all known-good entries are “tainted”<br />
  63. 63. Why Taint known good?<br />After an update to detection, previously analyzed files may be found to be bad<br />We don’t rescan all files<br />But if we see a match for md5 to a previous file, we will alert retroactively<br />
  64. 64. We Like to Provide Data<br />When a subcomponent alerts, it is stored for logging in its fully normalized state.<br />If a file is bad, when the DIN completes detection it passes the file to the Dispatcher<br />Response teams have the entire file as well as each portion that alerted in an easily analyzed format<br />
  65. 65. We Really Like to Provide Data<br />Verbose data back to Data Handler should also be as verbose as possible<br />In this case we place data into the payload and provide a custom message to Snort so we can use established methods of handling Snort alerts<br />04/16-16:38:48.1271450328 [**] [300:3221225473:1]URL:/users/pusscat/jbig2.pdf Hostname:metasploit.com AlertInfo:Probable exploit of CVE-2009-0658 (JBIG2) detected in object 8,declared as /Length 33/Filter [/FlateDecode/ASCIIHexDecode/JBIG2Decode]  [**] {TCP} 64.214.53.2:0 -> 216.75.1.230:004/16-16:38:48.12714503280:0:0:0:0:0 -> 0:0:0:0:0:0 type:0x800 len:0x064.214.53.2:0 -> 216.75.1.230:0 TCP TTL:240 TOS:0x10 ID:0 IpLen:20 DgmLen:1280***AP*** Seq: 0x0  Ack: 0x0  Win: 0x0  TcpLen: 2055 52 4C 3A 2F 75 73 65 72 73 2F 70 75 73 73 63  URL:/users/pussc61 74 2F 6A 62 69 67 32 2E 70 64 66 20 48 6F 73  at/jbig2.pdfHos74 6E 61 6D 65 3A 6D 65 74 61 73 70 6C 6F 69 74  tname:metasploit2E 63 6F 6D 20 41 6C 65 72 74 20 49 6E 66 6F 3A  .com Alert Info:50 72 6F 62 61 62 6C 65 20 65 78 70 6C 6F 69 74  Probable exploit20 6F 66 20 43 56 45 2D 32 30 30 39 2D 30 36 35   of CVE-2009-06538 20 28 4A 42 49 47 32 29 20 64 65 74 65 63 74  8 (JBIG2) detect65 64 20 69 6E 20 6F 62 6A 65 63 74 20 38 2C 20  ed in object 8,64 65 63 6C 61 72 65 64 20 61 73 20 2F 4C 65 6E  declared as /Len67 74 68 20 33 33 2F 46 69 6C 74 65 72 20 5B 2F  gth 33/Filter [/46 6C 61 74 65 44 65 63 6F 64 65 2F 41 53 43 49  FlateDecode/ASCI49 48 65 78 44 65 63 6F 64 65 2F 4A 42 49 47 32  IHexDecode/JBIG244 65 63 6F 64 65 20 5D 20                       Decode ]<br />
  66. 66. Nugget Brain Storm…<br />Seriously, what would you do with a pointer and a size?<br />
  67. 67. The basic idea<br />Create file format templates which parse our elements and provide you a datastructure<br />Provide a full, common, scripting language interface to create rules (Ruby? Python? Both?)<br />Only do the heavy work (templating) once per file format.<br />
  68. 68. Demo One<br />JBIG, ASCII Hex Decoding & Inflation<br />
  69. 69. What just happened?<br />04/21-11:17:58.1271873878 [**] [300:3221225473:1] URL:/wrl/first.pdf Hostname:wrl Alert Info:Probable exploit of CVE-2009-0658 (JBIG2) detected in object 8, declared as /Length 29/Filter [/FlateDecode/ASCIIHexDecode/JBIG2Decode ] [**] {TCP} 192.168.0.1:0 -> 204.15.227.178:0<br />04/21-11:17:58.12718738780:0:0:0:0:0 -> 0:0:0:0:0:0 type:0x800 len:0x0<br />192.168.0.1:0 -> 204.15.227.178:0 TCP TTL:240 TOS:0x10 ID:0 IpLen:20 DgmLen:1280<br />***AP*** Seq: 0x0 Ack: 0x0 Win: 0x0 TcpLen: 20<br />55 52 4C 3A 2F 77 72 6C 2F 66 69 72 73 74 2E 70 URL:/wrl/first.p<br />64 66 20 48 6F 73 74 6E 61 6D 65 3A 77 72 6C 20 dfHostname:wrl<br />41 6C 65 72 74 20 49 6E 66 6F 3A 50 72 6F 62 61 Alert Info:Proba<br />62 6C 65 20 65 78 70 6C 6F 69 74 20 6F 66 20 43 ble exploit of C<br />56 45 2D 32 30 30 39 2D 30 36 35 38 20 28 4A 42 VE-2009-0658 (JB<br />49 47 32 29 20 64 65 74 65 63 74 65 64 20 69 6E IG2) detected in<br />20 6F 62 6A 65 63 74 20 38 2C 20 64 65 63 6C 61 object 8, decla<br />72 65 64 20 61 73 20 2F 4C 65 6E 67 74 68 20 32 red as /Length 2<br />39 2F 46 69 6C 74 65 72 20 5B 2F 46 6C 61 74 65 9/Filter [/Flate<br />44 65 63 6F 64 65 2F 41 53 43 49 49 48 65 78 44 Decode/ASCIIHexD<br />65 63 6F 64 65 2F 4A 42 49 47 32 44 65 63 6F 64 ecode/JBIG2Decod<br />65 20 5D 20 e ] <br />=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+<br />
  70. 70. Demo Two<br />What is that JavaScript up to?<br />
  71. 71. What just happened?<br />[**] [300:2147483653:1] URL:/wrl/first.pdf Hostname:wrl Alert Info:The JavaScript variables in object 6, declared as /Length 5994/Filter [/FlateDecode/ASCIIHexDecode ] , show a high degree of entropy [**]<br />You tell me, does this string of variable names look weird to you? <br /> EvctenMNtrWDQVBKGrwGxrxKfMiZoYziRxAFEfjMdXRzjGNqVZYEAqogviSvzHpGpCkihcVtXRWcHphvhAnPOXnrxmTXJEUIkcYzelWZUCuIyKArtJvcEQXzUjHEzuSjGEJugOyFQnaSplNWwQsqOoV<br />[**] [300:2147483649:1] URL:/wrl/first.pdf Hostname:wrl Alert Info:Found in the Javascript block, while searching object 6: unescape [**]<br />Wait, did someone say unescape…<br />
  72. 72. Get a little crazy?<br />Sig up some common GetEIPtechiniques…<br />Heuristically hunt down shellcode decoder stubs<br />Decode and parse shellcode<br />Give back some REAL data.<br />
  73. 73. Demo Three<br />What is that unescape up to….<br />
  74. 74. Wait, did that really just happen?<br />[**] [300:3221225482:1] URL:/wrl/first.pdf Hostname:wrl Alert Info:Reverse TCP connectbackshellcode detected. Connecting to 10.4.4.10 on port 4444 [**]<br />Looking at the following:<br />10 d4 77 74 71 20 f6 d3 e0 70 66 0c 7a 40 73 72 <br />78 2f be 37 04 91 a8 46 93 41 1c 24 b0 b4 b1 3d <br />43 b5 96 15 7d 4e 9b 7e 48 42 8d 12 f7 eb 4f 0d <br />7b 4a 25 08 d5 1d 0b ff c6 c0 e3 03 f5 b3 b2 34 <br />71 18 fdba 75 77 25 3c b8 7b 30 d4 43 78 1c 2a <br />.<br />.<br />.<br />bf 98 35 a5 af 98 1d 1f e0 17 95 0a 3a 5f 1f f0 <br />87 c2 71 f1 e5 a0 77 f5 fe 94 fc 13 85 d8 23 a2 <br />87 51 d0 81 8e 37 a0 70 2f bc 79 0a a1 c0 00 19 <br />87 38 c0 57 b9 37 a0 9f ef a2 71 a3 b8 a0 77 2c <br />27 97 8a 20 64 fe 1f b5 87 c8 65 f5 ef 9e 1f f5 <br />87 90 d1 a6 0a 37 a0 66 bc a2 75 a3 bc 9f 1d f7 <br />36 00 2a 0a 3a c9 b6 dc 29 4d 83 80 03 0b 75 f5 <br />Gave us the shellcode type as well as the IP and port combination the connect back goes to.<br />Wouldn’t it be great if something knew to start listening?<br />
  75. 75. Go nuts.<br />Take that IP address and Port, and auto-tcpdump when you get an alert<br />Watch everything the attacker does over that back channel on the fly<br />Poor-man’s netwitness. (Can I say that?)<br />
  76. 76. Seriously. Lick every window in Dubai.<br />How about a custom post-mortem debugger on every enterprise desktop?<br />Have it alert to your central dispatcher and dump whatever loaded file is the crash culprit.<br />Get both failed exploit attempts and possibly a few free 0-day to sell on the side!<br />
  77. 77. Start now. Dubai has a lot of windows.<br />Make use of BinCrowd! <br />Yank down the a whole community’s set of symbols for that questionable sample you just got a hold of – malware reuses code too!<br />Not all of your machines have hardware DEP?<br />Run one machine with DEP, use that custom post mortem, still get near real time knowledge of attacks<br />DLP is serious business<br />Store more than one checksum type for sensitive data. Custom nuggets can make it easy.<br />
  78. 78. We Brought Presents<br />Circus Tickets!<br />
  79. 79. What can you do with a u_char * and an u_int_32?<br />We have hosted on http://labs.snort.org a package that contains:<br />Snort Preprocessor for snagging .exe, .dll and .pdf files from live traffic<br />A commented library that will allow you to thread calls to a detection function<br />A “Dumb Nugget” to simply write these files to disk<br />A “Clam Nugget” to pass these files to ClamAV<br />Local cache system to reduce detection overhead<br />Alerting system that fires Snort alerts with arbitrary data<br />Disclaimer<br />For serious, this code was put together to pitch the idea to management it is…well it is what it is<br />This project is a research project in the VRT no timeline for release either as open source or a Sourcefire product has been determined<br />We’ll update it as we integrate the full dispatcher->data handler->deep inspection nugget code<br />
  80. 80. Project Team<br />System Architects:<br />Matthew Olney<br />Lurene Grenier<br />Patrick Mullen<br />Nigel Houghton<br />Programmers:<br />Ryan Pentney (OMG CODE OUTPUT)<br />Alain Zidouemba (ClamAV integration)<br />Database:<br />Alex Kambis<br />File Format Research<br />Monica Sojeong Hong<br />Alex Kirk<br />Infrastructure Support<br />Kevin “McLovin” Miklavcic<br />Christopher McBee<br />Head Didn’t Fire Us During POC phase<br />Matthew Watchinski, Sr. Director, Vulnerability Research<br />
  81. 81. VRT<br />Blog: <br />http://vrt-sourcefire.blogspot.com/<br />Place we store bad ideas:<br />http://labs.snort.org/<br />Twitter:<br />@vrt_sourcefire (VRT Twitter Account)<br />@kpyke (Matthew Olney)<br />@pusscat (Lurene Grenier)<br />@xram_lrak (Matthew Watchinski)<br />
  82. 82. Fin<br />

×