Behind The Scenes
 Security Research



    Rodrigo Rubira Branco (BSDaemon)
   Director, Vulnerability & Malware Research Labs
                     rbranco *noSPAM* qualys.com
                         http://twitter.com/bsdaemon
Agenda

  Objectives

  Introduction

  Security research as a worldwide wype
   – Where are the real hackers?
   – What is the difference between the public exploits and the private ones?


  Security conferences around the globe

  The problems in the security industry

  Future




                       Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   2
Objectives


  Discuss about security problems and security industry
  problems

  Demonstrate how vulnerability finding works (or is
  supposed to work) – focusing on building the team

  Explain the challenges the exploit writer faces nowadays

  Be fun?




                 Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   3
Security nowadays


   Buggy programs deployed on critical servers

   Rapidly-evolving threats, attackers and tools
   (exploitation frameworks)

   Lack of developers training, resources and people to fix
   problems and create safe code

   That’s why we are here today, right?




                 Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   4
Security nowadays – 0day challenge

      First host                                                            All vulnerable hosts
      attacked                                                                     attacked




                         Reaction time
                      Slammer: 10 mins
            Future worms: < 1 minute [Staniford et. al. 2002]


                            “0day Statistics
                            Average 0day lifetime:
                            348 days
                            Shortest life:
                            99 days
                            Longest life:
                            1080 (3 years)"

                            - Justine Aitel
                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   5
!exploitable


  _declspec(naked) int main() {
       _asm {
                mov eax, 0x41414141
                call eax
       }
   }




                    Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   6
!exploitable


  This is incorrectly classified as EXPLOITABLE because
  the tool always assume that the attacker has control over
  all the input operands

  So, what is the point? The point is that security research
  is a long-way run, shortcuts will only give you tainted
  information (hidden joke)




                 Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   7
State Transition for Memory Corruption


    Case 1
    (green):
    Format String
    Case 2 and 3
    (red and
    blue): buffer
    overflow
    Case 4
    (purple):
    unpredictable

Source:
Automatic Diagnosis and                                     c: corrupting instruction
Response to Memory                                          t: takeover instruction
Corruption Vulnerabilities                                  f: faulting instruction
                             Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   8
Source:
                   http://www.h2hc.com.br/repositorio/2008/Nico.pdf




Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   9
The Extinction of Hackers - FX


  FX wrote the article long ago (well, I only have a copy of
  the file so didn’t manage to get the right timing and just
  forgot to ask him before coming)

  He seems to be actually right:
  – Kids use the computers for games
  – Things are a lot easier to do nowadays (in means that you don’t
    really need to know how to use the computer to actually do
    something useful with it)
  – Things are a lot more complex nowadays (yeah, in the opposite
    direction of the other statement) -> Exploiting software is
    becoming a complex task (no easy 0x41414141 anymore)



                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   10
Vulnerability Coordination?


  Basically is a mess (but it is getting better)

  Lots of conflicts (and lots of conflicts of interest)

  Difficult to really scientificaly measure the impacts of any
  decision:
  – Full disclosure (everybody is aware, instead of only the person
    who is already exploiting it)




                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   11
Vulnerability Coordination

  Case 1: Security issue affecting 3 vendors. What one of
  them replies?
  – “We don’t know how to fix the code, do you mind in ask to the
    other vendor to SHARE the patch?”


  Case 2: Security issue in an ‘encryption’ mechanism of
  a security software. What the vendor replies?
  – “Ok, we going to change the documentation to say it is not
    encryption”


  Case 3: Security issue in a web software. What the
  vendor says about it?
  – “There is no security issue. Millions of people use the software
    and none reported the problem before”


                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   12
Who buys?


 Because everybody is buying vulnerabilities, it is
 becoming difficult to see vulnerability discussions on
 public

 Reliable/good exploits are not available for free anymore
 (at large)

 Governments, private sector, brokers, criminals
  – They all want our 0days
  – Legislation helps some countries
  – This is destroying the new generations



                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   13
Security Myth #1: The ‘EXPERT’


  The Market for Lemons: Quality Uncertainty and the
  Market Mechanism – George Akerlof

  Asymetric knowledge
  – Complex subject
  – Industry defines its own standards
  – Politics defines ‘auditing’ (SOX)




                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   14
Security Myth #2: There is no 0day risk


  Crime and Punishment: An Economic Approach -
  Gary S. Becker

  There is no 100% security
  – Since there is no guarantees, why should vendor cares?
  – Security highly makes use of the ‘ass factor’:
      » “Why should I buy from vendor X?”
      » “Because everybody is doing so”




                    Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   15
Security Myth #3: Computer power is growing

  Moore’s law does not help the security industry

  Forcing the perimeter to inspect everything expecting to
  benefit from the growing capabilities of hardware seems
  to be non-sense

  Simple like that: As the computers have more power,
  engineers start to use it, to save bandwitdth and provide
  security:
  – Compression
  – Encryption


  In the end, the gateway has more work to do

                  Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   16
Security Myth #4: IPS
Many people seems to highly believe the IPS is a bunch of parsers
running in a central location:
– Most likely it will be more pattern-match-like based technology
– This is true for security filters as well (such as the XSS protections in
  IE)
– Really parsing the complex file formats on gateways is infeasible,
  parcial parsing is used


Everybody believes IPS is a bridge
– It does interact with packets (for inspection)
– It can be exploited as any other piece of software (not common)


Virtual Patching
– Trust in it only when a vulnerability is public, but there is no patch (in
  this case, you have no options)
– False positives ARE very important: Check them before buying
                       Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   17
Security Myth #5: Rogue <you name it>


  Amazing question by Michael Golub: “What are the
  features an AV has, that a Rogue AV doesn’t?”
  – Both does not guarantee anything
  – Both have ‘upgrade to premium’ options
  – Both will have a nice GUI (rogue AV usually is nicer)
  – Both will affect the performance of your computer (rogue AV
    usually is faster)
  – Both will have false alarms (false positives) -> Well, I never
    heard that millions of users were affected by the rogue AV
    quarantining a fundamental dll of Windows and thus letting the
    computer unusable




                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   18
So, why build a research team?


  Companies highly benefit from security research:
  – Better understanding
  – Real-life awareness
  – Solving complex problems with lower budgets


  Having security researchers are not only for security
  vendors:
  – As presented in Hackito: A researcher is capable of solving
    complex tasks, such as analyzing huge amounts of logs
  – A researcher can provide real understanding of the threats in an
    organization, ‘translate’ the marketing materials (bullshit) to the
    real-world benefit it will provide



                    Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   19
The experts


  When choosing a researcher, careful to select him
  – Remember the ‘Market for Lemons’ lessons
  – Lots of conferences around the world: Be a speaker in many of
    them means nothing
  – Lots of security issues been released: Be one of the guys who
    found them means nothing
      » Many crappy security issues been released (it is part of the priorities
        list of vendors to have as much as possible released issues)
  – Ask around, go to the conferences and see it yourself, be
    involved if you care




                     Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   20
What to expect

  It is important to define targets
  – If you don’t know what you want, don’t expect much:
    Researchers are lazy creatures (yeah, we are!)


  Don’t define how do you want, only your expectations
  – If you really knew what you want, you’ve hired developers, not
    researchers
  – I always like to tell people: “Ok, now you are giving me the
    solution. I better know the problem first”


  Security research is not only exploitation
  – Defensive capabilities
  – Complex problems
  – Malware Analysis

                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   21
Targets/Deliverables


  Some companies count exploits/person/year
  – Highly complex way to track work
  – Will work if you have very capable manager (able to differentiate
    complex from simple exploits)
  – Usually done when the company is a research company or
    somehow has exploit writers and security researchers in different
    teams


  Small groups together
  – Very effective way to work and keep track of work
  – Changing the groups and see the results – Easy to spot laziness




                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   22
Research capabilities


  Very often, even companies that does not have research
  area have research capabilities
  – They are hidden inside development teams
  – Usually embedded in the job description of senior workers in
    such teams


  Create a communication link
  – Since the research is hidden inside teams, sometimes it does
    not achieve its maximal capability
  – Gathering information inside the company will reveal much more
    knowledge then initially expected




                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   23
Management

 Many processes or No processes
  – Somehow in the between
  – Important to define areas where the research has freedom to act
     » For example, to speed up the communication process with vendors
  – No paperwork means higher productivity and less boredom
     » Still, some way to track work will be required by the company


 Your team gotta travel
  – Yes, they will want to go to H2HC in Brazil
  – They will love to spend the weekend with friends in Sao Jose
    dos Campos for Vale Security Conf (at least I do ☺ )
  – They will be at Hackito in Paris as well
  – And there is NOTHING you can do about that
     » Better define good-to-go conferences in advance
     » Give real time for real research (yeah, REAL)

                    Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   24
Fuzzing, tools and others


  If your research team is going to find vulnerabilities you:
  – Want them to have real hardware resources (fuzzers are heavy
    animals)
  – Want them to have real understanding of what they are testing
    (and this is VERY time consuming)
  – Want them to have good tools for the process (expect to buy
    tools such as IDA, Zynamics Binavi, Canvas)


  Please, please, please:
  – No dumb fuzzing anymore (I’m tired of crashes)
  – Add analysis capabilities to the game
  – Let them expend time writing tools



                   Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   25
Future


I can’t foresee the future!

Hope more researchers will start to complain about the industry

Strongly believe the real experts at some point will receive the
real attention, since the problem is constanly growing and not
been fixed: Something wrong is going on...




                    Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   26
Special Thanks


  Michael Golub
  – A good friend
  – The main point of reference/discussions for many/most of the
    presented ideas


  Vale Security Conference Staff
  – Great initiative, congrats and tks!
  – For trusting me for discuss this subject here




                    Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo   27
End! Really !?


                  Rodrigo Rubira Branco (BSDaemon)
                 Director, Vulnerability & Malware Research Labs
                                   rbranco *noSPAM* qualys.com
                                       http://twitter.com/bsdaemon

Vale Security Conference - 2011 - 17 - Rodrigo Rubira Branco (BSDaemon)

  • 1.
    Behind The Scenes Security Research Rodrigo Rubira Branco (BSDaemon) Director, Vulnerability & Malware Research Labs rbranco *noSPAM* qualys.com http://twitter.com/bsdaemon
  • 2.
    Agenda Objectives Introduction Security research as a worldwide wype – Where are the real hackers? – What is the difference between the public exploits and the private ones? Security conferences around the globe The problems in the security industry Future Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 2
  • 3.
    Objectives Discussabout security problems and security industry problems Demonstrate how vulnerability finding works (or is supposed to work) – focusing on building the team Explain the challenges the exploit writer faces nowadays Be fun? Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 3
  • 4.
    Security nowadays Buggy programs deployed on critical servers Rapidly-evolving threats, attackers and tools (exploitation frameworks) Lack of developers training, resources and people to fix problems and create safe code That’s why we are here today, right? Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 4
  • 5.
    Security nowadays –0day challenge First host All vulnerable hosts attacked attacked Reaction time Slammer: 10 mins Future worms: < 1 minute [Staniford et. al. 2002] “0day Statistics Average 0day lifetime: 348 days Shortest life: 99 days Longest life: 1080 (3 years)" - Justine Aitel Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 5
  • 6.
    !exploitable _declspec(naked)int main() { _asm { mov eax, 0x41414141 call eax } } Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 6
  • 7.
    !exploitable Thisis incorrectly classified as EXPLOITABLE because the tool always assume that the attacker has control over all the input operands So, what is the point? The point is that security research is a long-way run, shortcuts will only give you tainted information (hidden joke) Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 7
  • 8.
    State Transition forMemory Corruption Case 1 (green): Format String Case 2 and 3 (red and blue): buffer overflow Case 4 (purple): unpredictable Source: Automatic Diagnosis and c: corrupting instruction Response to Memory t: takeover instruction Corruption Vulnerabilities f: faulting instruction Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 8
  • 9.
    Source: http://www.h2hc.com.br/repositorio/2008/Nico.pdf Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 9
  • 10.
    The Extinction ofHackers - FX FX wrote the article long ago (well, I only have a copy of the file so didn’t manage to get the right timing and just forgot to ask him before coming) He seems to be actually right: – Kids use the computers for games – Things are a lot easier to do nowadays (in means that you don’t really need to know how to use the computer to actually do something useful with it) – Things are a lot more complex nowadays (yeah, in the opposite direction of the other statement) -> Exploiting software is becoming a complex task (no easy 0x41414141 anymore) Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 10
  • 11.
    Vulnerability Coordination? Basically is a mess (but it is getting better) Lots of conflicts (and lots of conflicts of interest) Difficult to really scientificaly measure the impacts of any decision: – Full disclosure (everybody is aware, instead of only the person who is already exploiting it) Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 11
  • 12.
    Vulnerability Coordination Case 1: Security issue affecting 3 vendors. What one of them replies? – “We don’t know how to fix the code, do you mind in ask to the other vendor to SHARE the patch?” Case 2: Security issue in an ‘encryption’ mechanism of a security software. What the vendor replies? – “Ok, we going to change the documentation to say it is not encryption” Case 3: Security issue in a web software. What the vendor says about it? – “There is no security issue. Millions of people use the software and none reported the problem before” Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 12
  • 13.
    Who buys? Becauseeverybody is buying vulnerabilities, it is becoming difficult to see vulnerability discussions on public Reliable/good exploits are not available for free anymore (at large) Governments, private sector, brokers, criminals – They all want our 0days – Legislation helps some countries – This is destroying the new generations Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 13
  • 14.
    Security Myth #1:The ‘EXPERT’ The Market for Lemons: Quality Uncertainty and the Market Mechanism – George Akerlof Asymetric knowledge – Complex subject – Industry defines its own standards – Politics defines ‘auditing’ (SOX) Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 14
  • 15.
    Security Myth #2:There is no 0day risk Crime and Punishment: An Economic Approach - Gary S. Becker There is no 100% security – Since there is no guarantees, why should vendor cares? – Security highly makes use of the ‘ass factor’: » “Why should I buy from vendor X?” » “Because everybody is doing so” Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 15
  • 16.
    Security Myth #3:Computer power is growing Moore’s law does not help the security industry Forcing the perimeter to inspect everything expecting to benefit from the growing capabilities of hardware seems to be non-sense Simple like that: As the computers have more power, engineers start to use it, to save bandwitdth and provide security: – Compression – Encryption In the end, the gateway has more work to do Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 16
  • 17.
    Security Myth #4:IPS Many people seems to highly believe the IPS is a bunch of parsers running in a central location: – Most likely it will be more pattern-match-like based technology – This is true for security filters as well (such as the XSS protections in IE) – Really parsing the complex file formats on gateways is infeasible, parcial parsing is used Everybody believes IPS is a bridge – It does interact with packets (for inspection) – It can be exploited as any other piece of software (not common) Virtual Patching – Trust in it only when a vulnerability is public, but there is no patch (in this case, you have no options) – False positives ARE very important: Check them before buying Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 17
  • 18.
    Security Myth #5:Rogue <you name it> Amazing question by Michael Golub: “What are the features an AV has, that a Rogue AV doesn’t?” – Both does not guarantee anything – Both have ‘upgrade to premium’ options – Both will have a nice GUI (rogue AV usually is nicer) – Both will affect the performance of your computer (rogue AV usually is faster) – Both will have false alarms (false positives) -> Well, I never heard that millions of users were affected by the rogue AV quarantining a fundamental dll of Windows and thus letting the computer unusable Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 18
  • 19.
    So, why builda research team? Companies highly benefit from security research: – Better understanding – Real-life awareness – Solving complex problems with lower budgets Having security researchers are not only for security vendors: – As presented in Hackito: A researcher is capable of solving complex tasks, such as analyzing huge amounts of logs – A researcher can provide real understanding of the threats in an organization, ‘translate’ the marketing materials (bullshit) to the real-world benefit it will provide Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 19
  • 20.
    The experts When choosing a researcher, careful to select him – Remember the ‘Market for Lemons’ lessons – Lots of conferences around the world: Be a speaker in many of them means nothing – Lots of security issues been released: Be one of the guys who found them means nothing » Many crappy security issues been released (it is part of the priorities list of vendors to have as much as possible released issues) – Ask around, go to the conferences and see it yourself, be involved if you care Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 20
  • 21.
    What to expect It is important to define targets – If you don’t know what you want, don’t expect much: Researchers are lazy creatures (yeah, we are!) Don’t define how do you want, only your expectations – If you really knew what you want, you’ve hired developers, not researchers – I always like to tell people: “Ok, now you are giving me the solution. I better know the problem first” Security research is not only exploitation – Defensive capabilities – Complex problems – Malware Analysis Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 21
  • 22.
    Targets/Deliverables Somecompanies count exploits/person/year – Highly complex way to track work – Will work if you have very capable manager (able to differentiate complex from simple exploits) – Usually done when the company is a research company or somehow has exploit writers and security researchers in different teams Small groups together – Very effective way to work and keep track of work – Changing the groups and see the results – Easy to spot laziness Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 22
  • 23.
    Research capabilities Very often, even companies that does not have research area have research capabilities – They are hidden inside development teams – Usually embedded in the job description of senior workers in such teams Create a communication link – Since the research is hidden inside teams, sometimes it does not achieve its maximal capability – Gathering information inside the company will reveal much more knowledge then initially expected Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 23
  • 24.
    Management Many processesor No processes – Somehow in the between – Important to define areas where the research has freedom to act » For example, to speed up the communication process with vendors – No paperwork means higher productivity and less boredom » Still, some way to track work will be required by the company Your team gotta travel – Yes, they will want to go to H2HC in Brazil – They will love to spend the weekend with friends in Sao Jose dos Campos for Vale Security Conf (at least I do ☺ ) – They will be at Hackito in Paris as well – And there is NOTHING you can do about that » Better define good-to-go conferences in advance » Give real time for real research (yeah, REAL) Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 24
  • 25.
    Fuzzing, tools andothers If your research team is going to find vulnerabilities you: – Want them to have real hardware resources (fuzzers are heavy animals) – Want them to have real understanding of what they are testing (and this is VERY time consuming) – Want them to have good tools for the process (expect to buy tools such as IDA, Zynamics Binavi, Canvas) Please, please, please: – No dumb fuzzing anymore (I’m tired of crashes) – Add analysis capabilities to the game – Let them expend time writing tools Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 25
  • 26.
    Future I can’t foreseethe future! Hope more researchers will start to complain about the industry Strongly believe the real experts at some point will receive the real attention, since the problem is constanly growing and not been fixed: Something wrong is going on... Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 26
  • 27.
    Special Thanks Michael Golub – A good friend – The main point of reference/discussions for many/most of the presented ideas Vale Security Conference Staff – Great initiative, congrats and tks! – For trusting me for discuss this subject here Kernel Hacking: If you really know, you can hack! – http://www.kernelhacking.com/rodrigo 27
  • 28.
    End! Really !? Rodrigo Rubira Branco (BSDaemon) Director, Vulnerability & Malware Research Labs rbranco *noSPAM* qualys.com http://twitter.com/bsdaemon