Enterprise Open Source
Intelligence Gathering
Tom Eston
Open source intelligence (OSINT) is a form
of intelligence collection management...
Open source intelligence (OSINT) is a form
of intelligence collection management...


...involves finding, selecting, and acquiring information
from publicly available sources and analyzing it
                     to produce actionable intelligence.

                                            - wikipedia
What do the Internets say?
18% had a data
loss event via blog
       or message
           board...



    - Proofpoint, Inc. 2009 Survey
18% had a data
loss event via blog
       or message
           board...


  11% in 2008
    - Proofpoint, Inc. 2009 Survey
17%
experienced
data loss
related to social
networks...

- Proofpoint, Inc. 2009 Survey
17%
experienced
data loss
related to social
networks...
12 % in 2008
- Proofpoint, Inc. 2009 Survey
“A brand is the
personification of a product,
service, or even entire company.”
        - Robert Blanchard, former P&G executive
5 things you will learn
• What is out there on your company?
• Metadata
• Removal of Internet postings, metadata
• Setting up a simple (cheap) monitoring
  program
• Building a Internet Posting Policy
What gets posted?

• Customer and Employee Complaints
• Exposure of Confidential Information
• Security Vulnerabilities
Customer Complaints
Employee Complaints
FAIL
Exposure of Confidential
     Information
What about
Vulnerabilities?
Things you wouldn't
     expect...
Where does this
information get posted?
       ...and how to find it!
Social Networks
300 Million Users

110 Million Users

40 Million Users

Grew 752% in 2008
Finding Information on
   Social Networks
• Socnet Search Engines
• Maltego (Twitter/Facebook)
• RSS feeds/Google Hacks
 • Google Alerts + Google Reader = WIN
• Manual Searching
 • Facebook status updates
Socnet Search Engines
•   Wink, Spock, Twoogle, Knowem, WhosTalkin
    (there are many more, see my blog post)
•   Twitter Search
•   Social Bookmark Sites
    •   Delicious, StumbleUpon
•   Don’t forget about photos/video!
    •   Flickr Photo Search
    •   YouTube and Vimeo Video Search
Maltego + Mesh = WIN




*Screen shot from the “Maltego and Twitter!” post on paterva.com
Searching Facebook
•   Good: Maltego Facebook Transform (violates TOS)
    ** No longer working! :-(
•   Better: Login and use the search! FB doesn’t make status
    updates public...yet.
•   Best:
    site:facebook.com inurl:group (bofa | "bank of america") =
    Groups
    •   inurl:pages = Facebook Pages
    •   allinurl: people "John Doe" site:facebook.com = Public Profiles
    •   Yahoo! Pipe for Facebook Groups:
        Facebook Discussion Board RSS Feed
    •   Create Google Alert(s)
Searching LinkedIn
• Similar to Facebook
• Google dorks
 • site:linkedin.com inurl:pub (bofa | "bank
    of america") = Public Profiles
  • inurl:updates = Profile Updates
  • inurl:companies = Company Profiles
Blogs and News

• Blogpulse, Technoratti, IceRocket
• Social Mention
  (Search Engine for blogs, comments)
• Google/Yahoo News
Document Repositories

• DocStoc
• Scribd
• SlideShare
• PDF Search
  Engine
Message Boards
• Internet Forums (yes, even 4chan)
• Craigslist
• Full Disclosure Mailing List (vulnerabilities)
• Google Groups/Yahoo Groups
All your metadata are
    belong to us...
What is Metadata?

• Metadata = Data that describes Data
• Catalog, index files, documents and more
• Often overlooked by:
 • Document/File Creators
 • Your Company
Why do we care?
• Can expose potential vulnerable software/
  hardware in use! (client side attack)
 • OS and version numbers
• Location information (GPS from
  smartphones)
• User names, naming schemes, file paths
Where do you find it?

• Microsoft Office Documents
• PDF
• JPEG’s (photos)
• Other file types
Metadata is everywhere!
How do you find it?

• Google
• Document Repositories
• Wget to download photos
  (many other tools)
• Your Company Website
Tools to analyze
         Metadata
• EXIFtool (cmd line or GUI)
• Maltego
• Metagoofil
• Metadata Extraction Tool
• FOCA
Real World Example
Removing Internet
Postings and Metadata
Removing posts from
    the Internet
• Hard, but not impossible.
  Search Engine Cache FTL
• Submit request to Search Engines to
  remove (there are multiple)
• Legal team involvement, especially w/
  socnets
Metadata Removal
      Techniques
• MS Office Documents
 • Office 2002/03: CMD Line app “Remove
    Hidden Data” (Offrhd.exe)
 • Office 2007: Document Inspector
• EXIFtool (photos)
 • Can be scripted to auto remove
Metadata Removal
     Continued...

• PDFs: File -> Document Properties
• EXIFtool
• Many third-party tools! ($)
Setting up a monitoring
        program
What do you want to
     monitor?
• Impossible to monitor everything!
• Pick the most popular social networks,
  news sites, blogs, forums...
• Monitoring should be defined with your
  PR/Marketing groups!
Free Tools
• Yahoo! Pipes (mashups)
• RSS Feeds/RSS Reader
  Google Reader FTW
• Maltego (community version)
  Good for defining relationships, not
  automated
 • Maltego for specific searching when you
    need “more details”
Yahoo! Pipes
Google Reader RSS
What works best?
    • Assign someone! (someone in infosec,
      social media skill sets)
    • Create RSS Feeds from identified sites
    • Utilize Yahoo! Pipes, create RSS from pipes
    • Monitor w/Google Reader
    • Sites you can’t monitor
      automatically...determine manual methods.

Build this into your Incident Response Procedures!
Building a Internet
  Posting Policy
Define your Social
    Media Strategy
• Partner with Marketing/Public Relations/HR
• What is acceptable for employees to post?
• At work/off work
 • Employees have mobile devices, home
    computers!
Define what gets
       monitored?
• Difficult or impossible to monitor
  everything
• Determine with your partners what should
  be monitored
• Careful with policy conflicts!
Cisco Example
Intel Example
Communicate to your
    employees!
How can you enforce a policy if employees don’t know
                     about it?
Where to learn more?

• Great paper on Metadata
  (SANS Reading Room):
  “Document Metadata, the Silent
  Killer” - Larry Pesce
• Maltego Tutorials:
  Chris Gates, EthicalHacker.net
• My blog: spylogic.net
OSINT 3 Part Series
• All the details from this presentation!
• Part 1 - Social Networks
  http://bit.ly/osint1
• Part 2 - Blogs, Message Boards, Metadata
  http://bit.ly/osint2
• Part 3 - Monitoring, Social Media Policies
  http://bit.ly/osint3
Enterprise Open Source Intelligence Gathering

Enterprise Open Source Intelligence Gathering

  • 1.
  • 2.
    Open source intelligence(OSINT) is a form of intelligence collection management...
  • 3.
    Open source intelligence(OSINT) is a form of intelligence collection management... ...involves finding, selecting, and acquiring information from publicly available sources and analyzing it to produce actionable intelligence. - wikipedia
  • 4.
    What do theInternets say?
  • 5.
    18% had adata loss event via blog or message board... - Proofpoint, Inc. 2009 Survey
  • 6.
    18% had adata loss event via blog or message board... 11% in 2008 - Proofpoint, Inc. 2009 Survey
  • 7.
    17% experienced data loss related tosocial networks... - Proofpoint, Inc. 2009 Survey
  • 8.
    17% experienced data loss related tosocial networks... 12 % in 2008 - Proofpoint, Inc. 2009 Survey
  • 9.
    “A brand isthe personification of a product, service, or even entire company.” - Robert Blanchard, former P&G executive
  • 10.
    5 things youwill learn • What is out there on your company? • Metadata • Removal of Internet postings, metadata • Setting up a simple (cheap) monitoring program • Building a Internet Posting Policy
  • 11.
    What gets posted? •Customer and Employee Complaints • Exposure of Confidential Information • Security Vulnerabilities
  • 12.
  • 13.
  • 17.
  • 18.
  • 19.
  • 23.
  • 25.
    Where does this informationget posted? ...and how to find it!
  • 26.
  • 27.
    300 Million Users 110Million Users 40 Million Users Grew 752% in 2008
  • 28.
    Finding Information on Social Networks • Socnet Search Engines • Maltego (Twitter/Facebook) • RSS feeds/Google Hacks • Google Alerts + Google Reader = WIN • Manual Searching • Facebook status updates
  • 29.
    Socnet Search Engines • Wink, Spock, Twoogle, Knowem, WhosTalkin (there are many more, see my blog post) • Twitter Search • Social Bookmark Sites • Delicious, StumbleUpon • Don’t forget about photos/video! • Flickr Photo Search • YouTube and Vimeo Video Search
  • 30.
    Maltego + Mesh= WIN *Screen shot from the “Maltego and Twitter!” post on paterva.com
  • 31.
    Searching Facebook • Good: Maltego Facebook Transform (violates TOS) ** No longer working! :-( • Better: Login and use the search! FB doesn’t make status updates public...yet. • Best: site:facebook.com inurl:group (bofa | "bank of america") = Groups • inurl:pages = Facebook Pages • allinurl: people "John Doe" site:facebook.com = Public Profiles • Yahoo! Pipe for Facebook Groups: Facebook Discussion Board RSS Feed • Create Google Alert(s)
  • 32.
    Searching LinkedIn • Similarto Facebook • Google dorks • site:linkedin.com inurl:pub (bofa | "bank of america") = Public Profiles • inurl:updates = Profile Updates • inurl:companies = Company Profiles
  • 33.
    Blogs and News •Blogpulse, Technoratti, IceRocket • Social Mention (Search Engine for blogs, comments) • Google/Yahoo News
  • 34.
    Document Repositories • DocStoc •Scribd • SlideShare • PDF Search Engine
  • 35.
    Message Boards • InternetForums (yes, even 4chan) • Craigslist • Full Disclosure Mailing List (vulnerabilities) • Google Groups/Yahoo Groups
  • 36.
    All your metadataare belong to us...
  • 37.
    What is Metadata? •Metadata = Data that describes Data • Catalog, index files, documents and more • Often overlooked by: • Document/File Creators • Your Company
  • 38.
    Why do wecare? • Can expose potential vulnerable software/ hardware in use! (client side attack) • OS and version numbers • Location information (GPS from smartphones) • User names, naming schemes, file paths
  • 39.
    Where do youfind it? • Microsoft Office Documents • PDF • JPEG’s (photos) • Other file types
  • 40.
  • 41.
    How do youfind it? • Google • Document Repositories • Wget to download photos (many other tools) • Your Company Website
  • 42.
    Tools to analyze Metadata • EXIFtool (cmd line or GUI) • Maltego • Metagoofil • Metadata Extraction Tool • FOCA
  • 43.
  • 46.
  • 47.
    Removing posts from the Internet • Hard, but not impossible. Search Engine Cache FTL • Submit request to Search Engines to remove (there are multiple) • Legal team involvement, especially w/ socnets
  • 48.
    Metadata Removal Techniques • MS Office Documents • Office 2002/03: CMD Line app “Remove Hidden Data” (Offrhd.exe) • Office 2007: Document Inspector • EXIFtool (photos) • Can be scripted to auto remove
  • 49.
    Metadata Removal Continued... • PDFs: File -> Document Properties • EXIFtool • Many third-party tools! ($)
  • 50.
    Setting up amonitoring program
  • 51.
    What do youwant to monitor? • Impossible to monitor everything! • Pick the most popular social networks, news sites, blogs, forums... • Monitoring should be defined with your PR/Marketing groups!
  • 52.
    Free Tools • Yahoo!Pipes (mashups) • RSS Feeds/RSS Reader Google Reader FTW • Maltego (community version) Good for defining relationships, not automated • Maltego for specific searching when you need “more details”
  • 53.
  • 54.
  • 55.
    What works best? • Assign someone! (someone in infosec, social media skill sets) • Create RSS Feeds from identified sites • Utilize Yahoo! Pipes, create RSS from pipes • Monitor w/Google Reader • Sites you can’t monitor automatically...determine manual methods. Build this into your Incident Response Procedures!
  • 56.
    Building a Internet Posting Policy
  • 57.
    Define your Social Media Strategy • Partner with Marketing/Public Relations/HR • What is acceptable for employees to post? • At work/off work • Employees have mobile devices, home computers!
  • 58.
    Define what gets monitored? • Difficult or impossible to monitor everything • Determine with your partners what should be monitored • Careful with policy conflicts!
  • 59.
  • 60.
  • 61.
    Communicate to your employees! How can you enforce a policy if employees don’t know about it?
  • 62.
    Where to learnmore? • Great paper on Metadata (SANS Reading Room): “Document Metadata, the Silent Killer” - Larry Pesce • Maltego Tutorials: Chris Gates, EthicalHacker.net • My blog: spylogic.net
  • 63.
    OSINT 3 PartSeries • All the details from this presentation! • Part 1 - Social Networks http://bit.ly/osint1 • Part 2 - Blogs, Message Boards, Metadata http://bit.ly/osint2 • Part 3 - Monitoring, Social Media Policies http://bit.ly/osint3

Editor's Notes

  • #7 How many of us as security professionals think of reputational issues in regards to the company brand?