Open Source
Intelligence
Leveraging Data into Intelligence
Deep Shankar Yadav
Digital Forensics Analyst
root@charlie:~$ whoami
• Digital Forensics Analyst
• Co-Leader – OWASP AppSec India
• OSINT Enthusiast
• Consultant to a few Law Enforcement
Agencies
• @TheDeepSYadav
Agenda
• What is Intelligence?
• What is OSINT?
• Why OSINT?
• What is Value of OSINT?
• Sources of Data
• Demo
What is Intelligence?
• Data : Raw Report, Images or Broadcast
• Information : Collected Data of Generic Interest
• Intelligence : concisely tailored answer reflecting
a deliberate process of discovery, discrimination,
distillation, and delivery of data precisely suited to
need
Types of Intelligence
• HUMINT – Human Intelligence Sleeper Cells
• SIGINT - Signal Intelligence Traffic Analysis,
TEMPEST
• MSINT – Measurement and Signature Intelligence
Radar, Nuclear and Earthquake
• Tactical Intelligence – Intelligence about Weapons
used by Forces
• OSINT – Open Source Intelligence
What is OSINT?
Open-source intelligence (OSINT) is intelligence
collected from publicly available sources.
It’s not a tool, It’s not a website, It’s not with any
fee but it’s not free….. :)
Why OSINT?
Why OSINT?
• Internet is not limited to Google Searches.
• Not even limited to search engines, social media and
blogs.
• Huge number of sensational hacks in recent times
Organizations getting hacked even after using so
called "sophisticated" defense mechanisms.
• Basic recon usually ignored during security
assessments.
• If your SECRET is out there in the open, someone
WILL find it.
• It's just data until you leverage it to create
intelligence.
If you are lucky you will get
everything
Sensitive
Information
Hard
coded
keys in
Github
Credential
leaks in
Pastebin
0-days
sold in
darknet
Hack info
in micro
blog
Corporate
email
credentials
Open
Bugs or
ports
Why is it becoming easy?
Typical Pentesting Methodology
Post-
Exploit
Cover
Tracks
Write
report
I.G Scan Enumerate Exploit
What everyone focus on:
Enumera
te
Post-
Exploit
Cover
Tracks
Write
report
I.G Scan Exploit
Attacker Methodology
Informaion Gathering
Discover what makes
the company money
Do whatever it
takes...
Steal it
Discover what is
valuable to the
atacker
OSINT PROCESS
Source Identification
Data harvesting
Data processing and
Integration
Data Analysis
Results Delivery
Source Identification
Data Harvesting
Data Processing
Data Analysis
• Selecting Data as per Case.
• Removing Noise (Buy Disprin Before it :P )
• Making Intelligence on behalf of data. (Give wake
up call to your inner Sherlock)
Results Delivery
Data Sources
• Web Directories – WWW Library and Internet Public Library
• Search Engines – Google, Yahoo, Bing, Baidu, MSN etc. etc.
• Important Engine – Wolframe Alpha, ROBTEX
• Serial Search Engines – Soolve , myallsearch
• Hardware Search Engine – Shodan
• Scientific Search Engine – Google Scholar, Academia
• Encyclopedia – Wikipedia
• Book Stores – Amazon, Google Books etc.
• Translations – Google, Bing and many more
• Aeronautical – flightradar24.com and many more
• Blogs – Blogspot, Wordpress and many more
Cntd.
• Company Information – MCA.gov.in, Zaubacorp
• Images – Google Images, Flickr, PICASA and much important –
Tineye
• Internet Tools – Archive.org, Whois, VisualRoute,
ip2location.com etc etc
• Thousand of Journals and Print Media Sources
• Maps – Google Maps, Google Earth, Wikimap
• People Finders – pipl, spokeo etc etc, Public Records -
Intellius
• Social Networking Websites – Facebook, Twitter, G Plus,
LinkedIn etc etc
• Matrimonial Websites
• Job Portals
Data Harvesting
A.K.A:
• Information Gathering:
The act of collecing informaion
• Foot printing:
Is the technique of gathering informaion about
computer systems and the eniies they belong to.
• Web mining:
The act of collecing informaion from the web
Data Harvesting – How?
•
•
•
•
•
•
•
•
Scraping (raw)
Open APIs
Commercial APIS
Network Scanning
Purchasing data
Open source Data sets
Databases
Logfiles
Offensive OSINT – goals
•
•
•
•
•
Phishing
Social Engineering
Denial of Services
Password brute force
Target infiltraion
atacks
What data is interesting?
Emails
Users / Employees names
-Interests
-People relationships
-Alias
Emails
•
•
•
PGP servers
Search engines
Whois
Employees
linkedin.com
jigsaw.com
people123.com
pipl.com
peekyou.com
/ Usernames / Alias
Glassdoor.com
Hoovers.com
Corpwatch.org
intelius.com
Google Finance / Etc.
Usernamecheck.com
checkusernames.com
Username checks
Social Media
Metadata
Metadata: is data about data.
Is used to facilitate the understanding, use and management
of data.
•
•
•
•
•
Office documents
Openoffice documents
PDF documents
Images EXIF metadata
Others
Brace yourself Demo is starting
Some Notable tools to work upon
• Datasploit (http://github.com/upgoingstar)
• Metagoofil
• The Harvestor
• FOCA
• Creepy
• Maltego
Current Problems in OSINT
•
•
•
Source availability
Changes in Terms of Use
Generaing valid intelligence
Any Queries?
Sources
• Slideshare.com
• Google.com
• Some Deep Web Forums
I might have copied some images and content from
other ppt’s and articles and credits are given where
required so don’t worry  I got them via googling
only
Deep Shankar Yadav
• mail@deepshankaryadav.net
• advisorcybercell@gmail.com
• http://www.deepshankaryadav.net
• fb.me/deepshankaryadav
• @TheDeepSYadav

OSINT- Leveraging data into intelligence

  • 1.
    Open Source Intelligence Leveraging Datainto Intelligence Deep Shankar Yadav Digital Forensics Analyst
  • 2.
    root@charlie:~$ whoami • DigitalForensics Analyst • Co-Leader – OWASP AppSec India • OSINT Enthusiast • Consultant to a few Law Enforcement Agencies • @TheDeepSYadav
  • 3.
    Agenda • What isIntelligence? • What is OSINT? • Why OSINT? • What is Value of OSINT? • Sources of Data • Demo
  • 4.
    What is Intelligence? •Data : Raw Report, Images or Broadcast • Information : Collected Data of Generic Interest • Intelligence : concisely tailored answer reflecting a deliberate process of discovery, discrimination, distillation, and delivery of data precisely suited to need
  • 5.
    Types of Intelligence •HUMINT – Human Intelligence Sleeper Cells • SIGINT - Signal Intelligence Traffic Analysis, TEMPEST • MSINT – Measurement and Signature Intelligence Radar, Nuclear and Earthquake • Tactical Intelligence – Intelligence about Weapons used by Forces • OSINT – Open Source Intelligence
  • 6.
    What is OSINT? Open-sourceintelligence (OSINT) is intelligence collected from publicly available sources. It’s not a tool, It’s not a website, It’s not with any fee but it’s not free….. :)
  • 7.
  • 8.
    Why OSINT? • Internetis not limited to Google Searches. • Not even limited to search engines, social media and blogs. • Huge number of sensational hacks in recent times Organizations getting hacked even after using so called "sophisticated" defense mechanisms. • Basic recon usually ignored during security assessments. • If your SECRET is out there in the open, someone WILL find it. • It's just data until you leverage it to create intelligence.
  • 9.
    If you arelucky you will get everything
  • 10.
    Sensitive Information Hard coded keys in Github Credential leaks in Pastebin 0-days soldin darknet Hack info in micro blog Corporate email credentials Open Bugs or ports
  • 11.
    Why is itbecoming easy?
  • 13.
  • 14.
    What everyone focuson: Enumera te Post- Exploit Cover Tracks Write report I.G Scan Exploit
  • 15.
    Attacker Methodology Informaion Gathering Discoverwhat makes the company money Do whatever it takes... Steal it Discover what is valuable to the atacker
  • 17.
    OSINT PROCESS Source Identification Dataharvesting Data processing and Integration Data Analysis Results Delivery
  • 18.
  • 19.
  • 20.
  • 21.
    Data Analysis • SelectingData as per Case. • Removing Noise (Buy Disprin Before it :P ) • Making Intelligence on behalf of data. (Give wake up call to your inner Sherlock)
  • 22.
  • 23.
    Data Sources • WebDirectories – WWW Library and Internet Public Library • Search Engines – Google, Yahoo, Bing, Baidu, MSN etc. etc. • Important Engine – Wolframe Alpha, ROBTEX • Serial Search Engines – Soolve , myallsearch • Hardware Search Engine – Shodan • Scientific Search Engine – Google Scholar, Academia • Encyclopedia – Wikipedia • Book Stores – Amazon, Google Books etc. • Translations – Google, Bing and many more • Aeronautical – flightradar24.com and many more • Blogs – Blogspot, Wordpress and many more
  • 24.
    Cntd. • Company Information– MCA.gov.in, Zaubacorp • Images – Google Images, Flickr, PICASA and much important – Tineye • Internet Tools – Archive.org, Whois, VisualRoute, ip2location.com etc etc • Thousand of Journals and Print Media Sources • Maps – Google Maps, Google Earth, Wikimap • People Finders – pipl, spokeo etc etc, Public Records - Intellius • Social Networking Websites – Facebook, Twitter, G Plus, LinkedIn etc etc • Matrimonial Websites • Job Portals
  • 25.
    Data Harvesting A.K.A: • InformationGathering: The act of collecing informaion • Foot printing: Is the technique of gathering informaion about computer systems and the eniies they belong to. • Web mining: The act of collecing informaion from the web
  • 26.
    Data Harvesting –How? • • • • • • • • Scraping (raw) Open APIs Commercial APIS Network Scanning Purchasing data Open source Data sets Databases Logfiles
  • 27.
    Offensive OSINT –goals • • • • • Phishing Social Engineering Denial of Services Password brute force Target infiltraion atacks
  • 28.
    What data isinteresting? Emails Users / Employees names -Interests -People relationships -Alias
  • 29.
  • 30.
    Employees linkedin.com jigsaw.com people123.com pipl.com peekyou.com / Usernames /Alias Glassdoor.com Hoovers.com Corpwatch.org intelius.com Google Finance / Etc. Usernamecheck.com checkusernames.com
  • 31.
  • 32.
  • 33.
    Metadata Metadata: is dataabout data. Is used to facilitate the understanding, use and management of data. • • • • • Office documents Openoffice documents PDF documents Images EXIF metadata Others
  • 36.
  • 37.
    Some Notable toolsto work upon • Datasploit (http://github.com/upgoingstar) • Metagoofil • The Harvestor • FOCA • Creepy • Maltego
  • 38.
    Current Problems inOSINT • • • Source availability Changes in Terms of Use Generaing valid intelligence
  • 39.
  • 41.
    Sources • Slideshare.com • Google.com •Some Deep Web Forums I might have copied some images and content from other ppt’s and articles and credits are given where required so don’t worry  I got them via googling only
  • 42.
    Deep Shankar Yadav •mail@deepshankaryadav.net • advisorcybercell@gmail.com • http://www.deepshankaryadav.net • fb.me/deepshankaryadav • @TheDeepSYadav