Intrigue Core: Scaling Assessment Automation

Jonathan Cran
Jonathan CranSecurity Research at Kenna Security
Scaling Up!
Automated Attack Surface Discovery
With Intrigue Core
NahamCon - June 2020
jcran@intrigue.io
Hi, I’m Jonathan Cran
• Human Hacker / Builder
• Head of Research @ Kenna
Security, Risk Based VM
• Founder of Intrigue.io - Attack
Surface Monitoring
• Formerly… Bugcrowd, Rapid7,
Pwnie Express
Intrigue Core: Scaling Assessment Automation
Intrigue Ident
Application and Network Fingeprinting With Ident
• Techniques
• FTP Banner
• HTTP cookies
• HTTP headers
• HTTP body
• HTTP title
• HTTP generator tag
• SMTP Banner
• SNMP Banner
• SSH BAnner
• TELNET Banner
• Application Fingerprinting
• Over 700 Unique Fingerprints
• Mappable to CVEs
• Network Fingerprinting
• With the integration of Recog, ~4000+
• Telnet, SSH, SNMP, SMTP, FTP
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Try It Out!
$ docker run -it intrigueio/intrigue-ident -u https://intrigue.io
Try options like —list, —vulnerabilities —debug —json
Intrigue Core
What Is Intrigue Core?
• Automation Framework and
Orchestration Engine
• Scriptable internally or via API
• Distributed via Docker
• Powers data collection for
intrigue.io
docker run 
-e LANG=C.UTF-8 
-v ~/intrigue-core-data:/data 
-p 0.0.0.0:7777:7777
-i -t intrigueio/intrigue-core:latest
Key Features & Capabilities
• Simple Web interface and API
• Web & Service Fingerprinting
• Web Spidering
• Vulnerability Inference
• Vulnerability Checking
• Automated Web Screenshots
• Metadata parsing
• Integrations to most open data
providers
• Notifications & Alerting
• Export to JSON / etc
• Exposure Analysis
• Geolocation
• Threat X-references
• Network & Cloud Provider
Determination
Core Entities
• NetworkService

• Credential

• WebAccount

• GithubAccount

• GithubRepository

• IpAddress

• AwsS3Bucket

• Info

• FileHash

• Uri

• AwsIAMAccount

• EmailAddress

• Nameserver

• AutonomousSystem

• DnsRecord

• NetBlock

• SoftwarePackage

• Organization

• PhoneNumber

• AwsRegion

• PhysicalLocation

• AwsCredential

• AnalyticsId

• Person

• Domain

• SslCertificate
Core Discovery Techniques
AWS S3 Brute

AWS S3 Put File

Apache 'Server
Status' Parser

DNS Cache
Snoop

DNS DKIM
Lookup

DNS MX Lookup

DNS Morph

DNS NSEC
record walk

DNS Permute

DNS SPF
Recursive
Lookup

DNS Service
Record
Bruteforce

DNS Subdomain
Bruteforce

DNS Zone
Transfer

Email Brute
Gmail GLXU

Email Harvester

Email Validate via
MailboxLayer

Enumerate
Nameservers

Enumerate an
FTP server

Geolocate IP
Address

Gitrob

Import ARIN IPv4
Ranges

Import AWS IPv4
Ranges

Import CVEs
from NVD (JSON)

Import Data File

Import Domains
from Domainlists

Import Latest
Pulses from
Alienvault OTX

Import Shodan
JSON

Import Umbrella
Top Domains

Import Umbrella
Top Sites

Masscan Scan

NetBlock Expand

Nmap Scan

Phone Number
Lookup

Rdpscan Scan

SNMP Walk

SaaS Google
Calendar Check

SaaS Google
Groups Check

SaaS Jira Check

SaaS Trello
Check

Scrape
PublicWWW

TCP Bind And
Collect

URI Analyze
Target

URI Brute (List)

URI Brute
Common
Content

URI Brute
Focused Content

URI Bruteforce

URI Bruteforce
Credentials

URI Check
HTTP/2 Support

URI Check
Security Headers

URI Check
Subdomain
Hijack

URI Enumerate
JS

URI Extract
Metadata

URI Gather
Linked Content

URI Gather
Robots.txt

URI Gather SSL
Certificate

URI Gather
Sitemap
(sitemap.xml)

URI Gather Well-
Known Files
(RFC5785)

URI Screenshot

URI Spider

URI Youtube
Metadata

Web Account
Check

Whois Lookup

Wordpress
Enumerate
Plugins

Wordpress
Enumerate Users
Core Dataset Integrations
AWS EC2
Gather
Instances

AWS IAM
Gather
Accounts

Search
Alienvault OTX

Search
Alienvault OTX
Hashes

Search
Alienvault OTX
Related
Hostnames

Search BGP

Search
BinaryEdge

Search
BinaryEdge
Risk Score

Search
BinaryEdge
Torrent

Search Bing

Search Bing
Organization
Name to
Domains

Search
BuiltWith

Search CRT

Search
Censys.io

Search
CertSpotter

Search
CleanBrowsin
g DNS

Search
Comodo DNS

Search
DeHashed

Search
EDGAR

DNS Search
Sonar

Search Github

Search Github
Code

Search
Grayhat
Warfare

Search Have I
Been Pwned
(HIBP)

Search
Hunter.io

Search
OpenCorporat
es

Search
OpenDNS

Search
Phishtank

Search Project
Honeypot

Search
Pulsedive

Search Quad9
DNS

Search
Robtex

Search
Shodan

Search
SpyOnWeb

Search
Sublist3r

Search
ThreatCrowd

Search
Towerdata

Search
ViewDNS
(Reverse
Whois)

Search
VirusTotal

Search
Whoisology

Search
Yandex DNS

Security Trails
Historical DNS
Lookup

Security Trails
Historical
WHOIS

Security Trails
Nameserver
Search

Security Trails
Subdomain
Search
Scaling Data
Collection
Core Automated Scoping
• Automated scoping enables intelligent iteration
• Rule-based
• 👍 Anything the user scopes in
• 👍 Anything explicitly requested at the time of task creation
• 👍 Per-entity scoping rules
Machines
• Machines are automation that enable
entity-level decisions
• Machines are (usually) recursive and
build a “graph”
• Can be Active or Passive
• Designed with a collection purpose in
mind
• Example:
• Domain
• Sub-brute
• Check Project Sonar
• Ip Address
• Nmap Scan
• NetworkService
• Grab Banner
Core Entity Enrichment and Normalization
• Enrichment ensures we can automate without overwhelming complexity
• Example:
• Domain
• Look it up, Grab SOA (if available)
• Grab MX, NS, TXT, etc
• Resolve to an IP (which creates an entity and starts enrichment)
Core Vulnerability Inference
• Ident Fingerprints are correlated to NVD
• Converted to CPEs
• Version-based vulnerability inference
• Command line or as a library
Scaling Data
Analysis
Taking Analysis
To the Next Level
With ElasticSearch
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:279145205744:domain/nahamcon/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": “x.x.x.x”
}
}
}
]
}
Make sure to whitelist your IP
Getting Data Into ES
Core Handlers
• Notifiers, Alerters and Exporters
• Configured on individual Task, Machine or Project
• Can be configured to run at completion
• Handy for building data pipelines
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Intrigue Core: Scaling Assessment Automation
Thank You Nahamcon!
🙏🙏🙏
Join Us in Slack!
Email Hello@Intrigue.Io for an Invite
1 of 36

Recommended

JWT: jku x5uJWT: jku x5u
JWT: jku x5usnyff
6.7K views58 slides
Jwt == insecurity?Jwt == insecurity?
Jwt == insecurity?snyff
18.6K views55 slides
I See YouI See You
I See YouAndrew Beard
1.5K views37 slides
Defcon CTF qualsDefcon CTF quals
Defcon CTF qualssnyff
1.5K views30 slides

More Related Content

What's hot(19)

HTTP For the Good or the BadHTTP For the Good or the Bad
HTTP For the Good or the Bad
Xavier Mertens4.3K views
Security and Privacy on the Web in 2016Security and Privacy on the Web in 2016
Security and Privacy on the Web in 2016
Francois Marier671 views

Similar to Intrigue Core: Scaling Assessment Automation

Starwest 2008Starwest 2008
Starwest 2008Caleb Sima
798 views40 slides
Web Security - IntroductionWeb Security - Introduction
Web Security - IntroductionSQALab
32.7K views68 slides
Romulus OWASPRomulus OWASP
Romulus OWASPGrupo Gesfor I+D+i
2.1K views54 slides

Similar to Intrigue Core: Scaling Assessment Automation(20)

Hacking Client Side InsecuritiesHacking Client Side Insecurities
Hacking Client Side Insecurities
amiable_indian2.5K views
Starwest 2008Starwest 2008
Starwest 2008
Caleb Sima798 views
technical-information-gathering-slides.pdftechnical-information-gathering-slides.pdf
technical-information-gathering-slides.pdf
MarceloCunha57164925 views
Web Security - IntroductionWeb Security - Introduction
Web Security - Introduction
SQALab32.7K views
Web Security - Introduction v.1.3Web Security - Introduction v.1.3
Web Security - Introduction v.1.3
Oles Seheda78.6K views
Romulus OWASPRomulus OWASP
Romulus OWASP
Grupo Gesfor I+D+i2.1K views
04.  xss and encoding04.  xss and encoding
04. xss and encoding
Eoin Keary2K views
Spa Secure Coding GuideSpa Secure Coding Guide
Spa Secure Coding Guide
Geoffrey Vandiest152 views
Cloud Native Identity with SPIFFECloud Native Identity with SPIFFE
Cloud Native Identity with SPIFFE
Prabath Siriwardena2.7K views
Android application analyzerAndroid application analyzer
Android application analyzer
Sanjay Gondaliya204 views
Recon like a proRecon like a pro
Recon like a pro
Nirmalthapa24116 views
Blue Teaming On A BudgetBlue Teaming On A Budget
Blue Teaming On A Budget
KevinRiley8364 views
API SECURITYAPI SECURITY
API SECURITY
Tubagus Rizky Dharmawan307 views

Recently uploaded(20)

childcare.pdfchildcare.pdf
childcare.pdf
fatma alnaqbi13 views
DU_SERIES_Session1.pdfDU_SERIES_Session1.pdf
DU_SERIES_Session1.pdf
RohitRadhakrishnan8807 views
AI Powered event-driven translation botAI Powered event-driven translation bot
AI Powered event-driven translation bot
Jimmy Dahlqvist16 views
Serverless cloud architecture patternsServerless cloud architecture patterns
Serverless cloud architecture patterns
Jimmy Dahlqvist17 views
WEB 2.O TOOLS: Empowering education.pptxWEB 2.O TOOLS: Empowering education.pptx
WEB 2.O TOOLS: Empowering education.pptx
narmadhamanohar218 views
KHNOG 5: APNIC ServicesKHNOG 5: APNIC Services
KHNOG 5: APNIC Services
APNIC414 views
Audience profile.pptxAudience profile.pptx
Audience profile.pptx
MollyBrown8612 views
Is Entireweb better than GoogleIs Entireweb better than Google
Is Entireweb better than Google
sebastianthomasbejan10 views
google forms survey (1).pptxgoogle forms survey (1).pptx
google forms survey (1).pptx
MollyBrown8614 views
UiPath Document Understanding_Day 2.pptxUiPath Document Understanding_Day 2.pptx
UiPath Document Understanding_Day 2.pptx
RohitRadhakrishnan8265 views
informationinformation
information
khelgishekhar6 views
 FS Design 2024 V2.pptx FS Design 2024 V2.pptx
FS Design 2024 V2.pptx
paswanlearning7 views
Existing documentaries (1).docxExisting documentaries (1).docx
Existing documentaries (1).docx
MollyBrown8613 views
Sustainable MarketingSustainable Marketing
Sustainable Marketing
Theo van der Zee7 views
DU Series - Day 4.pptxDU Series - Day 4.pptx
DU Series - Day 4.pptx
UiPathCommunity77 views

Intrigue Core: Scaling Assessment Automation

  • 1. Scaling Up! Automated Attack Surface Discovery With Intrigue Core NahamCon - June 2020 jcran@intrigue.io
  • 2. Hi, I’m Jonathan Cran • Human Hacker / Builder • Head of Research @ Kenna Security, Risk Based VM • Founder of Intrigue.io - Attack Surface Monitoring • Formerly… Bugcrowd, Rapid7, Pwnie Express
  • 5. Application and Network Fingeprinting With Ident • Techniques • FTP Banner • HTTP cookies • HTTP headers • HTTP body • HTTP title • HTTP generator tag • SMTP Banner • SNMP Banner • SSH BAnner • TELNET Banner • Application Fingerprinting • Over 700 Unique Fingerprints • Mappable to CVEs • Network Fingerprinting • With the integration of Recog, ~4000+ • Telnet, SSH, SNMP, SMTP, FTP
  • 8. Try It Out! $ docker run -it intrigueio/intrigue-ident -u https://intrigue.io Try options like —list, —vulnerabilities —debug —json
  • 10. What Is Intrigue Core? • Automation Framework and Orchestration Engine • Scriptable internally or via API • Distributed via Docker • Powers data collection for intrigue.io docker run -e LANG=C.UTF-8 -v ~/intrigue-core-data:/data -p 0.0.0.0:7777:7777 -i -t intrigueio/intrigue-core:latest
  • 11. Key Features & Capabilities • Simple Web interface and API • Web & Service Fingerprinting • Web Spidering • Vulnerability Inference • Vulnerability Checking • Automated Web Screenshots • Metadata parsing • Integrations to most open data providers • Notifications & Alerting • Export to JSON / etc • Exposure Analysis • Geolocation • Threat X-references • Network & Cloud Provider Determination
  • 12. Core Entities • NetworkService • Credential • WebAccount • GithubAccount • GithubRepository • IpAddress • AwsS3Bucket • Info • FileHash • Uri • AwsIAMAccount • EmailAddress • Nameserver • AutonomousSystem • DnsRecord • NetBlock • SoftwarePackage • Organization • PhoneNumber • AwsRegion • PhysicalLocation • AwsCredential • AnalyticsId • Person • Domain • SslCertificate
  • 13. Core Discovery Techniques AWS S3 Brute AWS S3 Put File Apache 'Server Status' Parser DNS Cache Snoop DNS DKIM Lookup DNS MX Lookup DNS Morph DNS NSEC record walk DNS Permute DNS SPF Recursive Lookup DNS Service Record Bruteforce DNS Subdomain Bruteforce DNS Zone Transfer Email Brute Gmail GLXU Email Harvester Email Validate via MailboxLayer Enumerate Nameservers Enumerate an FTP server Geolocate IP Address Gitrob Import ARIN IPv4 Ranges Import AWS IPv4 Ranges Import CVEs from NVD (JSON) Import Data File Import Domains from Domainlists Import Latest Pulses from Alienvault OTX Import Shodan JSON Import Umbrella Top Domains Import Umbrella Top Sites Masscan Scan NetBlock Expand Nmap Scan Phone Number Lookup Rdpscan Scan SNMP Walk SaaS Google Calendar Check SaaS Google Groups Check SaaS Jira Check SaaS Trello Check Scrape PublicWWW TCP Bind And Collect URI Analyze Target URI Brute (List) URI Brute Common Content URI Brute Focused Content URI Bruteforce URI Bruteforce Credentials URI Check HTTP/2 Support URI Check Security Headers URI Check Subdomain Hijack URI Enumerate JS URI Extract Metadata URI Gather Linked Content URI Gather Robots.txt URI Gather SSL Certificate URI Gather Sitemap (sitemap.xml) URI Gather Well- Known Files (RFC5785) URI Screenshot URI Spider URI Youtube Metadata Web Account Check Whois Lookup Wordpress Enumerate Plugins Wordpress Enumerate Users
  • 14. Core Dataset Integrations AWS EC2 Gather Instances AWS IAM Gather Accounts Search Alienvault OTX Search Alienvault OTX Hashes Search Alienvault OTX Related Hostnames Search BGP Search BinaryEdge Search BinaryEdge Risk Score Search BinaryEdge Torrent Search Bing Search Bing Organization Name to Domains Search BuiltWith Search CRT Search Censys.io Search CertSpotter Search CleanBrowsin g DNS Search Comodo DNS Search DeHashed Search EDGAR DNS Search Sonar Search Github Search Github Code Search Grayhat Warfare Search Have I Been Pwned (HIBP) Search Hunter.io Search OpenCorporat es Search OpenDNS Search Phishtank Search Project Honeypot Search Pulsedive Search Quad9 DNS Search Robtex Search Shodan Search SpyOnWeb Search Sublist3r Search ThreatCrowd Search Towerdata Search ViewDNS (Reverse Whois) Search VirusTotal Search Whoisology Search Yandex DNS Security Trails Historical DNS Lookup Security Trails Historical WHOIS Security Trails Nameserver Search Security Trails Subdomain Search
  • 16. Core Automated Scoping • Automated scoping enables intelligent iteration • Rule-based • 👍 Anything the user scopes in • 👍 Anything explicitly requested at the time of task creation • 👍 Per-entity scoping rules
  • 17. Machines • Machines are automation that enable entity-level decisions • Machines are (usually) recursive and build a “graph” • Can be Active or Passive • Designed with a collection purpose in mind • Example: • Domain • Sub-brute • Check Project Sonar • Ip Address • Nmap Scan • NetworkService • Grab Banner
  • 18. Core Entity Enrichment and Normalization • Enrichment ensures we can automate without overwhelming complexity • Example: • Domain • Look it up, Grab SOA (if available) • Grab MX, NS, TXT, etc • Resolve to an IP (which creates an entity and starts enrichment)
  • 19. Core Vulnerability Inference • Ident Fingerprints are correlated to NVD • Converted to CPEs • Version-based vulnerability inference • Command line or as a library
  • 21. Taking Analysis To the Next Level With ElasticSearch
  • 29. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "es:*", "Resource": "arn:aws:es:us-east-1:279145205744:domain/nahamcon/*", "Condition": { "IpAddress": { "aws:SourceIp": “x.x.x.x” } } } ] } Make sure to whitelist your IP
  • 31. Core Handlers • Notifiers, Alerters and Exporters • Configured on individual Task, Machine or Project • Can be configured to run at completion • Handy for building data pipelines
  • 36. Thank You Nahamcon! 🙏🙏🙏 Join Us in Slack! Email Hello@Intrigue.Io for an Invite