SlideShare a Scribd company logo
Secure	
  Because	
  Math:	
  A	
  Deep-­‐Dive	
  on	
  
Machine	
  Learning-­‐Based	
  Monitoring	
  	
  
(#SecureBecauseMath)	
  
Alex	
  Pinto	
  
Chief	
  Data	
  Scien2st	
  |	
  MLSec	
  Project	
  	
  
@alexcpsec	
  
@MLSecProject!
Alex	
  Pinto	
  
•  Chief	
  Data	
  Scien2st	
  at	
  MLSec	
  Project	
  
•  Machine	
  Learning	
  Researcher	
  and	
  Trainer	
  
•  Network	
  security	
  and	
  incident	
  response	
  aficionado	
  	
  
•  Tortured	
  by	
  SIEMs	
  as	
  a	
  child	
  
•  Hacker	
  Spirit	
  Animal™:	
  CAFFEINATED	
  CAPYBARA!
whoami	
  
(hPps://secure.flickr.com/photos/kobashi_san/)	
  
 
•  Security	
  Singularity	
  
•  Some	
  History	
  
•  TLA	
  
•  ML	
  Marke2ng	
  PaPerns	
  
•  Anomaly	
  Detec2on	
  
•  Classifica2on	
  
•  Buyer’s	
  Guide	
  
•  MLSec	
  Project	
  
Agenda	
  
Security	
  Singularity	
  Approaches	
  
(Side	
  Note)	
  
First	
  hit	
  on	
  Google	
  images	
  for	
  “Network	
  Security	
  Solved”	
  is	
  a	
  
picture	
  of	
  Jack	
  Daniel!
Security	
  Singularity	
  Approaches	
  
•  “Machine	
  learning	
  /	
  math	
  /	
  algorithms…	
  these	
  terms	
  are	
  
used	
  interchangeably	
  quite	
  frequently.”	
  
•  “Is	
  behavioral	
  baselining	
  and	
  anomaly	
  detec2on	
  part	
  of	
  
this?”	
  
•  “What	
  about	
  Big	
  Data	
  Security	
  Analy2cs?”	
  
	
  
(hPp://bigdatapix.tumblr.com/)	
  
Are	
  we	
  even	
  trying?	
  
•  “Hyper-­‐dimensional	
  security	
  
analy2cs”	
  
•  “3rd	
  genera2on	
  Ar2ficial	
  
Intelligence”	
  
•  “Secure	
  because	
  Math”	
  
	
  
•  Lack	
  of	
  ability	
  to	
  differen2ate	
  
hurts	
  buyers,	
  investors.	
  
•  Are	
  we	
  even	
  funding	
  the	
  right	
  
things?	
  
Is	
  this	
  a	
  communicaCon	
  issue?	
  
Guess	
  the	
  Year!	
  
•  “(…)	
  behavior	
  analysis	
  system	
  that	
  enhances	
  your	
  
network	
  intelligence	
  and	
  security	
  by	
  audi2ng	
  network	
  
flow	
  data	
  from	
  exis2ng	
  infrastructure	
  devices”	
  
•  "Mathema2cal	
  models	
  (…)	
  that	
  determine	
  baseline	
  
behavior	
  across	
  users	
  and	
  machines,	
  detec2ng	
  (...)	
  
anomalous	
  and	
  risky	
  ac2vi2es	
  (...)”	
  
•  ”(…)	
  maintains	
  historical	
  profiles	
  of	
  usage	
  per	
  user	
  and	
  
raises	
  an	
  alarm	
  when	
  observed	
  ac2vity	
  departs	
  from	
  
established	
  paPerns	
  of	
  usage	
  for	
  an	
  individual.”	
  	
  
A	
  liGle	
  history	
  
•  Dorothy	
  E.	
  Denning	
  (professor	
  at	
  the	
  
Department	
  of	
  Defense	
  Analysis	
  at	
  the	
  
Naval	
  Postgraduate	
  School)	
  
•  1986	
  (SRI)	
  -­‐	
  First	
  research	
  that	
  led	
  
to	
  IDS	
  
•  Intrusion	
  Detec2on	
  Expert	
  System	
  
(IDES)	
  
•  Already	
  had	
  sta2s2cal	
  anomaly	
  
detec2on	
  built-­‐in	
  
•  1993:	
  Her	
  colleagues	
  release	
  the	
  Next	
  
Genera2on	
  (!)	
  IDES	
  
Three	
  LeGer	
  Acronyms	
  -­‐	
  KDD	
  
•  Ajer	
  the	
  release	
  of	
  Bro	
  (1998)	
  and	
  Snort	
  (1999),	
  DARPA	
  
thought	
  we	
  were	
  covered	
  for	
  this	
  signature	
  thing	
  
•  DARPA	
  released	
  datasets	
  for	
  user	
  anomaly	
  detec2on	
  in	
  
1998	
  and	
  1999	
  
•  And	
  then	
  came	
  the	
  KDD-­‐99	
  dataset	
  –	
  over	
  6200	
  cita2ons	
  
on	
  Google	
  Scholar	
  
Three	
  LeGer	
  Acronyms	
  
Three	
  LeGer	
  Acronyms	
  -­‐	
  KDD	
  
Trolling,	
  maybe?	
  
Not	
  here	
  to	
  bash	
  academia	
  
A	
  Probable	
  Outcome	
  
GRAD	
  
SCHOOL	
  
FRESHMAN	
  
ZOMG	
  
RESULTS	
  !!
11!1!	
  
ZOMG!	
  
RESULTS???	
  
MATH,	
  STAHP!	
  
MATH	
  IS	
  
HARD,	
  LET’S	
  
GO	
  SHOPPING	
  
ML	
  MarkeCng	
  PaGerns	
  
•  The	
  “Has-­‐beens”	
  	
  
•  Name	
  is	
  a	
  bit	
  harsh,	
  but	
  hey,	
  you	
  hardly	
  use	
  ML	
  
anymore,	
  let	
  us	
  try	
  it	
  
•  The	
  “Machine	
  Learning	
  ¯ˉ_(ツ)_/¯ˉ”	
  
•  Hey,	
  that	
  sounds	
  cool,	
  let’s	
  put	
  that	
  in	
  our	
  brochure	
  
•  The	
  “Sweet	
  Spot”	
  
•  People	
  that	
  actually	
  are	
  trying	
  to	
  do	
  something	
  
•  Anomaly	
  Detec2on	
  vs.	
  Classifica2on	
  
Anomaly	
  DetecCon	
  
Anomaly	
  DetecCon	
  
•  Works	
  wonders	
  for	
  well	
  
defined	
  “industrial-­‐like”	
  
processes.	
  
•  Looking	
  at	
  single,	
  
consistently	
  measured	
  
variables	
  
•  Historical	
  usage	
  in	
  financial	
  
fraud	
  preven2on.	
  
Anomaly	
  DetecCon	
  
Anomaly	
  DetecCon	
  
•  What	
  fits	
  this	
  mold?	
  
•  Network/Neqlow	
  behavior	
  analysis	
  	
  
•  User	
  behavior	
  analysis	
  
•  What	
  are	
  the	
  challenges?	
  
•  Curse	
  of	
  Dimensionality	
  
•  Lack	
  of	
  ground	
  truth	
  and	
  normality	
  poisoning	
  
•  Hanlon’s	
  Razor	
  
AD:	
  Curse	
  of	
  Dimensionality	
  
•  We	
  need	
  “distances”	
  to	
  measure	
  
the	
  features/variables	
  
•  Usually	
  ManhaPan	
  or	
  Euclidian	
  
•  For	
  high-­‐dimensional	
  data,	
  the	
  
distribu2on	
  of	
  distances	
  between	
  
all	
  pairwise	
  points	
  in	
  the	
  space	
  
becomes	
  concentrated	
  around	
  an	
  
average	
  distance.	
  
AD:	
  Curse	
  of	
  Dimensionality	
  
•  The	
  volume	
  of	
  the	
  high	
  
dimensional	
  sphere	
  
becomes	
  negligible	
  in	
  
rela2on	
  to	
  the	
  volume	
  of	
  
the	
  high	
  dimensional	
  cube.	
  
•  The	
  prac2cal	
  result	
  is	
  that	
  
everything	
  just	
  seems	
  too	
  
far	
  away,	
  and	
  at	
  similar	
  
distances.	
  
(hPp://www.datasciencecentral.com/m/blogpost?
id=6448529%3ABlogPost%3A175670)	
  
A	
  PracCcal	
  example	
  
•  NetFlow	
  data,	
  company	
  with	
  n	
  internal	
  nodes.	
  
•  2(nˆ2	
  -­‐	
  n)	
  communica2on	
  direc2ons	
  
•  2*2*2*65535(nˆ2	
  -­‐	
  n)	
  measures	
  of	
  network	
  ac2vity	
  
•  1000	
  nodes	
  -­‐>	
  Half	
  a	
  trillion	
  possible	
  dimensions	
  
Breaking	
  the	
  Curse	
  
•  Different	
  /	
  crea2ve	
  
distance	
  metrics	
  
•  Organizing	
  the	
  space	
  into	
  
sub-­‐manifolds	
  where	
  
Euclidean	
  distances	
  make	
  
more	
  sense.	
  
•  Aggressive	
  feature	
  
removal	
  
•  A	
  few	
  interes2ng	
  results	
  
available	
  
Breaking	
  the	
  Curse	
  
AD:	
  Normality-­‐poisoning	
  aGacks	
  
•  Ground	
  Truth	
  (labels)	
  >>	
  Features	
  >>	
  Algorithms	
  
•  There	
  is	
  no	
  (or	
  next	
  to	
  none)	
  Ground	
  Truth	
  in	
  AD	
  
•  What	
  is	
  “normal”	
  in	
  your	
  environment?	
  
•  Problem	
  asymmetry	
  
•  Solu2ons	
  are	
  biased	
  to	
  the	
  prevalent	
  class	
  
•  Very	
  hard	
  to	
  fine-­‐tune,	
  becomes	
  prone	
  to	
  a	
  lot	
  of	
  false	
  
nega2ves	
  or	
  false	
  posi2ves	
  
AD:	
  Normality-­‐poisoning	
  aGacks	
  
AD:	
  Hanlon’s	
  Razor	
  
Never attribute to malice
that which is adequately
explained by stupidity.
AD:	
  Hanlon’s	
  Razor	
  
vs!
Evil	
  Hacker!
Hipster	
  Developer	
  	
  
(a.k.a.	
  MaP	
  Johansen)!
What	
  about	
  User	
  Behavior?	
  
•  Surprise,	
  it	
  kinda	
  works!	
  (as	
  supervised,	
  that	
  is)	
  
•  As	
  specific	
  implementa2ons	
  for	
  specific	
  solu2ons	
  
•  Good	
  stuff	
  from	
  Square,	
  AirBnB	
  
•  Well	
  defined	
  scope	
  and	
  labeling.	
  
•  Can	
  it	
  be	
  general	
  enough?	
  
•  File	
  exfiltra2on	
  example	
  (roles/info	
  classifica2on	
  
are	
  mandatory?)	
  
•  Can	
  I	
  “average	
  out”	
  user	
  behaviors	
  in	
  different	
  
applica2ons?	
  
ClassificaCon!	
  
VS!
•  Lots	
  of	
  available	
  academic	
  research	
  around	
  this	
  
•  Classifica2on	
  and	
  clustering	
  of	
  malware	
  samples	
  
•  More	
  success	
  into	
  classifying	
  ar2facts	
  you	
  already	
  know	
  to	
  
be	
  malware	
  then	
  to	
  actually	
  detect	
  it.	
  (Lineage)	
  
•  State	
  of	
  the	
  art?	
  My	
  guess	
  is	
  AV	
  companies!	
  
•  All	
  of	
  them	
  have	
  an	
  absurd	
  amount	
  of	
  samples	
  
•  Have	
  been	
  researching	
  and	
  consolida2ng	
  data	
  on	
  them	
  
for	
  decades.	
  
Lots	
  of	
  Malware	
  AcCvity	
  
•  Can	
  we	
  do	
  bePer	
  than	
  “AV	
  Heuris2cs”?	
  
•  Lots	
  and	
  lots	
  of	
  available	
  data	
  that	
  has	
  been	
  made	
  public	
  
•  Some	
  of	
  the	
  papers	
  also	
  suffer	
  from	
  poten2ally	
  bad	
  ground	
  
truth.	
  
Lots	
  of	
  Malware	
  AcCvity	
  
VS!
Lots	
  of	
  Malware	
  AcCvity	
  
VS!
Everyone	
  makes	
  mistakes!	
  
•  Private	
  Beta	
  of	
  our	
  Threat	
  Intelligence-­‐based	
  models:	
  
•  Some	
  use	
  TI	
  indicator	
  feeds	
  as	
  blocklists	
  
•  More	
  mature	
  companies	
  use	
  the	
  feeds	
  to	
  learn	
  about	
  
the	
  threats	
  (Trained	
  professionals	
  only)	
  
•  Our	
  models	
  extrapolate	
  the	
  knowledge	
  of	
  exis2ng	
  threat	
  
intelligence	
  feeds	
  as	
  those	
  experienced	
  analysis	
  would.	
  
•  Supervised	
  model	
  w/same	
  data	
  analyst	
  has	
  
•  Seeded	
  labeling	
  from	
  TI	
  feeds	
  
How	
  is	
  it	
  going	
  then,	
  Alex?	
  
•  Very	
  effec2ve	
  first	
  triage	
  for	
  SOCs	
  and	
  Incident	
  Responders	
  
•  Send	
  us:	
  log	
  data	
  from	
  firewalls,	
  DNS,	
  web	
  proxies	
  
•  Receive:	
  Report	
  with	
  a	
  short	
  list	
  of	
  poten2al	
  
compromised	
  machines	
  
•  Would	
  you	
  rather	
  download	
  all	
  the	
  feeds	
  and	
  integrate	
  it	
  
yourself?	
  
•  MLSecProject/Combine	
  
•  MLSecProject/TIQ-­‐test	
  
	
  
Yeah,	
  but	
  why	
  should	
  I	
  care?	
  
•  Huge	
  amounts	
  of	
  TI	
  feeds	
  available	
  now	
  (open/commercial)	
  
•  Non-­‐malicious	
  samples	
  s2ll	
  challenging,	
  but	
  we	
  have	
  
expanded	
  to	
  a	
  lot	
  of	
  collec2on	
  techniques	
  from	
  different	
  
sources.	
  
•  Very	
  high-­‐ranked	
  Alexa	
  /	
  Quan2cast	
  /	
  OpenDNS	
  
Random	
  domains	
  as	
  seeds	
  for	
  search	
  of	
  trust	
  
•  Helped	
  by	
  the	
  customer	
  logs	
  as	
  well	
  in	
  a	
  semi-­‐
supervised	
  fashion	
  
What	
  about	
  the	
  Ground	
  Truth	
  
(labels)?	
  
•  Vast	
  majority	
  of	
  features	
  are	
  derived	
  from	
  structural/
intrinsic	
  data:	
  
•  GeoIP,	
  ASN	
  informa2on,	
  BGP	
  Prefixes	
  
•  pDNS	
  informa2on	
  for	
  the	
  IP	
  addresses,	
  hostnames	
  
•  WHOIS	
  informa2on	
  
•  APacker	
  can’t	
  change	
  those	
  things	
  without	
  cost.	
  
•  Log	
  data	
  from	
  the	
  customer,	
  can,	
  of	
  course.	
  But	
  this	
  does	
  
not	
  make	
  it	
  worse	
  than	
  human	
  specialist.	
  
But	
  what	
  about	
  data	
  tampering?	
  
•  False	
  posi2ves	
  /	
  false	
  nega2ves	
  are	
  an	
  intrinsic	
  part	
  of	
  ML.	
  
•  “False	
  posi2ves	
  are	
  very	
  good,	
  and	
  would	
  have	
  fooled	
  our	
  
human	
  analysts	
  at	
  first.”	
  
•  Their	
  feedback	
  helps	
  us	
  improve	
  the	
  models	
  for	
  everyone.	
  
•  Remember	
  it	
  is	
  about	
  ini2al	
  triage.	
  A	
  Tier-­‐2/Tier-­‐3	
  analyst	
  
must	
  inves2gate	
  and	
  provide	
  feedback	
  to	
  the	
  model.	
  
And	
  what	
  about	
  false	
  posiCves?	
  
•  1)	
  What	
  are	
  you	
  trying	
  to	
  achieve	
  with	
  adding	
  Machine	
  
Learning	
  to	
  the	
  solu2on?	
  
•  2)	
  What	
  are	
  the	
  sources	
  of	
  Ground	
  Truth	
  for	
  your	
  models?	
  
•  3)	
  How	
  can	
  you	
  protect	
  the	
  features	
  /	
  ground	
  truth	
  from	
  
adversaries?	
  
•  4)	
  How	
  does	
  the	
  solu2on/processes	
  around	
  it	
  handle	
  false	
  
posi2ves?	
  !
Buyer’s	
  Guide	
  
 
#NotAllAlgorithms!
Buyer’s	
  Guide	
  
MLSec	
  Project	
  
•  Don’t	
  take	
  my	
  word	
  for	
  it!	
  Try	
  it	
  out!!	
  
•  Help	
  us	
  test	
  and	
  improve	
  the	
  models!	
  
•  Looking	
  for	
  par2cipants	
  and	
  data	
  sharing	
  agreements	
  
•  Limited	
  capacity	
  at	
  the	
  moment,	
  so	
  be	
  pa2ent.	
  :)	
  
	
  
•  Visit	
  hGps://www.mlsecproject.org	
  ,	
  message	
  @MLSecProject	
  
or	
  just	
  e-­‐mail	
  me.!
Thanks!	
  
•  Q&A?	
  
•  Don’t	
  forget	
  the	
  feedback!	
  
Alex	
  Pinto	
  	
  
@alexcpsec	
  
@MLSecProject	
  
”We	
  are	
  drowning	
  on	
  informa2on	
  and	
  starved	
  for	
  knowledge"	
  	
  
	
   	
   	
   	
   	
  	
  	
   	
  	
  	
  -­‐	
  John	
  NaisbiP	
  	
  

More Related Content

What's hot

From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
Alex Pinto
 
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionBeyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Alex Pinto
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Alex Pinto
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Alex Pinto
 
2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies
2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies
2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies
Joshua R Nicholson
 
Databases, Web Services and Tools For Systems Immunology
Databases, Web Services and Tools For Systems ImmunologyDatabases, Web Services and Tools For Systems Immunology
Databases, Web Services and Tools For Systems Immunology
Yannick Pouliot
 
Software Analytics: Towards Software Mining that Matters
Software Analytics: Towards Software Mining that MattersSoftware Analytics: Towards Software Mining that Matters
Software Analytics: Towards Software Mining that Matters
Tao Xie
 
HotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William Enck
HotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William EnckHotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William Enck
HotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William Enck
Tao Xie
 
Software Analytics: Data Analytics for Software Engineering
Software Analytics: Data Analytics for Software EngineeringSoftware Analytics: Data Analytics for Software Engineering
Software Analytics: Data Analytics for Software Engineering
Tao Xie
 
Software Mining and Software Datasets
Software Mining and Software DatasetsSoftware Mining and Software Datasets
Software Mining and Software Datasets
Tao Xie
 
Filar seymour oreilly_bot_story_
Filar seymour oreilly_bot_story_Filar seymour oreilly_bot_story_
Filar seymour oreilly_bot_story_
EndgameInc
 
Luncheon 2016-07-16 - Topic 2 - Advanced Threat Hunting by Justin Falck
Luncheon 2016-07-16 -  Topic 2 - Advanced Threat Hunting by Justin FalckLuncheon 2016-07-16 -  Topic 2 - Advanced Threat Hunting by Justin Falck
Luncheon 2016-07-16 - Topic 2 - Advanced Threat Hunting by Justin Falck
North Texas Chapter of the ISSA
 
Big Data: the weakest link
Big Data: the weakest linkBig Data: the weakest link
Big Data: the weakest link
CS, NcState
 
SOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation LabyrinthSOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation Labyrinth
chrissanders88
 
User Expectations in Mobile App Security
User Expectations in Mobile App SecurityUser Expectations in Mobile App Security
User Expectations in Mobile App Security
Tao Xie
 
CarolinaCon Presentation on Streaming Analytics
CarolinaCon Presentation on Streaming AnalyticsCarolinaCon Presentation on Streaming Analytics
CarolinaCon Presentation on Streaming Analytics
John Eberhardt
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and Security
Tao Xie
 
Threat Hunting with Elastic at SpectorOps: Welcome to HELK
Threat Hunting with Elastic at SpectorOps: Welcome to HELKThreat Hunting with Elastic at SpectorOps: Welcome to HELK
Threat Hunting with Elastic at SpectorOps: Welcome to HELK
Elasticsearch
 
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
Databricks
 
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago
 

What's hot (20)

From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
From Threat Intelligence to Defense Cleverness: A Data Science Approach (#tid...
 
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based DetectionBeyond Matching: Applying Data Science Techniques to IOC-based Detection
Beyond Matching: Applying Data Science Techniques to IOC-based Detection
 
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013Applying Machine Learning to Network Security Monitoring - BayThreat 2013
Applying Machine Learning to Network Security Monitoring - BayThreat 2013
 
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
Data-Driven Threat Intelligence: Useful Methods and Measurements for Handling...
 
2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies
2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies
2016 FS-ISAC Annual Summit (Miami) - Developing Effective Encryption Strategies
 
Databases, Web Services and Tools For Systems Immunology
Databases, Web Services and Tools For Systems ImmunologyDatabases, Web Services and Tools For Systems Immunology
Databases, Web Services and Tools For Systems Immunology
 
Software Analytics: Towards Software Mining that Matters
Software Analytics: Towards Software Mining that MattersSoftware Analytics: Towards Software Mining that Matters
Software Analytics: Towards Software Mining that Matters
 
HotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William Enck
HotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William EnckHotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William Enck
HotSoS16 Tutorial "Text Analytics for Security" by Tao Xie and William Enck
 
Software Analytics: Data Analytics for Software Engineering
Software Analytics: Data Analytics for Software EngineeringSoftware Analytics: Data Analytics for Software Engineering
Software Analytics: Data Analytics for Software Engineering
 
Software Mining and Software Datasets
Software Mining and Software DatasetsSoftware Mining and Software Datasets
Software Mining and Software Datasets
 
Filar seymour oreilly_bot_story_
Filar seymour oreilly_bot_story_Filar seymour oreilly_bot_story_
Filar seymour oreilly_bot_story_
 
Luncheon 2016-07-16 - Topic 2 - Advanced Threat Hunting by Justin Falck
Luncheon 2016-07-16 -  Topic 2 - Advanced Threat Hunting by Justin FalckLuncheon 2016-07-16 -  Topic 2 - Advanced Threat Hunting by Justin Falck
Luncheon 2016-07-16 - Topic 2 - Advanced Threat Hunting by Justin Falck
 
Big Data: the weakest link
Big Data: the weakest linkBig Data: the weakest link
Big Data: the weakest link
 
SOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation LabyrinthSOC2016 - The Investigation Labyrinth
SOC2016 - The Investigation Labyrinth
 
User Expectations in Mobile App Security
User Expectations in Mobile App SecurityUser Expectations in Mobile App Security
User Expectations in Mobile App Security
 
CarolinaCon Presentation on Streaming Analytics
CarolinaCon Presentation on Streaming AnalyticsCarolinaCon Presentation on Streaming Analytics
CarolinaCon Presentation on Streaming Analytics
 
Software Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and SecuritySoftware Analytics: Data Analytics for Software Engineering and Security
Software Analytics: Data Analytics for Software Engineering and Security
 
Threat Hunting with Elastic at SpectorOps: Welcome to HELK
Threat Hunting with Elastic at SpectorOps: Welcome to HELKThreat Hunting with Elastic at SpectorOps: Welcome to HELK
Threat Hunting with Elastic at SpectorOps: Welcome to HELK
 
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
Deep Learning in Security—An Empirical Example in User and Entity Behavior An...
 
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
ACM Chicago March 2019 meeting: Software Engineering and AI - Prof. Tao Xie, ...
 

Viewers also liked

Machine Learning Essentials (dsth Meetup#3)
Machine Learning Essentials (dsth Meetup#3)Machine Learning Essentials (dsth Meetup#3)
Machine Learning Essentials (dsth Meetup#3)
Data Science Thailand
 
Boston Spark Meetup May 24, 2016
Boston Spark Meetup May 24, 2016Boston Spark Meetup May 24, 2016
Boston Spark Meetup May 24, 2016
Chris Fregly
 
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
Chris Fregly
 
陸永祥/全球網路攝影機帶來的機會與挑戰
陸永祥/全球網路攝影機帶來的機會與挑戰陸永祥/全球網路攝影機帶來的機會與挑戰
陸永祥/全球網路攝影機帶來的機會與挑戰
台灣資料科學年會
 
Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...
Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...
Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...
Chris Fregly
 
Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...
Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...
Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...
Chris Fregly
 
TensorFlow 深度學習快速上手班--電腦視覺應用
TensorFlow 深度學習快速上手班--電腦視覺應用TensorFlow 深度學習快速上手班--電腦視覺應用
TensorFlow 深度學習快速上手班--電腦視覺應用
Mark Chang
 
高嘉良/Open Innovation as Strategic Plan
高嘉良/Open Innovation as Strategic Plan高嘉良/Open Innovation as Strategic Plan
高嘉良/Open Innovation as Strategic Plan
台灣資料科學年會
 
Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...
Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...
Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...
Chris Fregly
 
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
Sri Ambati
 
Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...
Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...
Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...
Chris Fregly
 
02 math essentials
02 math essentials02 math essentials
02 math essentials
Poongodi Mano
 
Machine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine LearningMachine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine Learning
Arshad Ahmed
 
Machine Learning Preliminaries and Math Refresher
Machine Learning Preliminaries and Math RefresherMachine Learning Preliminaries and Math Refresher
Machine Learning Preliminaries and Math Refresher
butest
 
[系列活動] 資料探勘速遊
[系列活動] 資料探勘速遊[系列活動] 資料探勘速遊
[系列活動] 資料探勘速遊
台灣資料科學年會
 
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...
Chris Fregly
 
The Genome Assembly Problem
The Genome Assembly ProblemThe Genome Assembly Problem
The Genome Assembly Problem
Mark Chang
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
Mark Chang
 
DRAW: Deep Recurrent Attentive Writer
DRAW: Deep Recurrent Attentive WriterDRAW: Deep Recurrent Attentive Writer
DRAW: Deep Recurrent Attentive Writer
Mark Chang
 
NTHU AI Reading Group: Improved Training of Wasserstein GANs
NTHU AI Reading Group: Improved Training of Wasserstein GANsNTHU AI Reading Group: Improved Training of Wasserstein GANs
NTHU AI Reading Group: Improved Training of Wasserstein GANs
Mark Chang
 

Viewers also liked (20)

Machine Learning Essentials (dsth Meetup#3)
Machine Learning Essentials (dsth Meetup#3)Machine Learning Essentials (dsth Meetup#3)
Machine Learning Essentials (dsth Meetup#3)
 
Boston Spark Meetup May 24, 2016
Boston Spark Meetup May 24, 2016Boston Spark Meetup May 24, 2016
Boston Spark Meetup May 24, 2016
 
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
High Performance Distributed TensorFlow with GPUs - TensorFlow Chicago Meetup...
 
陸永祥/全球網路攝影機帶來的機會與挑戰
陸永祥/全球網路攝影機帶來的機會與挑戰陸永祥/全球網路攝影機帶來的機會與挑戰
陸永祥/全球網路攝影機帶來的機會與挑戰
 
Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...
Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...
Gradient Descent, Back Propagation, and Auto Differentiation - Advanced Spark...
 
Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...
Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...
Deploy Spark ML and Tensorflow AI Models from Notebooks to Microservices - No...
 
TensorFlow 深度學習快速上手班--電腦視覺應用
TensorFlow 深度學習快速上手班--電腦視覺應用TensorFlow 深度學習快速上手班--電腦視覺應用
TensorFlow 深度學習快速上手班--電腦視覺應用
 
高嘉良/Open Innovation as Strategic Plan
高嘉良/Open Innovation as Strategic Plan高嘉良/Open Innovation as Strategic Plan
高嘉良/Open Innovation as Strategic Plan
 
Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...
Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...
Big Data Spain - Nov 17 2016 - Madrid Continuously Deploy Spark ML and Tensor...
 
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
qconsf 2013: Top 10 Performance Gotchas for scaling in-memory Algorithms - Sr...
 
Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...
Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...
Kafka Summit SF Apr 26 2016 - Generating Real-time Recommendations with NiFi,...
 
02 math essentials
02 math essentials02 math essentials
02 math essentials
 
Machine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine LearningMachine Learning without the Math: An overview of Machine Learning
Machine Learning without the Math: An overview of Machine Learning
 
Machine Learning Preliminaries and Math Refresher
Machine Learning Preliminaries and Math RefresherMachine Learning Preliminaries and Math Refresher
Machine Learning Preliminaries and Math Refresher
 
[系列活動] 資料探勘速遊
[系列活動] 資料探勘速遊[系列活動] 資料探勘速遊
[系列活動] 資料探勘速遊
 
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...
Advanced Spark and TensorFlow Meetup 08-04-2016 One Click Spark ML Pipeline D...
 
The Genome Assembly Problem
The Genome Assembly ProblemThe Genome Assembly Problem
The Genome Assembly Problem
 
Generative Adversarial Networks
Generative Adversarial NetworksGenerative Adversarial Networks
Generative Adversarial Networks
 
DRAW: Deep Recurrent Attentive Writer
DRAW: Deep Recurrent Attentive WriterDRAW: Deep Recurrent Attentive Writer
DRAW: Deep Recurrent Attentive Writer
 
NTHU AI Reading Group: Improved Training of Wasserstein GANs
NTHU AI Reading Group: Improved Training of Wasserstein GANsNTHU AI Reading Group: Improved Training of Wasserstein GANs
NTHU AI Reading Group: Improved Training of Wasserstein GANs
 

Similar to Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#SecureBecauseMath)

From ensembles to computer networks
From ensembles to computer networksFrom ensembles to computer networks
From ensembles to computer networks
CSIRO
 
rsec2a-2016-jheaton-morning
rsec2a-2016-jheaton-morningrsec2a-2016-jheaton-morning
rsec2a-2016-jheaton-morning
Jeff Heaton
 
Delivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and VisualizationDelivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and Visualization
Raffael Marty
 
Genetic Malware
Genetic MalwareGenetic Malware
Genetic Malware
Okta
 
Genetic Malware
Genetic MalwareGenetic Malware
Genetic Malware
Lauren Sheppard
 
[PH-Neutral 0x7db] Exploit Next Generation®
[PH-Neutral 0x7db] Exploit Next Generation®[PH-Neutral 0x7db] Exploit Next Generation®
[PH-Neutral 0x7db] Exploit Next Generation®
Nelson Brito
 
Splunk September 2023 User Group PDX.pdf
Splunk September 2023 User Group PDX.pdfSplunk September 2023 User Group PDX.pdf
Splunk September 2023 User Group PDX.pdf
Amanda Richardson
 
2015 05-07 - vu amsterdam - testing safety critical systems
2015 05-07 - vu amsterdam - testing safety critical systems2015 05-07 - vu amsterdam - testing safety critical systems
2015 05-07 - vu amsterdam - testing safety critical systems
Jaap van Ekris
 
Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...
Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...
Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...
OpenDNS
 
Deep Learning AtoC with Image Perspective
Deep Learning AtoC with Image PerspectiveDeep Learning AtoC with Image Perspective
Deep Learning AtoC with Image Perspective
Dong Heon Cho
 
Keith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysisKeith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysis
Keith Jones, PhD
 
AI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat DetectionAI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat Detection
Databricks
 
михаил дударев
михаил дударевмихаил дударев
михаил дударев
apps4allru
 
Fix What Matters: A Data Driven Approach to Vulnerability Management
Fix What Matters: A Data Driven Approach to Vulnerability ManagementFix What Matters: A Data Driven Approach to Vulnerability Management
Fix What Matters: A Data Driven Approach to Vulnerability Management
Michael Roytman
 
Testing Safety Critical Systems (10-02-2014, VU amsterdam)
Testing Safety Critical Systems (10-02-2014, VU amsterdam)Testing Safety Critical Systems (10-02-2014, VU amsterdam)
Testing Safety Critical Systems (10-02-2014, VU amsterdam)
Jaap van Ekris
 
Now you see me, now you don't: chasing evasive malware - Giovanni Vigna
Now you see me, now you don't: chasing evasive malware - Giovanni Vigna Now you see me, now you don't: chasing evasive malware - Giovanni Vigna
Now you see me, now you don't: chasing evasive malware - Giovanni Vigna
Lastline, Inc.
 
2016-04-28 - VU Amsterdam - testing safety critical systems
2016-04-28 - VU Amsterdam - testing safety critical systems2016-04-28 - VU Amsterdam - testing safety critical systems
2016-04-28 - VU Amsterdam - testing safety critical systems
Jaap van Ekris
 
Building Your Application Security Data Hub - OWASP AppSecUSA
Building Your Application Security Data Hub - OWASP AppSecUSABuilding Your Application Security Data Hub - OWASP AppSecUSA
Building Your Application Security Data Hub - OWASP AppSecUSA
Denim Group
 
Watchtowers of the Internet - Source Boston 2012
Watchtowers of the Internet - Source Boston 2012Watchtowers of the Internet - Source Boston 2012
Watchtowers of the Internet - Source Boston 2012
Stephan Chenette
 
DITEC JAN 31 2015 (PDF)
DITEC JAN 31 2015 (PDF)DITEC JAN 31 2015 (PDF)
DITEC JAN 31 2015 (PDF)
Dr Robert D. Childs
 

Similar to Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#SecureBecauseMath) (20)

From ensembles to computer networks
From ensembles to computer networksFrom ensembles to computer networks
From ensembles to computer networks
 
rsec2a-2016-jheaton-morning
rsec2a-2016-jheaton-morningrsec2a-2016-jheaton-morning
rsec2a-2016-jheaton-morning
 
Delivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and VisualizationDelivering Security Insights with Data Analytics and Visualization
Delivering Security Insights with Data Analytics and Visualization
 
Genetic Malware
Genetic MalwareGenetic Malware
Genetic Malware
 
Genetic Malware
Genetic MalwareGenetic Malware
Genetic Malware
 
[PH-Neutral 0x7db] Exploit Next Generation®
[PH-Neutral 0x7db] Exploit Next Generation®[PH-Neutral 0x7db] Exploit Next Generation®
[PH-Neutral 0x7db] Exploit Next Generation®
 
Splunk September 2023 User Group PDX.pdf
Splunk September 2023 User Group PDX.pdfSplunk September 2023 User Group PDX.pdf
Splunk September 2023 User Group PDX.pdf
 
2015 05-07 - vu amsterdam - testing safety critical systems
2015 05-07 - vu amsterdam - testing safety critical systems2015 05-07 - vu amsterdam - testing safety critical systems
2015 05-07 - vu amsterdam - testing safety critical systems
 
Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...
Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...
Speak Security: Under the Hood of the OpenDNS Security Research Labs with Dhi...
 
Deep Learning AtoC with Image Perspective
Deep Learning AtoC with Image PerspectiveDeep Learning AtoC with Image Perspective
Deep Learning AtoC with Image Perspective
 
Keith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysisKeith J. Jones, Ph.D. - Crash Course malware analysis
Keith J. Jones, Ph.D. - Crash Course malware analysis
 
AI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat DetectionAI on Spark for Malware Analysis and Anomalous Threat Detection
AI on Spark for Malware Analysis and Anomalous Threat Detection
 
михаил дударев
михаил дударевмихаил дударев
михаил дударев
 
Fix What Matters: A Data Driven Approach to Vulnerability Management
Fix What Matters: A Data Driven Approach to Vulnerability ManagementFix What Matters: A Data Driven Approach to Vulnerability Management
Fix What Matters: A Data Driven Approach to Vulnerability Management
 
Testing Safety Critical Systems (10-02-2014, VU amsterdam)
Testing Safety Critical Systems (10-02-2014, VU amsterdam)Testing Safety Critical Systems (10-02-2014, VU amsterdam)
Testing Safety Critical Systems (10-02-2014, VU amsterdam)
 
Now you see me, now you don't: chasing evasive malware - Giovanni Vigna
Now you see me, now you don't: chasing evasive malware - Giovanni Vigna Now you see me, now you don't: chasing evasive malware - Giovanni Vigna
Now you see me, now you don't: chasing evasive malware - Giovanni Vigna
 
2016-04-28 - VU Amsterdam - testing safety critical systems
2016-04-28 - VU Amsterdam - testing safety critical systems2016-04-28 - VU Amsterdam - testing safety critical systems
2016-04-28 - VU Amsterdam - testing safety critical systems
 
Building Your Application Security Data Hub - OWASP AppSecUSA
Building Your Application Security Data Hub - OWASP AppSecUSABuilding Your Application Security Data Hub - OWASP AppSecUSA
Building Your Application Security Data Hub - OWASP AppSecUSA
 
Watchtowers of the Internet - Source Boston 2012
Watchtowers of the Internet - Source Boston 2012Watchtowers of the Internet - Source Boston 2012
Watchtowers of the Internet - Source Boston 2012
 
DITEC JAN 31 2015 (PDF)
DITEC JAN 31 2015 (PDF)DITEC JAN 31 2015 (PDF)
DITEC JAN 31 2015 (PDF)
 

Recently uploaded

Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
TeukuEriSyahputra
 
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
Timothy Spann
 
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
Rebecca Bilbro
 
06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus
Timothy Spann
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
hqfek
 
High Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENT
High Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENTHigh Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENT
High Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENT
ranjeet3341
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
keesa2
 
CAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdfCAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdf
frp60658
 
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdfNamma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
22ad0301
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
Vietnam Cotton & Spinning Association
 
Senior Engineering Sample EM DOE - Sheet1.pdf
Senior Engineering Sample EM DOE  - Sheet1.pdfSenior Engineering Sample EM DOE  - Sheet1.pdf
Senior Engineering Sample EM DOE - Sheet1.pdf
Vineet
 
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
PsychoTech Services
 
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
nitachopra
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
ytypuem
 
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service LucknowCall Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
hiju9823
 
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
actyx
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
GeorgiiSteshenko
 
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
Call Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call GirlCall Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call Girl
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
sapna sharmap11
 
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
ywqeos
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
eudsoh
 

Recently uploaded (20)

Template xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptxTemplate xxxxxxxx ssssssssssss Sertifikat.pptx
Template xxxxxxxx ssssssssssss Sertifikat.pptx
 
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
06-20-2024-AI Camp Meetup-Unstructured Data and Vector Databases
 
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
PyData London 2024: Mistakes were made (Dr. Rebecca Bilbro)
 
06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus06-18-2024-Princeton Meetup-Introduction to Milvus
06-18-2024-Princeton Meetup-Introduction to Milvus
 
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
一比一原版爱尔兰都柏林大学毕业证(本硕)ucd学位证书如何办理
 
High Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENT
High Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENTHigh Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENT
High Profile Call Girls Navi Mumbai ✅ 9833363713 FULL CASH PAYMENT
 
一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理一比一原版悉尼大学毕业证如何办理
一比一原版悉尼大学毕业证如何办理
 
CAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdfCAP Excel Formulas & Functions July - Copy (4).pdf
CAP Excel Formulas & Functions July - Copy (4).pdf
 
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdfNamma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
Namma-Kalvi-11th-Physics-Study-Material-Unit-1-EM-221086.pdf
 
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
[VCOSA] Monthly Report - Cotton & Yarn Statistics May 2024
 
Senior Engineering Sample EM DOE - Sheet1.pdf
Senior Engineering Sample EM DOE  - Sheet1.pdfSenior Engineering Sample EM DOE  - Sheet1.pdf
Senior Engineering Sample EM DOE - Sheet1.pdf
 
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
Interview Methods - Marital and Family Therapy and Counselling - Psychology S...
 
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
Call Girls Goa👉9024918724👉Low Rate Escorts in Goa 💃 Available 24/7
 
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
一比一原版(曼大毕业证书)曼尼托巴大学毕业证如何办理
 
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service LucknowCall Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
Call Girls Lucknow 0000000000 Independent Call Girl Service Lucknow
 
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
一比一原版斯威本理工大学毕业证(swinburne毕业证)如何办理
 
Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)Telemetry Solution for Gaming (AWS Summit'24)
Telemetry Solution for Gaming (AWS Summit'24)
 
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
Call Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call GirlCall Girls Hyderabad  (india) ☎️ +91-7426014248 Hyderabad  Call Girl
Call Girls Hyderabad (india) ☎️ +91-7426014248 Hyderabad Call Girl
 
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
一比一原版(lbs毕业证书)伦敦商学院毕业证如何办理
 
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
一比一原版马来西亚博特拉大学毕业证(upm毕业证)如何办理
 

Secure Because Math: A Deep-Dive on Machine Learning-Based Monitoring (#SecureBecauseMath)

  • 1. Secure  Because  Math:  A  Deep-­‐Dive  on   Machine  Learning-­‐Based  Monitoring     (#SecureBecauseMath)   Alex  Pinto   Chief  Data  Scien2st  |  MLSec  Project     @alexcpsec   @MLSecProject!
  • 2. Alex  Pinto   •  Chief  Data  Scien2st  at  MLSec  Project   •  Machine  Learning  Researcher  and  Trainer   •  Network  security  and  incident  response  aficionado     •  Tortured  by  SIEMs  as  a  child   •  Hacker  Spirit  Animal™:  CAFFEINATED  CAPYBARA! whoami   (hPps://secure.flickr.com/photos/kobashi_san/)  
  • 3.   •  Security  Singularity   •  Some  History   •  TLA   •  ML  Marke2ng  PaPerns   •  Anomaly  Detec2on   •  Classifica2on   •  Buyer’s  Guide   •  MLSec  Project   Agenda  
  • 5. (Side  Note)   First  hit  on  Google  images  for  “Network  Security  Solved”  is  a   picture  of  Jack  Daniel!
  • 6. Security  Singularity  Approaches   •  “Machine  learning  /  math  /  algorithms…  these  terms  are   used  interchangeably  quite  frequently.”   •  “Is  behavioral  baselining  and  anomaly  detec2on  part  of   this?”   •  “What  about  Big  Data  Security  Analy2cs?”     (hPp://bigdatapix.tumblr.com/)  
  • 7. Are  we  even  trying?   •  “Hyper-­‐dimensional  security   analy2cs”   •  “3rd  genera2on  Ar2ficial   Intelligence”   •  “Secure  because  Math”     •  Lack  of  ability  to  differen2ate   hurts  buyers,  investors.   •  Are  we  even  funding  the  right   things?  
  • 8. Is  this  a  communicaCon  issue?  
  • 9. Guess  the  Year!   •  “(…)  behavior  analysis  system  that  enhances  your   network  intelligence  and  security  by  audi2ng  network   flow  data  from  exis2ng  infrastructure  devices”   •  "Mathema2cal  models  (…)  that  determine  baseline   behavior  across  users  and  machines,  detec2ng  (...)   anomalous  and  risky  ac2vi2es  (...)”   •  ”(…)  maintains  historical  profiles  of  usage  per  user  and   raises  an  alarm  when  observed  ac2vity  departs  from   established  paPerns  of  usage  for  an  individual.”    
  • 10. A  liGle  history   •  Dorothy  E.  Denning  (professor  at  the   Department  of  Defense  Analysis  at  the   Naval  Postgraduate  School)   •  1986  (SRI)  -­‐  First  research  that  led   to  IDS   •  Intrusion  Detec2on  Expert  System   (IDES)   •  Already  had  sta2s2cal  anomaly   detec2on  built-­‐in   •  1993:  Her  colleagues  release  the  Next   Genera2on  (!)  IDES  
  • 11. Three  LeGer  Acronyms  -­‐  KDD   •  Ajer  the  release  of  Bro  (1998)  and  Snort  (1999),  DARPA   thought  we  were  covered  for  this  signature  thing   •  DARPA  released  datasets  for  user  anomaly  detec2on  in   1998  and  1999   •  And  then  came  the  KDD-­‐99  dataset  –  over  6200  cita2ons   on  Google  Scholar  
  • 12.
  • 14. Three  LeGer  Acronyms  -­‐  KDD  
  • 16. Not  here  to  bash  academia  
  • 17. A  Probable  Outcome   GRAD   SCHOOL   FRESHMAN   ZOMG   RESULTS  !! 11!1!   ZOMG!   RESULTS???   MATH,  STAHP!   MATH  IS   HARD,  LET’S   GO  SHOPPING  
  • 18. ML  MarkeCng  PaGerns   •  The  “Has-­‐beens”     •  Name  is  a  bit  harsh,  but  hey,  you  hardly  use  ML   anymore,  let  us  try  it   •  The  “Machine  Learning  ¯ˉ_(ツ)_/¯ˉ”   •  Hey,  that  sounds  cool,  let’s  put  that  in  our  brochure   •  The  “Sweet  Spot”   •  People  that  actually  are  trying  to  do  something   •  Anomaly  Detec2on  vs.  Classifica2on  
  • 20. Anomaly  DetecCon   •  Works  wonders  for  well   defined  “industrial-­‐like”   processes.   •  Looking  at  single,   consistently  measured   variables   •  Historical  usage  in  financial   fraud  preven2on.  
  • 22. Anomaly  DetecCon   •  What  fits  this  mold?   •  Network/Neqlow  behavior  analysis     •  User  behavior  analysis   •  What  are  the  challenges?   •  Curse  of  Dimensionality   •  Lack  of  ground  truth  and  normality  poisoning   •  Hanlon’s  Razor  
  • 23. AD:  Curse  of  Dimensionality   •  We  need  “distances”  to  measure   the  features/variables   •  Usually  ManhaPan  or  Euclidian   •  For  high-­‐dimensional  data,  the   distribu2on  of  distances  between   all  pairwise  points  in  the  space   becomes  concentrated  around  an   average  distance.  
  • 24. AD:  Curse  of  Dimensionality   •  The  volume  of  the  high   dimensional  sphere   becomes  negligible  in   rela2on  to  the  volume  of   the  high  dimensional  cube.   •  The  prac2cal  result  is  that   everything  just  seems  too   far  away,  and  at  similar   distances.   (hPp://www.datasciencecentral.com/m/blogpost? id=6448529%3ABlogPost%3A175670)  
  • 25. A  PracCcal  example   •  NetFlow  data,  company  with  n  internal  nodes.   •  2(nˆ2  -­‐  n)  communica2on  direc2ons   •  2*2*2*65535(nˆ2  -­‐  n)  measures  of  network  ac2vity   •  1000  nodes  -­‐>  Half  a  trillion  possible  dimensions  
  • 26. Breaking  the  Curse   •  Different  /  crea2ve   distance  metrics   •  Organizing  the  space  into   sub-­‐manifolds  where   Euclidean  distances  make   more  sense.   •  Aggressive  feature   removal   •  A  few  interes2ng  results   available  
  • 28. AD:  Normality-­‐poisoning  aGacks   •  Ground  Truth  (labels)  >>  Features  >>  Algorithms   •  There  is  no  (or  next  to  none)  Ground  Truth  in  AD   •  What  is  “normal”  in  your  environment?   •  Problem  asymmetry   •  Solu2ons  are  biased  to  the  prevalent  class   •  Very  hard  to  fine-­‐tune,  becomes  prone  to  a  lot  of  false   nega2ves  or  false  posi2ves  
  • 30. AD:  Hanlon’s  Razor   Never attribute to malice that which is adequately explained by stupidity.
  • 31. AD:  Hanlon’s  Razor   vs! Evil  Hacker! Hipster  Developer     (a.k.a.  MaP  Johansen)!
  • 32. What  about  User  Behavior?   •  Surprise,  it  kinda  works!  (as  supervised,  that  is)   •  As  specific  implementa2ons  for  specific  solu2ons   •  Good  stuff  from  Square,  AirBnB   •  Well  defined  scope  and  labeling.   •  Can  it  be  general  enough?   •  File  exfiltra2on  example  (roles/info  classifica2on   are  mandatory?)   •  Can  I  “average  out”  user  behaviors  in  different   applica2ons?  
  • 34. •  Lots  of  available  academic  research  around  this   •  Classifica2on  and  clustering  of  malware  samples   •  More  success  into  classifying  ar2facts  you  already  know  to   be  malware  then  to  actually  detect  it.  (Lineage)   •  State  of  the  art?  My  guess  is  AV  companies!   •  All  of  them  have  an  absurd  amount  of  samples   •  Have  been  researching  and  consolida2ng  data  on  them   for  decades.   Lots  of  Malware  AcCvity  
  • 35. •  Can  we  do  bePer  than  “AV  Heuris2cs”?   •  Lots  and  lots  of  available  data  that  has  been  made  public   •  Some  of  the  papers  also  suffer  from  poten2ally  bad  ground   truth.   Lots  of  Malware  AcCvity   VS!
  • 36. Lots  of  Malware  AcCvity   VS!
  • 38. •  Private  Beta  of  our  Threat  Intelligence-­‐based  models:   •  Some  use  TI  indicator  feeds  as  blocklists   •  More  mature  companies  use  the  feeds  to  learn  about   the  threats  (Trained  professionals  only)   •  Our  models  extrapolate  the  knowledge  of  exis2ng  threat   intelligence  feeds  as  those  experienced  analysis  would.   •  Supervised  model  w/same  data  analyst  has   •  Seeded  labeling  from  TI  feeds   How  is  it  going  then,  Alex?  
  • 39. •  Very  effec2ve  first  triage  for  SOCs  and  Incident  Responders   •  Send  us:  log  data  from  firewalls,  DNS,  web  proxies   •  Receive:  Report  with  a  short  list  of  poten2al   compromised  machines   •  Would  you  rather  download  all  the  feeds  and  integrate  it   yourself?   •  MLSecProject/Combine   •  MLSecProject/TIQ-­‐test     Yeah,  but  why  should  I  care?  
  • 40. •  Huge  amounts  of  TI  feeds  available  now  (open/commercial)   •  Non-­‐malicious  samples  s2ll  challenging,  but  we  have   expanded  to  a  lot  of  collec2on  techniques  from  different   sources.   •  Very  high-­‐ranked  Alexa  /  Quan2cast  /  OpenDNS   Random  domains  as  seeds  for  search  of  trust   •  Helped  by  the  customer  logs  as  well  in  a  semi-­‐ supervised  fashion   What  about  the  Ground  Truth   (labels)?  
  • 41. •  Vast  majority  of  features  are  derived  from  structural/ intrinsic  data:   •  GeoIP,  ASN  informa2on,  BGP  Prefixes   •  pDNS  informa2on  for  the  IP  addresses,  hostnames   •  WHOIS  informa2on   •  APacker  can’t  change  those  things  without  cost.   •  Log  data  from  the  customer,  can,  of  course.  But  this  does   not  make  it  worse  than  human  specialist.   But  what  about  data  tampering?  
  • 42. •  False  posi2ves  /  false  nega2ves  are  an  intrinsic  part  of  ML.   •  “False  posi2ves  are  very  good,  and  would  have  fooled  our   human  analysts  at  first.”   •  Their  feedback  helps  us  improve  the  models  for  everyone.   •  Remember  it  is  about  ini2al  triage.  A  Tier-­‐2/Tier-­‐3  analyst   must  inves2gate  and  provide  feedback  to  the  model.   And  what  about  false  posiCves?  
  • 43. •  1)  What  are  you  trying  to  achieve  with  adding  Machine   Learning  to  the  solu2on?   •  2)  What  are  the  sources  of  Ground  Truth  for  your  models?   •  3)  How  can  you  protect  the  features  /  ground  truth  from   adversaries?   •  4)  How  does  the  solu2on/processes  around  it  handle  false   posi2ves?  ! Buyer’s  Guide  
  • 45. MLSec  Project   •  Don’t  take  my  word  for  it!  Try  it  out!!   •  Help  us  test  and  improve  the  models!   •  Looking  for  par2cipants  and  data  sharing  agreements   •  Limited  capacity  at  the  moment,  so  be  pa2ent.  :)     •  Visit  hGps://www.mlsecproject.org  ,  message  @MLSecProject   or  just  e-­‐mail  me.!
  • 46. Thanks!   •  Q&A?   •  Don’t  forget  the  feedback!   Alex  Pinto     @alexcpsec   @MLSecProject   ”We  are  drowning  on  informa2on  and  starved  for  knowledge"                        -­‐  John  NaisbiP