AGILIS: an on-line map reduce environmentfor collaborative security<br />MIDLAB<br />Middleware Laboratory<br />Sapienza ...
Middleware Laboratory<br />MIDLAB<br />Focus and structure of the talk<br /> Requirements coming from the financial contex...
MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />Th...
Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br />"webification"...
Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br /> A payment car...
Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br />Distributed De...
Middleware Laboratory<br />MIDLAB<br />Economicsof a DDOS<br /><ul><li>render internet-based financial services unreachabl...
Use of Botnets (rented now with a credit card in a few minutes)
Three examples of DDOS campaign in Cyberwarfare:
Estonia 2007
Georgia 2008
Iran (in progress!). Stuxnet worm invaded Iran’s Supervisory Control and Data Acquisition systems</li></ul>McAfee report 2...
Middleware Laboratory<br />MIDLAB<br />Economicsof a DDOS<br /><ul><li>cost of downtime from major attacks exceeds U.S. $6...
damage to reputation
loss of personal information about customers
one out of five DDos attacks  is accompanied with an extorsion</li></ul>McAfee report 2010 “in the crossfire: criticalinfr...
Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br />Both previous ...
Middleware Laboratory<br />MIDLAB<br />Barriers to Collaboration<br /><ul><li>Barriers to collaboration
Understanding the economics
Trust
Legal Issues</li></ul>LLYODS<br />France Telecom<br />UBS<br />Internet<br />AT&T<br />SWIFT<br />Unicredit<br />EDF<br />...
MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />Co...
Middleware Laboratory<br />MIDLAB<br />Collaborative Cyber Security Platform<br />Monitoring and reaction to threats (MitM...
Middleware Laboratory<br />MIDLAB<br /><ul><li>Contract
set of processing and data sharing services provided by the SR along with the data protection, privacy, isolation, trust, ...
 The contractalsocontainsthe hardware and software requirements a member has to provision in order to be admitted into the...
Objective
each SR has a specic strategic objective to meet (e.g, large-scale stealthy scans detection, detecting Man-In-The-Middle a...
Deployment
highly flexible to accommodate the use of different technologies for the implementation of the processing and sharing with...
Middleware Laboratory<br />MIDLAB<br />The notion of semantic room: relationship with cloud computing<br /><ul><li>Private...
Deploymentof the semanticroomthrough the federationofcomputing and storagecapabilities at eachmember
Eachmemberbrings a private cloudto federate
Upcoming SlideShare
Loading in …5
×

AGILIS: an on-line map reduce environment for collaborative security

605 views

Published on

Published in: Technology, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
605
On SlideShare
0
From Embeds
0
Number of Embeds
10
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

AGILIS: an on-line map reduce environment for collaborative security

  1. 1. AGILIS: an on-line map reduce environmentfor collaborative security<br />MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />Roberto Baldoni<br />UniversitàdegliStudidi Roma “La Sapienza”<br />baldoni@dis.uniroma1.it, http://www.dis.uniroma1.it/~baldoni/<br />Prin Meeting - San Vito diCadore<br />Joint Work with IBM Haifa<br />in the contextofCoMiFin EU Project<br />14/2/2011<br />
  2. 2. Middleware Laboratory<br />MIDLAB<br />Focus and structure of the talk<br /> Requirements coming from the financial context;<br />Collaborative event processing for Cyber Security <br />Edge vs centralized event processing over the internet<br />Agilis<br />Esper<br /> Roberto Baldoni<br />
  3. 3. MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />The case of the Financial Critical Infrastructure<br />
  4. 4. Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br />"webification" of critical financial services, such as home banking, online trading, remote payments;<br />Cross-domain interactions, spanning different organization boundaries are in place in financial contexts;<br />Heterogeneous infrastructure systems such as telecommunication supply, banking, and credit card companies working on heterogeneous data;<br /> Roberto Baldoni<br />
  5. 5. Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br /> A payment card fraud (2008)<br />100 compromised payment cards used by a network of coordinated attackers retrieving cash from 130 different ATMs in 49 countries worldwide, totaling 9 million of US dollars. <br />High degree of coordination, half an hour to be executed<br />evade all the local monitoring techniques used for detecting anomalies in payment card usage patterns. <br />The fraud has been detected only later, after aggregating all the information gathered locally by each financial institution involved in the payment card scam<br /> Roberto Baldoni<br />
  6. 6. Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br />Distributed Denial Of Service Attack (2007, Northern Europe)<br /> render web-based financial services unreachable from legitimate users. <br />DDoS attack targeted a credit card company and two DNS. <br />Internet restored only after several trial-and-error activities carried out manually by network administrators of the attacked systems and of their Internet Service Providers (ISPs).<br />Long preparation time (days), short attack time (seconds)<br />Roberto Baldoni<br />
  7. 7. Middleware Laboratory<br />MIDLAB<br />Economicsof a DDOS<br /><ul><li>render internet-based financial services unreachable from legitimate users.
  8. 8. Use of Botnets (rented now with a credit card in a few minutes)
  9. 9. Three examples of DDOS campaign in Cyberwarfare:
  10. 10. Estonia 2007
  11. 11. Georgia 2008
  12. 12. Iran (in progress!). Stuxnet worm invaded Iran’s Supervisory Control and Data Acquisition systems</li></ul>McAfee report 2010 “in the crossfire: criticalinfrastructures in the ageof cyber war “<br /> Roberto Baldoni<br />
  13. 13. Middleware Laboratory<br />MIDLAB<br />Economicsof a DDOS<br /><ul><li>cost of downtime from major attacks exceeds U.S. $6 million per day
  14. 14. damage to reputation
  15. 15. loss of personal information about customers
  16. 16. one out of five DDos attacks is accompanied with an extorsion</li></ul>McAfee report 2010 “in the crossfire: criticalinfrastructures in the ageof cyber war “<br /> Roberto Baldoni<br />
  17. 17. Middleware Laboratory<br />MIDLAB<br />The case of Collaborative Cyber Security in Financial Ecosystem<br />Both previous attacks cannot be detected quickly through information available at the IT infrastructure of a single financial player (i.e., using local monitoring)<br />Need of Information Sharing<br />Exchange non-sensitive status information <br />Set up of agreements<br />Advantages of a global monitoring system<br />Damage mitigation<br />Quick reaction<br />Roberto Baldoni<br />
  18. 18. Middleware Laboratory<br />MIDLAB<br />Barriers to Collaboration<br /><ul><li>Barriers to collaboration
  19. 19. Understanding the economics
  20. 20. Trust
  21. 21. Legal Issues</li></ul>LLYODS<br />France Telecom<br />UBS<br />Internet<br />AT&T<br />SWIFT<br />Unicredit<br />EDF<br />Events<br />warnings<br />Roberto Baldoni<br />
  22. 22. MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />Collaborative event Processing for cyber security: The CoMiFin Project<br />ApplicationLevel<br />CollaborationLevel<br />Internet level<br />
  23. 23. Middleware Laboratory<br />MIDLAB<br />Collaborative Cyber Security Platform<br />Monitoring and reaction to threats (MitM, Stealty Scan , Phishing, …)<br />Black/white lists distribution (for credit reputation, trust level, …) <br />Anti-terrorism lists (with name check VAS)<br />Anti money laundering monitoring<br />Risk management support<br /> Some Requirements on the platform<br /> uneven workload along the time <br /> High throughput<br /> high computational power<br /> Large storage capabilities<br /> Timeliness <br /> Roberto Baldoni<br />
  24. 24. Middleware Laboratory<br />MIDLAB<br /><ul><li>Contract
  25. 25. set of processing and data sharing services provided by the SR along with the data protection, privacy, isolation, trust, security, dependability, performance requirements.
  26. 26. The contractalsocontainsthe hardware and software requirements a member has to provision in order to be admitted into the SR.
  27. 27. Objective
  28. 28. each SR has a specic strategic objective to meet (e.g, large-scale stealthy scans detection, detecting Man-In-The-Middle attacks)
  29. 29. Deployment
  30. 30. highly flexible to accommodate the use of different technologies for the implementation of the processing and sharing within the SR (i.e., the implementation of the SR logic or functionality).</li></ul>Roberto Baldoni<br />The notion of semantic room<br />
  31. 31. Middleware Laboratory<br />MIDLAB<br />The notion of semantic room: relationship with cloud computing<br /><ul><li>Private cloud
  32. 32. Deploymentof the semanticroomthrough the federationofcomputing and storagecapabilities at eachmember
  33. 33. Eachmemberbrings a private cloudto federate
  34. 34. Public Cloud
  35. 35. Deploymentof the semanticroom on a third party cloud provider
  36. 36. The third party ownsallcomputing and storagecapabilities
  37. 37. Hybridapproach</li></ul>Application<br />Level<br />Collaboration<br />Level<br />Internet Level<br />Roberto Baldoni<br />
  38. 38. Middleware Laboratory<br />MIDLAB<br />Data Management problems in the semantic room<br /> Jurisdiction and regulation (Where and how will data be governed?)<br /> Ownership of Data (Who owns the data in the semantic room?)<br /> Data Portability<br />Data anonymization<br />Data Retention/Permanence (What happens to data over time?)<br />Security and Privacy (How is data secure and protected?)<br />Reliability, Liability and Quality of Service of the partner of the semantic room<br /> Government Surveillance (How much data can the government get from a semantic room?)<br />………………….<br />Roberto Baldoni<br />
  39. 39. Middleware Laboratory<br />MIDLAB<br />contract<br />A specific collaborative platform: CoMiFin Architecture<br />Roberto Baldoni<br />
  40. 40. Middleware Laboratory<br />MIDLAB<br /> IBM System S [ICDCS 06]<br /> high cost of ownership<br />Centralized data management <br />No cooperative approach<br />Cooperative Intrusion Detection Systems (e.g. Dshiels)<br />Correlation among local warnings <br /> High cost of ownership<br /> Obscure data management<br />Related work<br /> Roberto Baldoni<br />
  41. 41. MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />Preventing Stealthy Scan Through centralized processing<br />ApplicationLevel<br />CollaborationLevel<br />Internet level<br />
  42. 42. Middleware Laboratory<br />MIDLAB<br />Collaborative Stealthy scan<br /> Attacker performs port scanning simultaneously at multiple sites trying to identify TCP/UDP ports that have been left open. Those ports can then be used as the attack vectors<br />Added value of collaboration: <br />Ability to identify an attacker trying to conceal his/her activity by accessing only a small number of ports within each individual domain<br />Action taken: <br />black list IP addresses <br />update historical records<br /> Roberto Baldoni<br />
  43. 43. Middleware Laboratory<br />MIDLAB<br />Collaborative Stealthy scan detection<br />Attack subjects: <br /><ul><li>External web servers in DMZ’s of the SR members</li></ul>Pattern: <br /><ul><li>“Unusually” high number of requests
  44. 44. Originating from a particular source IP address, and
  45. 45. Directed to distinct (machine, port) pairs</li></ul>Action taken:<br /><ul><li>Matching source IP’s are banned from the future access to external web servers</li></ul>Defining the attack<br /><ul><li>Use of common scanning tools (nmap)
  46. 46. Use of real trace (e.g., ITOC US Army)</li></ul> Roberto Baldoni<br />
  47. 47. Middleware Laboratory<br />MIDLAB<br />Collaborative Stealthy scan detection<br /><ul><li>Rank-Syn algorithm.
  48. 48. Analyze the sequence of SYN, ACK, RST packets in the three-way TCP handshake. Specifically, in normal activities the following sequence is verified (i) SYN, (ii) SYN-ACK, (iii) ACK.
  49. 49. In the presence of a SYN port scan, the connection looks like the following: (i) SYN, (ii) SYN-ACK, (iii) RST (or nothing)
  50. 50. For a given IP address, if the number of incomplete connections is higher than a certain threshold T, we can conclude that the IP address is likely carrying out malicious port scanning activities.</li></ul> Roberto Baldoni<br />
  51. 51. Middleware Laboratory<br />MIDLAB<br />Raw data: TCPdump<br />10:53:14.647181 IP 9.148.30.136.pop3 > 9.148.17.85.madcap: R 0:0(0) ack 1 win 0<br />10:53:14.653813 IP 9.148.17.85.sis-emt > 9.148.30.136.xfer: S 268426387:268426387(0) win 65535 <mss 1460,nop,nop,sackOK><br />10:53:14.653817 IP 9.148.30.136.xfer > 9.148.17.85.sis-emt: R 0:0(0) ack 268426388 win 0<br />Normalized data format:<br /><ul><li>LogEvent: sourceIp, destIp,sourcePort, destPort, startTime, endTime, bytesSent, bytesRecieved, returnStatus;</li></ul>Online data summary<br />Historical Data Format<br />BlackList: List of IP addresses<br />Stealthy scan detection<br /> Roberto Baldoni<br />
  52. 52. Example of semantic room for stealthy scan: Ingredients<br />EPL Query<br />EPL Query<br />EPL Query<br />EPL Query<br />EPL Query<br />Subscriber<br />Middleware Laboratory<br />MIDLAB<br />Branch j<br />Branch 1<br />Esper CEP Engine<br />Gateway<br />POJOs<br />I/O socket<br />I/O socket<br />adapter<br />Input Streams<br />sniffer<br />Main Engine<br />...<br />Output Streams<br />Branch N<br />POJOs<br />Gateway<br />I/O socket<br />suspected IPs<br />adapter<br />Scanner list<br />sniffer<br /> Roberto Baldoni<br />
  53. 53. Middleware Laboratory<br />MIDLAB<br />Example of semantic room for stealthy scan: Ingredients<br /> Roberto Baldoni<br />
  54. 54. Middleware Laboratory<br />MIDLAB<br />Testbed: latency measurement<br />Two Semantic rooms<br /><ul><li>Man-In-The-Middle Attack
  55. 55. Stealthy Scan</li></ul>Roberto Baldoni<br />
  56. 56. MIDLAB<br />Middleware Laboratory<br />Sapienza Università di Roma<br />Dipartimento di Informatica e Sistemistica<br />Preventing Stealthy Scan through edge processing<br />ApplicationLevel<br />CollaborationLevel<br />Internet level<br />
  57. 57. Middleware Laboratory<br />MIDLAB<br />Example of semantic room for stealthy scan: Ingredients<br /><ul><li>WebSphereeXtreme Scale (WXS): in-memory distributed storage
  58. 58. High-level language for processing logic: Jaql (SQL-like, supports flows)
  59. 59. Distributed processing runtime: MapReduce
  60. 60. Distributed file system for long-term storage: HDFS
  61. 61. Agilis consists of a distributed network of processing and storage elements hosted on a cluster of machines (also geographycally dispersed)</li></ul> Roberto Baldoni<br />
  62. 62. Middleware Laboratory<br />MIDLAB<br />Data Dissemination: Agilis<br />Re-define InputFormat, OutputFormat<br />TaskTracker<br />TaskTracker<br />Job Tracker<br />Jaql query<br />HDFS Adapter<br />WXS Adapter<br />Map-Reduce (Hadoop)<br />Jaql<br />Interpreter<br />Gateway<br />TaskTracker<br />Cat 1<br />Cat 2<br />Distributed In-Memory Store (WXS)<br />Storage container<br />AGILIS<br />Jaql Adapter<br />Storage container<br />Distributed File System (HDFS)<br /> Roberto Baldoni<br />
  63. 63. Middleware Laboratory<br />MIDLAB<br />Example of semantic room for stealthy scan: architecture<br /> Roberto Baldoni<br />
  64. 64. Middleware Laboratory<br />MIDLAB<br />Collaborative Stealthy scan detection with Agilis<br />Detection ofstealtyscan<br /> Roberto Baldoni<br />
  65. 65. Middleware Laboratory<br />MIDLAB<br />Demo: Done at Haifa IBM Research LAB (2009)<br />Simple and homemade attacks<br /> artificial traces<br />Simple stealty scan detection algorithm<br />8 Linux Machines on a LAN, each of which with <br />2GB of RAM and <br />20GB of disk space<br />One machine was hosting all the management processes (JT, XS Catalogue)<br />Each of the remaining 7 hosts modeled a single SR participant <br />DMZ web server under attack<br />TT and XS data server<br />Scenarios: <br />Single intruding host that generated a series of TCP/SYN requests targeting a fixed set of 300 unique ports on each the 7 attacked servers<br />requests injected at constant rate of 10, 20, and 30 req/server/sec<br />ratio of attack to legitimate traffic 1:5<br />blacklisting threshold: 20,000 requests and 1000 unique port<br />processing window: 4 minutes<br />Results:<br />No overload<br />Detection latency 700 sec, 430 sec, 330 sec<br /> Roberto Baldoni<br />
  66. 66. Middleware Laboratory<br />MIDLAB<br />Demo: work we are doing in our LAB (2010)<br />The video shows a semantic room implemented in Agilis using<br /><ul><li>Nmap for producing the attack
  67. 67. Real TCP dumps</li></ul>Joint work with Giorgia Lodi and Leonardo Aniello<br /> Roberto Baldoni<br />
  68. 68. Middleware Laboratory<br />MIDLAB<br />Testbed: latency measurement<br />Two Semantic rooms<br /><ul><li>Man-In-The-Middle Attack
  69. 69. Stealthy Scan</li></ul>Roberto Baldoni<br />

×