Your SlideShare is downloading. ×
SIPColumbiaSymposium..
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

SIPColumbiaSymposium..

386
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
386
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • VoIP cannot be fully deployable without solving problem that surround . And it is very important for Verizon’s reputation that they deploy a solution that is completely devoid of outages that DoS attacks can potentially cause. Such an outage wouldn’t mean much to people like Vonage or Skype but it definitely means a lot to Verizon.
  • S. Springs labs is the best home since systems necessary for testing are already there, no capital needed, and staff is close by. The Risk Assessment is really a business tool to help prioritize what should be done and in what order based on the anticipated risks to the business over time. Other test tools (Codenomicon, Agilent, etc.)
  • Verizon has made a strategic decision to proceed with all future deployments of VoIP to be SIP based From Research Focus Slide: Two detection and mitigation filters - SIP: Two types of rule-based detection and mitigation filters - Media: SIP-aware dynamic pinhole filtering CloudShield fast packet application server Developed a Testing & Analysis infrastructure Distributed computing architecture for traffic generation & simulation Testing & Analysis tools - ToS Architectural Integrity Verification Tool - Evaluation at carrier-class rates suitable for Verizon networks future deployments
  • We don’t need to re-invent the wheel. VoIP Security Alliance lead by Prof. Schulzrinne has lead down the VoIP threat model which we adopt here.
  • A multifaceted problem focusing on VoIP specific DoS Motivation From previous slides Application layer security Digest Authentication, TLS, S/MIME, IPSec, certificates SRTP/ZRTP for media Convergence leads to converged attacks Data network attacks DDoS, spoofing, content alteration, platform attacks Voice over IP network attacks Toll fraud, session hijacking, theft of service, spam/spit Most security problems are due to User Datagram Protocol (UDP) instead of TCP/TLS Plain text instead of S/MIME Message/Method vulnerability Flexible grammar --> syntax-based attacks Application level security is already defined by SIP RFC 3261, an improvement from relatively less secure RFC 2543. Although they are good ways to prevent outright compromise of network, we need to extend our security architecture to deal with attacks like: Call fraud, compromised machines, theft of service, identity assurance and more. VoIP has converged advantages from various networks together: cost effectiveness of data networks and convenience of voice networks. However this convergence has also made VoIP susceptible to data network attacks like DDoS, Message Alteration, Spoofing, Session anomalies and also voice attacks like Call Theft, Call Hijacking, Spam. UDP – Spoofing; Plain Text – Spam, Flood of messages (although all SIP networks are not connected yet); Message Vulnerability – Network downtime, alteration, etc. Application Layer Security SIP RFC 2543 – little security SIP RFC 3261 – security enhancements Digest Authentication TLS IPSec SRTP/ZRTP Perimeter Protection SIP aware Filtering Mechanisms SIP aware DOS Protection Detection and Mitigation
  • Explain DoS and DDoS. It is estimated that there are 600 million computers on Internet. Out of which 100-150 million are expected to be infected. DoS Implementation flaws Application level Flooding TOS: Billing Threats Unauthorized deletion or altering of billing records Unauthorized bypass of lawful billing systems Unauthorized billing Service Threats Enough funds Enough coverage Physical Threats Taking of service provider property DoS Implementation flaws Application level Flooding
  • UDP floods, SYN attacks can be protected by other products in the market. I.e. Arbor Networks, Netscreen, Cisco/Riverhead Technologies For sip there is no solution and this is where we come, It’s like “peeling the onion” Implementation flaws are dealt with Oulu University test tool such as PROTOS or better their commercial progeny Codenomicon.
  • Explain: ASM, DPPM, CAM, Regular Expression engine and Rapid Application and Visualization Engine language.
  • NOTE: Match earlier stuff on ToS with this slide and try to merge them While most billing issues can be effectively dealt with authentication and authorization, it is difficult for network to deal with compromised systems. As we are progressing, synonymous to developments in GSM phone, we will soon witness executable softwares (JVM) on voip phones and no authentication, authorization or encryption will be able to prevent compromised systems. The immediate solution to this problem is to analyze patterns to discover anomaly in the network usage and curb theft using turing test. Call spoofing and vishing not only will leave customers absolutely unsatisfied but also affect provider’s credibility in the long run.
  • All efforts are focused towards one very secure environment and inputs from previous research will immensely help us to lay foundation for new research we undertake.
  • Examine impact of TLS on DoS and ToS A good number of attacks identified will be eliminated TLS is not ready for “prime time” yet Few IP phone vendors are implementing SIP over TCP, a first step towards TLS
  • Application level security is already defined by SIP RFC 3261, an improvement from relatively less secure RFC 2543. Although they are good ways to prevent outright compromise of network, we need to extend our security architecture to deal with attacks like: Call fraud, compromised machines, theft of service, identity assurance and more. VoIP has converged advantages from various networks together: cost effectiveness of data networks and convenience of voice networks. However this convergence has also made VoIP susceptible to data network attacks like DDoS, Message Alteration, Spoofing, Session anomalies and also voice attacks like Call Theft, Call Hijacking, Spam. UDP – Spoofing; Plain Text – Spam, Flood of messages (although all SIP networks are not connected yet); Message Vulnerability – Network downtime, alteration, etc. Application Layer Security SIP RFC 2543 – little security SIP RFC 3261 – security enhancements Digest Authentication TLS IPSec SRTP/ZRTP Perimeter Protection SIP aware Filtering Mechanisms SIP aware DOS Protection Detection and Mitigation
  • SIP calls are stateful RTP media ports are negotiated during signaling, assigned dynamically, and taken down SIP signaling is done over a static port: 5060 INVITE message contains an SDP message indicating the caller’s incoming media port (e.g., 43564 ) Response 200OK has SDP with the callee’s incoming media port Each port creates a pinhole in firewall Pinholes are kept open only until a BYE message signals closing of both pinholes Firewall must keep a state table with all active pinholes to check if an arriving RTP packet can enter through an open pinhole, otherwise drop packet
  • NOTE: Try to take sub bullets out for ToS and just give a flavor, and discuss this later. Billing Threats Unauthorized deletion or altering of billing records Unauthorized bypass of lawful billing systems Unauthorized billing Service Threats Enough funds Enough coverage Physical Threats Taking of service provider property
  • user datagram protocol (UDP) instead of TCP/TLS plain text instead of S/MIME
  • Rate limiting – SIP is request/response. Dialog, Transaction – are a way to granularize a sip session in space and time so thresholding can be applied more effectively . More details in the following slides By doing analysis, we’ve come up with narrow ranges in space and time of the expected number and order of requests/responses
  • NOTE: Follow this with demo…
  • Transcript

    • 1. Columbia - Verizon Research Collaboration Secure SIP: Scalable DoS and ToS Prevention Mechanisms for SIP-based VoIP Systems, and Validation Test Tools May 6, 2010 Sarvesh Nagpal, Eilon Yardeni, Henning Schulzrinne Columbia University Gaston Ormazabal Verizon Laboratories
    • 2. Agenda
      • Discussion: A successful collaboration…
        • Value to Verizon
      • Project Overview
        • Background, Research Focus, and Goals
        • DoS
          • DoS Detection and Mitigation Strategy
          • DoS Validation Methodology - DoS Automated Attack Tool
        • ToS
          • ToS Integrity Verification Tool and Validation Methodology
      • Intellectual Property
      • Next Steps
      • Conclusions
    • 3. Discussion… A “successful” collaboration
    • 4. A Successful Collaboration
      • Want a realistic perspective on what makes projects succeed and what is unlikely to work
        • Industry must see value or need to pursue IP
          • Rapid commercialization/productization “in house” or with an external industry partner
        • Agreement on fair distribution of rights/obligations
      • Typical arrangement: GRA + professor
        • Who typically needs to supervise multiple projects at the same time
        • Often companies seem to have the illusion that they get the faculty's full attention...
      • Require full attention of industry SME
            • Student mentoring/coaching
              • Industry perspective
              • Writing/Presentation skills
            • Manage Deliverables
    • 5. Deliverables Management
      • Convert collective research insights into industry deliverables
      • Clear understanding of deliverables
        • Standards
        • Reports
        • Systems/Prototypes
      • Timelines
        • Start time and academic calendar
        • MS GRA vs. PhD
    • 6. Value to Verizon
      • Intellectual Property with SIX Patent Applications
      • Licensing Agreement
        • Taken research quickly into marketplace
        • Five vendors interested
      • Enhanced VoIP security through standards and vendor involvement
        • Worked with Verizon vendors to mitigate exposures
      • Rolled the requirements and lessons learned into the Verizon security architecture and new element requirements database for procurement
        • Columbia requirements valid for VoIP, Presence and Multimedia architectures
      • Setup laboratory facilities for VoIP security evaluations and product development
        • In Columbia, prototype rapid development incubator
        • In Verizon, incorporated Columbia/Verizon collaborative test tools for a more realistic complex IP-routed laboratory environment
    • 7. Verizon Business Impact
      • SIP DoS work impact on Verizon Business
        • Network & Information Security Organization
          • “ Better Security Reviews” of Advantage VoIP Service
        • Global Customer Service & Provisioning Organization
          • Sales Engineering – Premier Accounts Team Briefing
        • Global Network Engineering & Planning Organization
          • Support Technology organization to define new security architecture for VoIP Services
      • SIP ToS work impact on Verizon Business
        • Office of Chief Financial Officer
          • Credit&Collections
    • 8. Background & Research Focus
      • SIP is the VoIP protocol of choice for both wireline and wireless telephony
        • Control protocol for the Internet Multimedia Systems (IMS) architecture
      • VoIP services migrating to IP fast becoming attractive DoS and ToS targets
        • DoS attack traffic traversing network perimeter reduces availability of signaling and media for VoIP
        • Theft of Service must be prevented to maintain service integrity
          • Reduces ability to collect revenue and provider’s reputation both are at stake
      • Attack targets
        • SIP infrastructure elements (proxy, softswitch, SBC, CSCF-P/I/S)
        • End-points (SIP phones)
        • Supporting services (e.g., DNS, Directory, DHCP, HSS, DIAMETER, Authorization Servers)
      • Verizon needs to solve security problem for VoIP services
        • Protocol-aware application layer gateway for RTP
        • SIP DoS/DDoS detection and prevention for SIP channel
        • Theft of Service Architectural Integrity Verification Tool
      • Need to verify performance & scalability at carrier class rates
        • Security and Performance are a zero sum game
      • Columbia likes to work in real life problems & analyze large data sets
        • Goal of improving generic architectures and testing methodologies
        • Columbia has world-renowned expertise in SIP
    • 9. Goals
      • Study VoIP DoS and ToS for SIP
        • Definition – define SIP specific threats
        • Detection – how do we detect an attack?
        • Mitigation – defense strategy and implementation
        • Validation – validate our defense strategy
      • Generate requirements for future security network elements and prototypes
        • Share these requirements with vendors
      • Generate the test tools and strategies for their validation
        • Share these tools with vendors
    • 10. VoIP Threat Taxonomy Scope of our research - 2006 Scope of our research - 2007 *- VoIP Security and Privacy Threat Taxonomy, VoIP Security Alliance Report, October, 2005 ( http:// www.voipsa.org )
    • 11. Denial of Service & Theft of Service
      • Denial of Service – preventing users from effectively using the target services
        • Service degradation to a “not usable” point
        • Complete loss of service
      • Distributed Denial of Service attacks represent the main threat facing network operators *
        • Most attacks involve compromised hosts (bots)
          • botnets sized from a few thousands to over million
          • 25% of all computers on Internet may be botnets
      • Theft of Service – any unlawful taking of an economic benefit of a service provider
        • With intention to deprive of lawful revenue or property
      *- Worldwide ISP Security Report, September 2005, Arbor Networks *- Criminals 'may overwhelm the web', 25 January, 2007. BBC
    • 12. DoS Mitigation Strategy
      • Implementation flaws are easier to deal with
        • Systems can be tested before used in production
        • Systems can be patched when a new flaw is discovered
        • Attack signatures can be integrated with a firewall
      • Application level and flooding attacks are harder to defend against
        • SIP infrastructure element defense
      • Commercially available solutions for general UDP/SYN flooding but none for SIP
      •  Address application level and flooding attacks specifically for SIP
      •  Identify and address architectural weaknesses before they are exploited to commit ToS
    • 13. DoS Mitigation Solution Overview Untrusted DPPM sipd Trusted SIP SIP SIP RTP RTP Filter I Filter II Untrusted DPPM sipd Trusted SIP SIP SIP RTP RTP Filter I Filter II VoIP Traffic Attack Traffic
    • 14. Hardware Platform System Level Port Distribution Application Server Module Pentium 1GHz 10/100/1000 10/100 E1 E2 Backplane F0 C3 C4 Gigabit Ethernet Interconnects D0 D1 E1 E2 F0 C3 C4 D0 D1 3 4 P0 P0 1000 1000 0 1 2 ASM DPPM Intel IXP 2800 DPPM Intel IXP 2800
    • 15. Integrated DDOS and Dynamic Pinhole Filter DPPM Linux server Switch FCP/UDP Drop Lookup ASM Inbound Outbound SIP CAM Dynamic Table Static Table CAM SIP DDOS DDOS Table CAM sipd
    • 16. Integrated Testing and Analysis Environment GigE Switch GigE Switch SIP Proxy Call Handlers SIPUA/SIPp Controller secureSIP Attack Loaders SIPStone/SIPp Legitimate Loaders SIPUA/SIPp Firewall
    • 17. Theft of Service Overview
      • VoIP is different
        • Not a static but a real-time application
        • Direct comparisons with PSTN
          • According to Subex Azure 3% of total revenue is subject to “fraud”*
          • VoIP can be expected to be at least twice as large a proportion of revenue
        • Theft of Service is more daunting problem in VoIP
      • Implications of ToS
        • Lost revenue and bad reputation
        • Abused resources cause monetary losses to network providers
        • Unauthorized usage degrades whole system’s performance
      • Scenarios
        • Using services without paying
        • Illegal Resource Sharing (unlimited-plans)
        • Compromised Systems
        • Call Spoofing and Vishing
      *Billing World and OSS Magazine: “Top Telco Frauds and How to Stop Them”, January 2007, by Geoff Ibett
    • 18. The Bigger Picture - Columbia VoIP Testbed
      • Columbia VoIP test bed is collection of various open-source, commercial and home-grown SIP components
        • provides a unique platform for validating research
      • Columbia-Verizon Research partnership has addressed major security problems
        • signalling, media and social threats
      • Researched DoS solutions verified against powerful test setup at very high traffic rates
      • ToS successfully validated integrity of different setups of test bed
    • 19. Intellectual Property – Six Patent Applications
      • “ Fine Granularity Scalability and Performance of SIP Aware Border Gateways: Methodology and Architecture for Measurements”
        • Inventors : Henning Schulzrinne, Kundan Singh, Eilon Yardeni (Columbia), Gaston Ormazabal (Verizon)
      • “ Architectural Design of a High Performance SIP-aware Application Layer Gateway ”
        • Inventors : Henning Schulzrinne, Jonathan Lennox, Eilon Yardeni (Columbia), Gaston Ormazabal (Verizon)
      • “ Architectural Design of a High Performance SIP-aware DOS Detection and Mitigation System”
        • Inventors: Henning Schulzrinne, Eilon Yardeni, Somdutt Patnaik (Columbia), Gaston Ormazabal (Verizon)
      • “ Architectural Design of a High Performance SIP-aware DOS Detection and Mitigation System - Rate Limiting Thresholds”
        • Inventors: Henning Schulzrinne, Somdutt Patnaik (Columbia), Gaston Ormazabal (Verizon)
      • “ System and Method for Testing Network Firewall for Denial of Service (DoS) Detection and Prevention in Signaling Channel ”
        • Inventors: Henning Schulzrinne, Eilon Yardeni, Sarvesh Nagpal (Columbia), Gaston Ormazabal (Verizon)
      • “ Theft of Service Architectural Integrity Validation Tools for Session Initiation Protocol (SIP) Based Systems ”
        • Inventors: Henning Schulzrinne, Sarvesh Nagpal (Columbia), Gaston Ormazabal (Verizon)
    • 20. External – Publications, Presentations, Recognition
      • Presentation at NANOG 38 – Oct. 10 2006 (HS/GO)
        • “ Securing SIP: Scalable Mechanisms for Protecting SIP-Based VoIP Systems ”
          • Authors : Henning Schulzrinne, Eilon Yardeni, Somdutt Patnaik (Columbia), Gaston Ormazabal (Verizon)
        • Paper approved for publication in NANOG 38 2006 Proceedings
        • Made a headline in VON Magazine on October 11, 2006: http://www.vonmag.com/webexclusives/2006/10/10_NANOG_Talks_Securing_SIP.asp
      • Presentation to at Global 3G Evolution Forum – Tokyo, Japan, Jan. 2007 (GO)
      • Presentation at IPTComm 2007 – New York City, July, 2007 (GO)
      • Presentation at OSS/BSS Summit – Tucson, AZ, September, 2007 (GO)
      • Paper in development for current work (to be presented at IPTComm 2008)
        • “ Secure SIP: A scalable prevention mechanism for DoS attacks on SIP based VoIP systems”
          • Authors : Henning Schulzrinne, Eilon Yardeni, Sarvesh Nagpal (Columbia), Gaston Ormazabal (Verizon)
      • Work incorporated in a new Masters level course on VoIP Security taught at Columbia in Fall 2006
        • COMS 4995-1: Special Topics in Computer : VoIP Security (HS)
      • CATT Technological Impact Award - 2007
    • 21. Recommended Next Steps
      • Conversion of research into a product that Verizon can use
        • Verizon needs to determine optimal architectural placement of DoS prevention functionality for VoIP and Presence Security
          • Security vs. Performance
          • Hardware vs. Software Implementation
            • Proxy/Softswitch (SW)
            • SBC or New network element (HW/SW)
          • Use internally (protect VZ Network)
          • Use externally (sell new security services to large customers)
      • Need rapid commercialization
        • Licensing Agreement with equipment manufacturers
        • Exclusive vs. Non-exclusive
      • Continue relationship with Columbia
        • Research in related areas
          • Proposal to study SRTP
        • Maintain the testbeds for further research and to assist in product development during product testing cycle
        • Feedback loop of research and product cycle
        • Get other companies interested to synergize resources and share results
      • What can we see doing to make the working relationship even more productive?
    • 22. Conclusions
      • Research Results
        • Demonstrated SIP vulnerabilities for VoIP resulting in new DoS and ToS susceptibility
          • Work is fully reusable to secure a “Presence” infrastructure
        • Implemented some “carrier-class” mitigation strategies
          • Developed generic requirements
          • Remove SIP DoS traffic at carrier class rates
          • Prototype is first of its kind in the world
        • Built a validation testbed to measure performance
          • Developed customized test tools
          • Built a high powered SIP-specific Dos Attack tool in a parallel computing distributed testbed
            • Crashed a SIP Proxy in seconds
          • Built a Theft of Service Architectural Integrity Validation Tool using parallel computing
      • Intellectual Property
        • Worked resulted in six patent applications
      • Commercialization
        • Licensing agreements currently under negotiation
        • Revenue both to Columba and Verizon
        • Need to socialize new requirements and test tools with vendor community to address rapid field deployment
          • Vendors generally very interested in new requirements
          • Rapid implementation is now expected
    • 23. Thank You Thank you Questions?
    • 24. Backup Slides…
    • 25. SIP Security Overview
      • Application Layer Security
        • SIP RFC 2543 – little security
        • SIP RFC 3261 – security enhancements
          • Digest Authentication
          • TLS
          • IPSec
        • SRTP/ZRTP (RFC 3711)
      • Perimeter Protection
        • SIP aware Filtering Mechanisms
        • SIP aware DOS Protection
          • Detection and Mitigation
    • 26. SIP Security Overview - ??
      • Application layer security
          • Digest Authentication, TLS, S/MIME, IPSec, certificates
          • SRTP/ZRTP for media
      • Convergence leads to converged attacks
        • Data network attacks
          • DDoS, spoofing, content alteration, platform attacks
        • Voice over IP network attacks
          • Toll fraud, session hijacking, theft of service, spam/spit
      • Most security problems are due to
        • User Datagram Protocol (UDP) instead of TCP/TLS
        • Plain text instead of S/MIME
        • Message/Method vulnerability
        • Flexible grammar --> syntax-based attacks
    • 27. Dynamic Pinhole Filtering CAM Table SIPUA User2 SIPUA User1 128.59.19.163:43564 128.59.19.163:56432 SIP/2.0 200 OK From: <sip:user1@handler> c=IN IP4 128.59.19.162 m=audio 56432 RTP/AVP 0 INVITE sip:user1@proxy.com From: <sip:user2@loader> c=IN IP4 128.59.19.163 m=audio 43564 RTP/AVP 0
    • 28. SIP DoS and ToS Attack Taxonomy
      • DoS
        • Implementation flaws
        • Application level
        • Flooding
      • ToS
        • Billing Threats
        • Authorization Threats
        • Service Threats
    • 29. Strategy Focus
      • VULNERABILITY : Most security problems are due to:
        • flexible grammar  syntax-based attacks
        • Plain text  interception and modification
        • SIP over UDP  ability to spoof SIP requests
          • Registration/Call Hijacking
          • Modification of Media sessions
          • SIP ‘Method’ vulnerabilities
            • Session teardown
            • Request flooding
            • Error Message flooding
          • RTP flooding
      • STRATEGY: Two DoS detection and mitigation filters and ToS tools
        • SIP: Two types of rule-based detection and mitigation filters
        • Media: SIP-aware dynamic pinhole filtering
        • ToS Architectural Integrity Verification Tool
      Application Level Flooding
    • 30. SIP Detection and Mitigation Filters
      • Authentication Based - Return Routability Check
        • Require SIP built-in digest authentication mechanism
          • Null-authentication (no shared secret)
        • Filter out spoofed sources
      • Method Specific Based – Rate Limiting
        • Transaction based
          • Thresholding of message rates
            • INVITE
            • Errors
          • State Machine sequencing
            • Filter “out-of-state” messages
            • Allow “in-state” messages
        • Dialog based
          • Only useful in BYE and CANCEL messages
      • Dynamic Pinhole Filtering for RTP
          • Only signaled RTP media channels can traverse perimeter
            • Obtain from SDP interception
          • End systems are protected against flooding of random RTP
    • 31. Test Tools
      • SIPp, SIPStone, and SIPUA are benchmarking tools for SIP proxy and redirect servers
        • Establish calls using SIP in Loader/Handler mode
        • A controller software module (secureSIP) wrapped over SIPp/SIPUA/SIPStone launches legitimate and illegitimate calls at a pre-configured workload
      • SIPp
        • Robust open-source test tool / traffic generator for SIP
        • Customizable XML scenarios for traffic generation
        • 5 inbuilt timers to provide accurate statistics
        • Customized to launch attack (SIP DoS) traffic designed to cause proxy to fail
      • SIPStone continuously launches spoofed calls which the proxy is expected to filter
        • For this project enhanced with:
          • Null Digest Authentication
          • Optional spoofed source IP address SIP requests
      • SIPUA Test Suite
        • Has built-in Digest Authentication functionality
        • Sends 160 byte RTP packets every 20ms
          • Settable to shorter interval (10ms) if needed for granularity
        • Starts RTP sequence numbers from zero
        • Dumps call number, sequence number, current timestamp and port numbers to a file
    • 32. secureSIP Control Architecture
    • 33. secureSIP Test Results for DoS SIP DoS Measurements (showing max supported call rates) Dynamic Pinhole Firewall Filters OFF Firewall Filters ON Traffic Composition Good CPS Attack CPS CPU Load Good CPS Attack CPS CPU Load Non-Auth Traffic 690 0 87.81 690 0 88.04 Auth Good Traffic 240 0 19.83 240 0 39.64 480 0 81.20 480 0 81.75 Auth Good Traffic + Spoof Traffic 240 2950 83.64 240 16800 41.39 480 195 85.40 480 14400 82.72 Auth Good Traffic + Flood of Requests 240 3230 84.42 240 8400 40.83 480 570 86.12 480 7200 82.58 Auth Good Traffic + Flood of Responses 240 2970 87.2 240 8400 41.33 480 330 86.97 480 7200 82.58 Auth Good Traffic + Flood of Out-of-State 240 2805 86.24 240 8400 40.29 480 290 84.81 480 7200 82.19 Concurrent Calls Call rate (CPS) Delay due to Firewall Pinhole opening Pinhole closing 20000 300 0.73 0 25000 300 0.75 0 30000 300 0.83 15.51 30000 200 0.80 0.02