Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Nena 2016 - The Importance of a Complete Network Inventory: What We Learned


Published on

At the 2016 NENA Conference in Indianapolis, IN, Jack Kessler of 20/20 Technical Advisors, LLC and Dana Wahlberg of the State of Minnesota Emergency Communication Networks talked about the work 20/20 Technical did building the inventory for the State of Minnesota and how important it has become. A comprehensive network inventory in integral to the daily operation of a 9-1-1 network. As public safety migrates to NG9-1-1, it becomes even more important. This presentation shares the State of Minnesota's experiences of life without that comprehensive inventory, the benefits after completion, and how it is maintained and kept up-to-date.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Nena 2016 - The Importance of a Complete Network Inventory: What We Learned

  1. 1. Dana Wahlberg, State of MN 911 Program Manger Jack Kessler, Managing Director 20/20 Technical Advisors LLC
  2. 2. Minnesota’s Situation • MN paid cost recovery to carriers for network elements to connect from end offices to LSRs • Initially meant to be temporary back in the 1980s • Acquisitions and re-homings have perpetuated the cost recovery cycle • As contracts with carriers have expired, they have been difficult to renew • Meet point billing challenges • Billing discrepancies • Lack of itemized billing and carrier billing personnel changes over the years • Circuit segments remaining when a circuit had been disconnected • Carriers not keeping adequate records and fear of disconnecting active network
  3. 3. Minnesota’s Situation • In 1Q 2014 all MN PSAPs were migrated from the state’s 12 LSRs upon completion of the the statewide ESInet • LSRs remain functional network elements serving as aggregation points for carrier network (ES trunks) demark points • LSRs are becoming EOS/EOL and the State will go out to RFP seeking a solution to migrate all carrier traffic to alternate aggregation points • Need for complete and accurate end to end circuit inventory • Need for billing discrepancies to be resolved • Need to resize network more reflective of current customer
  4. 4. Minnesota’s Action • Went out to RFP to seek a vendor to: • Complete end to end network inventory of all 9-1-1 circuit elements • Work with carriers to obtain information as needed • Work with carriers on filing of annual CLEC 9-1-1 plan reflective of actual network • Assist State with reconciling billing discrepancies • Assist carriers with preparing electronic itemized bills per statute requirement • Assist State with renewing carrier contracts reflective of elements and cost • Assist State with procuring an automated telecom expense and asset management tool • Paper process for 100 carriers UNMANAGEABLE and ARCHAIC both for network adds/moves/changes/disconnects and for managing timely invoice remittances • Approximately 1000 bills received monthly as many as 120 separate invoices from a single carrier
  5. 5. What is a 9-1-1 network inventory? • End to end network database with associated network diagrams that identifies • County (Serving PSAP) • Carrier • Circuit ID • Circuit type • Overall circuit A and Z location • Actual A and Z terminations for carrier (meet point) • Corresponding port charges • Account number • Billing number • Individual components and associated tariff or contract charges
  6. 6. Why is it important? • Securing the 9-1-1 network infrastructure • Knowledge of all entry points comprising network • Ensuring network is monitored and protected • Identifying unnecessary equipment and circuitry which pose security risks • Fiscal responsibility • Ensure network no longer in use is disconnected from end to end • Ensure all disconnected elements are no longer billed on invoices
  7. 7. Benefits of a Network Inventory? • Makes management of the overall network more efficient • Adds • Is additional circuitry warranted • How will it impact the serving PSAP • Moves (Rehoming) • Will it improve diversity and/or minimize costs • Changes • Addressing a configuration issue • Disconnects • Reduces time to identify all carriers on path to disconnect and discontinue billing • Validate against current invoices
  8. 8. How Did We Start the Inventory? • The State selected and contracted with 20/20 Technical Advisors LLC to complete the inventory and related projects • The State provided 20/20 with a LOA allowing carriers to release network information and associated billing information to 20/20 • We are currently in our 3rd year • End to end inventory is 90% complete • Thousands of dollars in MRC has been achieved as a result of identifying and removing obsolete network elements
  9. 9. How Did We Start the Inventory? • First identified what data was available to us • There was lots of data available • Most data was in paper form in file cabinets and some digital • Invoices, contracts, certifications from carriers • PSAP plans, CLEC plans, carrier CSRs, LOAs • Carrier circuit profiles, basic network drawings
  10. 10. How Did We Start the Inventory? • There was incomplete or old data • The network was swiftly installed in the early 80s • Two router providers • One provider had circuit profiles built and the basic network drawings • Some of the data was obsolete because it was so old • Some data was not updated as changes had been made
  11. 11. Gathering the Data • The amount of data was large • Approximately 20 long file drawers of files - very organized • Hundreds of digital files • Scanned or collected approximately 7,000 documents
  12. 12. Gathering the Data • Products used • 20/20 Technical Advisors’ service-desk portal • Laptops • Two high-speed portable double-sided scanners • Secured storage – to prevent data loss during scanning • As the project went on we used that to share data with the State • Encrypted USB drives
  13. 13. Who Was Involved? • Minnesota Emergency Communication Networks Staff • 20/20 Technical Advisors, LLC Staff • Carriers • Router providers – IES and CenturyLink • Carrier engineers – design, maintenance, central office • Carrier billing staff • Carrier account mangers
  14. 14. Building the Inventory? • Started with wireline inventory • The Stete • Built standardized spreadsheets to house the data by county • Inventoried carrier circuits between • Carrier end office and selective router (ES) • Selective router to the PSAP (EM) • Data circuits for ALI lookup • Inter-tandem circuits between selective routers • Inventoried B-1 POTS lines and routing numbers at the PSAPs • Developed new comprehensive drawings
  15. 15. Building the Inventory • Currently building the wireless inventory • Built spreadsheets similar to the wireline • Drawings will be completed as the inventory is completed • NG9-1-1 • NG9-1-1 was built to the State’s specifications • Since it was built from ground up the data and drawings are adequate
  16. 16. Product Examples • Data in the circuit inventory • County served • Billing carrier • Type of circuit – EM, ES, ALI, B1, RTN, Inter- tandem • Overall circuit A location and Z location • Each circuit segment A location and Z location • Primary segment circuit ID • Segment circuit ID • Account number • Billing Telephone Number • If disconnected? • Percentage of circuit owned by carrier • Mileage • Total cost of each circuit segment • Costs that make up segment cost • ILEC or CLEC ES circuit
  17. 17. Drawings • Previous Drawings • Some were incomplete • Some where busy and hard to interpret • All circuits for a county on a one page drawing • Showing the number of circuits • Start and finish • Did not have drawings for everything
  18. 18. Starting Artifact
  19. 19. Drawings • New drawings • Detailed • Each county has a packet of drawings inclusive of end to end network • Each carrier that has end offices in a county have a drawing in the packet • Callout boxes that show • Carrier providing the segment • The end office supported • Circuit IDs • Where each segment starts and finishes
  20. 20. Call Out Box Close Up
  21. 21. Obstacles • Missing data • Carriers did not have info • Carriers did not know where to get info • Profiles and drawings lacking • Billing issues • Circuit discrepancies
  22. 22. Clearing Obstacles • Missing data • If circuits are being billed, work with carrier billing • Work with carrier engineers • Work with selective router engineers • Carriers did not have info • Records lost with system change overs • Records lost with acquisitions • Sometimes we had to work with each carrier that had part of the circuit
  23. 23. Clearing Obstacles • Carriers did not know where to get info • Especially small carriers • This was the toughest one • Really had to rely on the selective router carriers to help research • Profiles and drawings lacking • We built what we needed for a profile and gave it to the carrier
  24. 24. Issues We Found • Billing issues • Circuits that were disconnected but still being invoiced • Carriers had issues with reconciling the invoices when disconnects ordered • Circuit segments orphaned – the rest of the circuit disconnected but that circuit was missed • Invoice did not match contracts or certifications • Circuits were not being billed
  25. 25. Issues We Found • Circuit discrepancies • Mileage amounts in a circuit did not total up correctly • The number of circuits listed were wrong
  26. 26. Other Items Noticed • Carriers are over trunked in certain areas • Mainly found this in areas with a high concentration of remote central offices off of a host • Not all carriers calculated mileage the same way
  27. 27. Maintaining the Inventory • Build good processes • Processes for moves, adds and changes for any orders • Tracking and authorizing costs changes – tariff changes, fee increases, circuit change cost • Notification of changes • Who approves the changes • Who gets notified of changes • When are they implemented
  28. 28. Maintaining the Inventory • The State of Minnesota is implementing Telesoft • Telecom Expense Management and Asset Tool • Tracks and reports on circuit orders in a work flow – adds, moves, changes • Track invoices, contract dates, contract terms • Will flag discrepancies on invoices as they come in • Assists in invoice dispute management • EDI feeds from carriers for electronic invoices instead of paper • Carriers email invoices via spreadsheet if EDI is not available • Cost reporting
  29. 29. Summary • A good network inventory • Helps with securing your environment • Increases efficiency of managing your network • Helps with understanding network costs • Tracking the inventory assets and costs • Build good processes for managing the inventory • Use a good telecom expense management tool
  30. 30. Questions – Thank You • Dana Wahlberg • • 651-201-7546 • Jack Kessler • • 317-249-8100 ext. 1001