Presentation

Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
IMPACT – Team 3
Amrita Deo
Fatima Khan
Kayla Macdonald
Chelsea Metcalf
Sagarkumar Shah

Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement. 2
Team Introduction
Name College Major Current Year Intern
Assignment
Amrita Deo
(COS)
University of
California
Berkeley
Computer
Science
Junior Metadata
Management
Project
Fatima Khan
(COS)
George Mason
University
Information
Technology
Senior IMPACT –
Name Service
Browser
Kayla
Macdonald
(TT)
Cornell
University
Computer
Science
Senior N&T
Architecture
Chelsea
Metcalf (SV)
University of
Florida
Computer
Science
Ph.D. DTI Express
Sagarkumar
Shah
(COS)
University of
Texas at
Arlington
Computer
Science
Masters IMPACT –
Name Service
Browser

• IMPACT – Integrated Management Platform (for) Advanced
Communication Technologies
• IMPACT provides surveillance, alarm topology augmentation, correlation,
ticketing and automation capabilities for the Verizon Wireline global
network for Verizon operation center users
– Finds problems within our network
– IMPACT is internal to Verizon
– Trouble Tickets created in:
• ETMS (external networks)
• vRepair (internal networks)
• Our task
– Filter and analyze Event Services error logs part of IMPACT
– Suggestions to improve efficiency
IMPACT

• Elasticsearch
– An open source search server
– Developed in Java
• Logstash
– Data pipeline for managing documents
– Collects, processes, and forwards events and log messages
• Kibana
– Interactive dashboard to show the visualizations
– Easy to chart and clearly visualize data
ELK

Timeline
July 20:
Watch tutorial
videos
July 20:
Download
Logstash,
ElasticSearch,
and Kibana
July 21:
Examine the
log files
July 21:
Load log files
into Logstash
July 22 - 28:
Analyze log
files
July 22 - 27:
Create
visualizations &
dashboard in
Kibana
July 27 - 29:
Formulate
business
proposal and
presentation

• Download Elasticsearch,
Logstash, and Kibana
• Navigate to the Elasticsearch
folder and go into “bin”. Run:
elasticsearch.bat
• Navigate to the Logstash
folder and go into “bin”. Run: logstash –f <filepath> where filepath
is the configuration file
• Navigate to the Kibana folder and go into “bin”. Run: kibana.bat
Set Up

• Initial log files from testing environment (4 files)
• Error Log files (104 files) from production environment
• Instead used error log files from production servers (IPGOMP91 and
IPGOMP92)
• Analyzed error logs to find a way to make the application more efficient
Workflow
Time Class Name Message

• Configuration file
– Input
– Grok filters
Workflow

– Output
• Template Configuration
Workflow

Results & Analysis
common.pojo.ESUtil
ejb.AddAlarmsCommentMDB
ejb.ESDBServicesBean
ejb.EventServicesBean
ejb.TUSMessageProcessorMDB
52 days 151943

Results & Analysis – A Closer Look
Describes what state the
event is in; where
automation is in the
process
Tried to add a comment on
a ticket but the ticket is
locked doing something
else at that time and
cannot make the update
Error for alarm is
associated with another
event
Tried to add alarm to event
but it already closed
Unable to find key for alarm
Inner Circle:
First part of the error
message
Outer Circle
Second part of the
error message
Together make the
full error message

Dashboard Demonstration

• Two possible solutions:
1. Removing extra data from error log files, allowing more history in the log
• This level of effort would cost a little less than $4,000
– A one-time investment
• These current common errors occur frequently
– 76% of current error log is unnecessary reporting
– Allows each error log file to hold information for 9 days rather than 3 days
– Saves processing time for requests
» Development will not have to search through the NIWS database as often
2. Increase the application log disk space by 1.8 GB
• Would cost approximately $2,010
– A one-time investment
• More information can be kept in the logs
• This is a more cost effective solution
Suggested Solutions

TASK TIME
Reporting problem to Support 15 minutes
Support does research 60 minutes
Support gives problem to Development
Development does research 60 minutes
Not in database; Development  Support
Support does research on recent example of issue 60 minutes
Support  Development
Development finds the similar problem 60 minutes
Current Processes & Financials
TOTAL TIME: 255 minutes
255 minutes ÷ 60 minutes = 4.25 hours
4.25 hours x $48 hours blended rate = $204 per request
$204 x 7 requests per month = $1,428 per month
$1,428 x 12 months = $17,136 per year

TASK TIME
Reporting problem to Support 15 minutes
Support does research 60 minutes
Support gives problem to Development
Development does research 60 minutes
Not in database; Development  Support
Support does research on recent example of issue 60 minutes
Support  Development
Development finds the similar problem 60 minutes
Processes & Financials
for the Suggested Solutions
TOTAL TIME: 135 minutes
135 minutes ÷ 60 minutes = 2.25 hours
2.25 hours x $48 hours blended rate = $108 per request
$108 x 7 requests per month = $756 per month
$756 x 12 months = $9,072 per year

1. Removing extra information
80 hours LOE x $48 blended rate = $3,840 additional in year one cost
Savings after 5 years: ($8,064 x 5 yrs) = $40,320 - $3,840 = $36,480 total
2. More disk space
$10 + order and procurement time = $2,010 additional in year one cost
Savings after 5 years: ($8,064 x 5 yrs) = $40,320 - $2,010 = $38,310 total
Financials for Suggested Solutions
Year 1 2 …
Cost $3,840 + $9,072 = $12,912 $9,072 $9,072
Year 1 2 …
Cost $2,010 + $9,072 = $11,082 $9,072 $9,072

• Where IMPACT is now:
– 3 days of information per log file
– 4+ hours, $200+ per request
• Suggested changes:
– 9 days of information per log file
– Greater disk space (1.8 GB) - $2,000 one time cost
OR
– Separate log file for errors/warnings
• What will improve:
– Faster log lookups
– Save money, time and resources
• 2+ hours per request
• Either option will save $8,064 per year after the first year
Executive Summary

• Pros:
– Gave IT interns an exposure to business processes
– Networking with interns in other locations
– Working in groups and collaborating with other interns
• Cons:
– Vague instructions
– Time constraints
– Initially given testing log files
– Online document sharing websites were inaccessible for collaboration on the
Verizon network
– Time difference between interns in the group made it difficult to communicate
Our Experience

19Confidentialand proprietary materials for authorized Verizon personnel and outside agencies only. Use, disclosure or distribution of this material is not permitted to any unauthorized persons or third parties except by written agreement.
Thank you!

Backup Slides

• An open source search server
• Developed in Java
• Provides scalable search
• Near real-time search
• Allows for multiple independent instances of applications to work in a
shared environment
• Used Elasticsearch to modify the data loaded in Logstash so that it is
ready for Kibana
ElasticSearch

• Data pipeline for managing documents
• Open source tool
• Collects, processes, and forwards events and log messages
• Using Logstash:
– Uploaded the IMPACT error log files to Logstash
– Wrote the configuration file to process the log files
Logstash

• Flexible analytics
• Visualization platform
• Easy to chart and clearly visualize data
• Used Kibana to observe trends from the data
• Interactive dashboard to show the visualizations
• Easy to chart and clearly visualize data
Kibana

Presentation

Recommended

Recommended

More Related Content

Similar to Presentation

Similar to Presentation (20)

Presentation

Editor's Notes