Bill Alderson is an expert in responding to high-profile IT problems at government and commercial organizations. He has helped restore communications at the Pentagon after 9/11 and optimized networks and applications for the Joint Chiefs of Staff and other military organizations. In his presentation, Alderson will demonstrate the tools and techniques used to perform deep packet analysis and pinpoint the root causes of complex IT issues in order to optimize systems and avoid unnecessary upgrades.

2. Session Date & Time
• Date: Wednesday, June 29, 2016
Time: 1100-1145
Location: Tuscany Ballroom

3. • Bill Alderson responds to Information Technology high
visibility, high stakes technical problems. Network outage,
slowness, slow applications or disasters affecting
government and commercial Information Technology
Enterprise environments. ABC News told the story of how
Bill and his team helped restore communications at the
Pentagon immediately following 911. Bill assisted with six
deployments to Iraq and Afghanistan requested by Army
G2, Joint Chiefs and US Central Command diagnosing
Biometrics and others critical systems. One of his missions
is to help executives and technologists see both technical
and leadership root causes that can be obviated through
common sense best practices.

4. Bill Alderson Infographic Bio

5. • Deep packet analysis remains essential for definitive irrefutable diagnosis
and optimization of complex systems. Bill demonstrates the tools,
techniques and methods used to annotate complex technology findings so
that technologists, managers, executives and vendors can agree on root
cause. Once the problem is identified and agreed upon the true pinpoint
mitigation can begin. The days of shotgun style "forklift wholesale
upgrades" on everything have passed. We must optimize existing assets
allowing them to perform well.
Bill has proven ability to optimize large scale networks and applications
from experience in analyzing the Pentagon immediately following 911,
analysis of Biometrics applications across Iraq and Afghanistan, numerous
optimizations of Joint Chiefs of Staff and OSD network analysis. Experience
from analysis of the largest 100 commercial enterprise networks such as
Stock Exchanges, Financial, Insurance and Healthcare institutions will be
demonstrated with annotated examples for CIO, Executives and top level
technologists.

6. IT Critical Problem Resolution
Technology and Psychology
bill@apalytics.com

7. “Swiss Army Knife” Portfolio of Tools
Select Well.
Avoid Spending
Only on “Suites”
All-in-one-tools
Although easier to “buy”
don’t solve many problems.
They leave you “broke and broken”
with a gold plated toolset.

8. Optimization Troubleshooting Phases

9. Preparation & Setup

10. Analysis & Iteration

11. Reporting & Presentation

12. Problem Management

13. Down - Intermittent - Slow

14. Technical vs. Leadership Root Cause

15. The Needle

16. The Environment

17. Packet Traces

18. Store Every Packet?
Who’s can and is going to analyze them and when?

19. Finding The Stack With The Problem

20. Finding The Needle

21. Measured at the Server
Fast TCP connect
time. Fast Ack from
F5 does not show
true client response
time which is why
Apalytics provided
Internet Monitoring.
1.4 second Get
response is very slow
which is why detailed
platform and
application analysis
was performed.
The 2nd & 3rd Gets
were fast at 1
millisecond proving
some commands are

22. CF Longest Requests
1,958,266ms = ~32 minutes from one request
391,692ms = ~7 minutes

23. Page Analysis from the Internet
DNS does not play a role in slowness. Connection time varies and at time approaches 200 milliseconds which can be at the platform, internet, network, load
balancer or firewalls. Connection delay analysis will require multiple capture points to definitively pinpoint and should be considered when multi-point
capture test points can be configured at the Security Tap devices. But that is not material for improvement of this application at this timeFirst byte time is
the most concerning issue in the infrastructure. Last byte time is also a concern as it appears that platform TCP/IP stack services are slow to move data out
onto the wire after the first byte has started. It may also be that platform improvements may improve both response times and output speed. Page load
time is a composite of all elements of the page that must come together to provide the user with the visual page and the main context of the query. This
too is concerning, but it is caused by the slowness of the individual components of the page as they add serially to the response time which are represented
in the main concerns. An example of the total page would be small visual images and data making up the user interface view (i.e., logos) that are not part of
a computational or lookup, but rather a static image that should be served rapidly by the server.

24. Network Intrinsic Application Analysis

25. Multi-tier Analysis

26. Multi-Tier Identification

27. Application Monitoring Design Phase

28. Multi-tier Macro vs. Micro
Event
Process
Net-Ser-Tr-Sw-Q
Security Auth
User Click
Client
Network
WebSvr
Network
AppSrv
Network
SQLSvr
Network
AppSvr
Network
Mainframe
Network
AppSvr
Network
WebSvr
Network
Client
User Display UpdateMacro Response
Time
Micro
Response
Time

29. HTTP Post
from client
Web1
Middlewa
re
155ms
HTTP / SQL Multi-tier 1

30. Back to client
With HTTP
SQL Calls complete
Query and returns
Rows to Web1
SQL Calls finish .497
SQL Call start -.231
SQL Resp Time =.266
Web1
Middleware
12ms
HTTP / SQL Multi-tier 2

31. Logon A is 72
milliseconds…
Logon B is 420
milliseconds!
Oracle Logon Slow

32. Micro-Analysis Phase
Web App I/F #1&2 SQL TransLogger MF#1 MF#2 Time
Breakdown

33. TCP Satellite Retrans 3.5 Seconds

34. Processing Analysis

35. Packet Loss Analysis

36. Citrix Session Abort Signature “Chernobyl Packet”
The packet that
evidenced a problem
on a Citrix server.
This pattern was used
as a signature on the
Infinistream Sniffers
to find these
problems until they
were remediated.
Prior to this users
were stuck in this
cycle for hours.

37. Citrix User Filer Access Error Details

38. Blind vs. Pinpoint Upgrades
Blind Upgrade = Shotgun Approach = Forklift Upgrade

39. Root Cause Optimization
Definitive Root Cause Analysis Pinpoint Cause Measure ROI Potential
Pinpoint Purchases Validate & Prove ROI Award Innovation
Optimization
Root
Cause
Analysis

40. IT Critical Problem Resolution
Technology and Psychology
bill@apalytics.com