SlideShare a Scribd company logo
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HP Network Virtualization
(NV) Enablement
– How to POC
Matthew Brady – EMEA NV Pre-Sales
15th May 2014
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
POC Overview
Scoping (Discovery)
Objectives
Pre-Requisites
How to Select Network Scenarios
What to Cover
How to Test
How to Report
Follow Up
Agenda
Content Intended as Aide Memoire/Checklist
- Won’t Cover All Points
- Questions/Feedback Throughout!
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Overview
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4
Proof of Concept
“A proof of concept (POC) … is a realization of a certain method or idea to
demonstrate its feasibility, or a demonstration in principle, whose purpose
is to verify that some concept or theory has the potential of being used. A
proof of concept is usually small and may or may not be complete”
- Wikipedia
What is a POC?
• Not a Trial (Self-Guided)
• Not a Pilot (Actual Project Deliverables)
• Limited in Scope and Length
• Low number of vusers
• 2-3 days total time
• Single Application
• Limited Journeys
• Usually Assisted
Give-Get?
• Sponsor Required
• Conditional Commitment/Access to Power
• Final Results Presented to Power
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Scoping
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6
Overall
• What Is The Name And Description Of The Application To Be Covered By The POC?
• Who Will Be Involved?
• Where Will It Be Run, Using What Resources?
Architecture/Technology
• What Is The Outline Architecture Of The Application (Overview Only Is Required)?
• Which Client Platforms Will Be Used (Windows/Android/Ios)? Which Client Types
Should Be Included In The POC Test Activities?
• How Will It Be Delivered To Users (Web Browser, Client Application, Mobile App Etc.)
And Using Which Technologies (HTML5/JS/.Net/Java Etc.)?
Scoping Questions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7
Geography/Connectivity
• Where Will It Be Hosted When In Production?
• Who Will Be The End-users? Where Will They Be Located? How Will They Be
Connected?
• Is The Client Communication Encrypted (Https)?
• If Mobile Users, Which Operators And Connection Types (2.5g, 3g, 3.5g, 4g)?
• If Web Users, Which Connection Types (Dialup, Dsl, Cable, Satellite, Other)?
Types Of Testing/Tools
• Single-user Functional And/Or Multi-user Non-functional
• Manual, UFT, LR/PC, Sprinter And/Or SV
• Desktop, Web And/Or On-device
Scoping Questions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8
Scripting/Automation
• Do LR Scripts Exist For The Application Already? If So, How Many Are There? What
Protocols Do They Use? Which Ones Will Be Selected For The POC?
• If Lr Scripts Do Not Exist, When Will They Be Available? Which Protocols Will They
Use?
• Which Lr Version Is Currently Installed? Will The POC Use This Environment Or Will A
New Environment Be Setup For The POC? What OS Is Used For LR Controllers/Load
Generators?
Objectives
• What Is The Test Objective?
• 5000 Concurrent Users, 100k Transaction Per Hour, Functional Tests?
• How Will Test Success Be Determined (E.g. Response Times, Server CPU Etc.)?
• What Slas Or Other Requirements Exist?
Scoping Questions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Objectives
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10
NV With LR/PC
• Record/Sample Remote Network Conditions
• Configure And Apply Network Conditions In Tests, Including Additional Reporting
Capabilities
• Analyse The Impact Of Remote Users On System Load
• Analyse And Optimize The Application Flows For Performance
Success Criteria
Client Objectives – Multi User Testing
Criterion Status after POC
Install, license and configure HP NV software components Complete
Demonstrate HP Network Catcher to profile/sample mobile network
conditions
Complete
Execute automated tests using HP NV for HP to apply network
conditions
Complete
Demonstrate additional HP NV reporting and metrics Complete
Demonstrate using HP NV Analytics to optimize and improve
performance of application flows
Complete
Provide comparison report and analyse comparative throughput of
network virtualization in tests
Complete
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11
NV For Desktop/Mobile
• Install, License And Deploy Of HP NV For Mobile
• Record/Sample Mobile Network Conditions For UK Consumers
• Apply Network Conditions, Time Transactions And Capture Traffic Flowing From On-device
Tests
• Analyse And Optimize The Application Flows For Performance Including HTTPS Decoding
• Demonstrate Using API/Command Line To Control Network Conditions For Automation
Success Criteria
Client Objectives – Single User Testing
Criterion Status after POC
Install, license and configure HP NV software components Complete
Demonstrate HP Network Catcher to profile/sample mobile network
conditions
Complete
Execute manual and automated on-device tests using HP NV for
Mobile to apply network conditions
Complete
Demonstrate additional HP NV reporting and metrics Complete
Demonstrate using HP NV Analytics to optimize and improve
performance of application flows
Complete
Provide comparison report, and analyse comparative throughput of
network virtualization in tests
Complete
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12
1. Ensure Successful POC
• Keep it Simple, Well-defined Scope
• Ensure S/W Pre-installed
• Check & Verify Objectives, Prerequisites
• Limit Time/Effort but Provide Maximum Results
2. Demonstrate Maximum Value/Impact
• Work Towards the Final Report – Not Client-Defined Schedule
• Carefully Select Network Test Scenarios – Include Worst Cases
• Opt For Poorer Network Conditions
3. Progress Sales Process
• Agree Success Criteria Before POC
• Demonstrate Compliance to Success Criteria
• Provide Detailed Report With Executive Summary
• Present Results to Executive Team
HP Objectives?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
POC Prerequisites
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14
Environment
• Identify Testing Servers
• Controller(s), Load-Gens
• Scripting Workstation
• Testing Workstation
• Check Admin Rights
• Check Server Specification/OS Version etc.
Resources
• When can the Application be tested? Will it remain unchanged through the POC?
• Required Additional Software (Java, Silverlight)
• Internet Connectivity (for NC Global Library etc.)
• Software Download(s)
• Demo Licenses & Extensions
Prerequisites
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15
LoadGen
All Protocols Except Citrix
NV for LR/PC
Networ
k NV-M
LoadGen
NV-M
LoadGen
NV-M
NV-M
(Controller
Component
)
NV-M
(PC Server
Component
)
* 1 day minimum checkout period
LM??
License
NV
Analytics
NV
Predictor
(+MS
Office)
Controller PC Server
Internet
OR
OR
OR
EITHER
NC Global DB
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16
Desktop & Web Apps, Manual or Automated Testing (UFT, Sprinter etc.)
NV for Mobile (Desktop Mode)
Networ
k
App Under
TestNV-M
License
s
LM?
NV
Analytics
EITHER
OR
NV
TM
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17
On-Device Native, Hybrid & Web Apps
NV for Mobile (Mobile Mode)
Networ
k
Proxy
New Wifi
Hotspot
On-Device Web
Interface
Desktop Web
Console
NV-M
* 1 day minimum checkout period
NV
Analytics
SSL Key
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Select Network
Test Scenarios
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Let’s Begin with a Bit of Theory
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
The Network Impact
20
Tokyo
LondonNew York
New York
Data Center
WAN
Consider 3 Users - 1 in NY, 1 in London And 1 in Tokyo
• All Users Access the Same Application in the NY Data Center
• Will They All Experience the Application in the Same Way?
• If Not, Why Not?
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21
Network Impairments
21
Conditions which inhibit the flow of data across a network
• Key Impairments are:
• Network Delay (Latency)
• Jitter (Delay variation, inter-packet gap variation)
• Packet Loss (inverse of delivery rate)
• Bandwidth Constraints
• Bandwidth Utilization
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22
Mobile Network Impairments
22
Mobile Networks Experience Extreme Conditions of the Same Net
Impairments
• More Network Delay (Last Mile + Backhaul, RRC latency)
• More Jitter (Delay variation, inter-packet gap variation)
• More Packet Loss (inverse of delivery rate)
• More Bandwidth Constraints
• More Bandwidth Utilization
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23
Why is There Delay on a Terrestrial Network?
23
A Bt (msec) = d / (2/3)c
Signal
d
Packet PacketPacket Packet Packet
BUSY
Lost Packet
Network Delay
• Propagation Delay
• Processing / Insertion / Serialization Delay
• Queueing Delay
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24
New York
London
Frankfurt
Tokyo
Sydney
Signal Speed - Geography
24
• Speed of Light = 299,792.458 km/s
New
York
London Frankfurt Tokyo Sydney
New York 5,585km 6,219km 10,871km 15,998km
London 639km 9,585km 16,991km
Frankfurt 9,357km 16,480km
Tokyo 7,793km
New
York
London Frankfurt Tokyo Sydney
New York 28ms 31ms 54ms 80ms
London 3ms 48ms 85ms
Frankfurt 47ms 82ms
Tokyo 39ms
T = D / (2/3)c
Distance
Theoretical
Minimum
Signal Time
(One Way)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25
25
Combination of:
• Server Connect Latency (First Mile)
• Back-Haul/Propagation Latency
• Client Connect Latency (Last Mile)
Total Latency
First Mile
Last Mile
Propagation Latency
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.26
Internet Consumer (Last Mile) Latency
26
Connection
Type
RTT
Fiber 17ms
Cable 28ms
DSL 44ms
http://www.fcc.gov/measuring-broadband-america
Broadband Latency (FCC)
• “Fiber-to-the-home services provided 17 ms round-trip latency on average, while
cable-based services averaged 28 ms, and DSL-based services averaged 44 ms.”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.27
Mobile Latency
Mobile
Cell Tower
Mobile
Last Mile
50ms to 800ms
Up to 50% Packet Loss
BW 56Kbps to 20Mbps
Origin
Server
Mobile
Device
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.28
Mobile Latency – 1st Data point
28
http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf
Ofcom study, UK Mobile Networks, May 2012
Static Tests:
• “Latency… was an average of 117 milliseconds. Nearly two thirds of tests took more than 100
milliseconds”
User Panel:
• “Average latency was 192 milliseconds and around a third of tests delivered latency of higher than 200
milliseconds”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.29
Mobile Latency – 2nd Data point
29
Cisco Cloud Index, 2010-2015
“Global average fixed latency is 125ms and average mobile latency is 290ms… Global
mobile consumer average latency is 307.3ms”
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.30
Latency – Summary of Averages
30
0
50
100
150
200
250
300
350
Fiber Cable DSL Mobile
(Ofcom Tests)
Mobile
(Ofcom
Panel)
Mobile
(Cisco)
Mobile
Consumer
(Cisco)
Latency by Connec on Type
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.31
Packet Loss
31
Random Early Detection
Average queue monitored size & packets
dropped based on statistical probabilities. If
buffer is almost empty, all incoming packets are
accepted. As queue grows, probability of
dropping incoming packet grows. When buffer is
full, probability reaches 1 and all incoming
packets are dropped.
Packets get dropped on an end to end link due to the following:
• Network congestion causes queues to saturate
• Network congestion triggers RED
• Interference on wireless networks
• Hardware errors
• Firewall Policies
Packet loss is the inverse of delivery rate
100% - packet loss rate = delivery rate
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.32
Packet Loss
32
http://pam2012.ftw.at/papers/PAM2012paper6.pdf
Packet Loss, Akamai/MIT Research
• Tested 149 Million connections in 51 countries
• Compared PL for mobile networks from non-mobile networks
• Most connections have no packet loss (Median PL is 0%).
• 75th percentile usually zero too
• Average connection packet loss: 2.5%
• Fixed network average is 2.2%
• Implies connections are usually fine, but sometimes terrible
• The average PL for the non-zero half is 5%
• Suggests “small” connections have higher Packet Loss
• HTTP Connections are often small
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.33
Jitter
33
Jitter
No Jitter
Jitter-Sensitive Applications:
• Video
• VoIP
Jitter = Variability over time of latency across a network
• PDV – Packet Delay Variation
Constant Latency = Zero Jitter
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.34
Mobile Bandwidth in Theory
34
Protocol Max Downlink Max Uplink
GPRS 56-115 Kbps
CDMA200 (1X) 153.6 Kbps
Edge 236.8 Kbps
3G Bar 2 Mbps stationary client
384 Kbps moving client
EV-DO 2.5-4.9 Mbps 0.15-1.8 Mbps
EDGE Evolution 1.6 Mbps 0.5 Mbps
HSDPA 2 Mbps 200 Kbps
HSPA+ 14.4 Mbps
4G Bar 1 Gbps stationary client
100 Mbps moving client
HSPA+ Release 11 21-672 Mbps 5.8-168 Mbps
LTE 100-300 Mbps 50-75 Mbps
WiMax 37-365 Mbps 17-376 Mbps
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.35
Mobile Bandwidth
35
You only get those speeds it if you’re this
guy…
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.36
Mobile Bandwidth
36
Source (PCWorld): http://bit.ly/JX7MQw
PCWorld/Novarum
3G
• Download: 0.59 – 3.84 Mbps
• Upload: 0.56 – 1.44 Mbps
4G
• Download: 2.81 – 9.12 Mbps
• Upload: 0.97 – 5.86 Mbps
Date: Apr, 2013
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.37
Download BW Impact on Load Time
DSL
Cable FIBER
Poor Cell Conn
Avg Cell Conn
Load Time (ms)
Download BW (Kbps)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.38
Upload BW Impact on Load Time
DSL
Cable FIBER
Poor Cell Conn Avg Cell Conn
Load Time (ms)
Upload BW (Kbps)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.39
Latency Impact
DSL
Cable
FIBER
Poor Cell Conn
Avg Cell Conn
Load Time (ms)
Latency (ms)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.40
Network Profiles – Conclusions for Performance
Upload BW doesn’t matter much until <200 kbps
Download BW Matters at Low Values <1Mbps
• Unless Transfer Volume (Data) is Large
Latency matters the most
Packet Loss is Important but causes variation in results
• Only use in Worst Network Conditions
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Select Network Test
Scenarios
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.42
1. Determine the Full Range of Client Connection Types, for example:
• Remote Office Tokyo, London
• Consumer Web UK, Germany
• Consumer Mobile 2.5G India, UK, Germany
• Consumer Mobile 3G UK, Germany, Japan
• Consumer Mobile 4G UK, Japan
2. Select Test Range of Network Conditions (Not All):
• Remote Office London
• Remote Office Tokyo
• Consumer Web Europe Typical
• Consumer Mobile 4G UK Best
• Consumer Mobile 3G Europe Typical
• Consumer Mobile 2.5G India Worst
Selecting the Network Values for a POC
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.43
1. Work Out One-way Propagation Delay Server To Clients
• Calculate KM Distance/200,000
2. Attempt To Validate Using Ping Tests
• Or Use Other Tools – Try To Exclude The Last Mile
3. Demonstrate Mechanisms To Capture Network Conditions …
• NC Server/Agents
• NC App
• NC Global Library
• Other Tools/Data
But Do Not Use The Results In The POC
Selecting the Network Values for a POC
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.44
5. Select Test Range of Network Conditions (Not All):
• Remote Office London
• Remote Office Tokyo
• Consumer Web Europe Typical
• Consumer Mobile 4G UK Best
• Consumer Mobile 3G Europe Typical
• Consumer Mobile 2.5G India Worst
6. … Which Are:
• Representative
• Predictable
• In Sequence
Selecting the Network Values for a POC (cont.)
Use:
• Manual, Static Definitions
• Stepped Latency & Bandwidth (Equally Spread?)
• Static (Zero?) Packet Loss (Except Final Case)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.45
Example Network Test Profiles
• Baseline 0ms 100Mbps/100Mbps 0% PL
• Remote Office London 20ms 10Mbps/10Mbps 0% PL
• Consumer Web Europe Typical 50ms 5Mbps/2Mbps 0% PL
• Consumer Mobile 4G UK Best 80ms 10Mbps/2Mbps 0% PL
• Remote Office Tokyo 100ms 10Mbps/10Mbps 0% PL
• Consumer Mobile 3G Europe Typical 130ms 2Mbps/512kbps 0% PL
• Consumer Mobile 2.5G India Worst 250ms 512kbps/128kbps 1% PL
Notes:
• Always One Way Latency in NV
• Baseline Should Have BW Defined to Ensure Stability Between Tests
• Network Profiles Should be Common Between Test Types (Functional/Non-functional)
Selecting the Network Values for a POC (cont.)
(Too Many?)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What to Cover
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.47
Capabilities
Network Discovery
• (NC Server/Agents – Avoid)?
• NC Global DB
• NC Mobile App
• Other Tools
Single User Test - Desktop
• Define Network Conditions
• Proxy (SSL/HTTPS)?
• Transaction Manager
• (Integration with UFT Etc. – Avoid?)
Single User Test - Mobile
• Define Network Conditions
• Proxy (SSL/HTTPS)
• Web Console/Mobile Web
Multi User Test
• Define Network Conditions
• Runtime Metrics
• LR Analysis
• NV Predictor
• (Packet Capture – NV Analytics – Avoid?)
• (API Transaction Marking – Avoid?)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Test
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.49
Check SSL/HTTPS
• Use Proxy (Always Needed for Mobile)
• Verify Decryption Works (Doesn’t Always)
Assess Test Steps
• Analyse User Journeys using NV Transaction Manager
• Use NV Analytics to Focus on Network-Impacted Steps
• Create Modified (Focussed) TM Steps for Network Testing
Execute Tests
• Baseline
• Network Models
Test Process – Single User
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.50
Remember
• Save .Shunra files AND TM (Timing) Exports
• Unless using Mobile Console
• Good Naming Conventions!
Gotchas …
• Check the Adapter
• Connect TM!
• TM Marking (Before 1st/After Last Packet)
• TM Times vs NV Analytics Times
Test Process – Single User (cont.)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.51
Check SSL/HTTPS
• If Yes – Avoid Packet Capture During Test
• Use Manual Test?
Check Script Timings
• Script Think Time
• Paced vs Real-World?
• Ramp
Design Test Scenario
• Add/Define Groups to Align with NV Virtual Locations
Execution Sequence:
1. Full Test w/o NV
2. Low-Volume Test with NV
3. Full Test with NV
Test Process – Multi User (LR/PC)
Will Network Make Any Difference to Capacity?
• 0.1s Baseline Execution Time – 25s Think Time
• 0.2s Baseline Execution Time – 300s Pacing
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.52
Remember
• Save Scenarios
• Save Results & Analysis Files
• Document Network Profiles (Export?)
Gotchas
• “Local User” – Required For LR/PC Analysis
• Blank Virtual Location – For NV Predictor
• BW per Virtual Location vs Per Vuser
• Packet Capture
• Transaction Marking
Test Process – Multi User (LR/PC – cont.)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
How to Report
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.54
Example Docs
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Follow Up
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.56
Presentation
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Q&A

More Related Content

What's hot

Why a Mobile Test Strategy is just Test Strategy
Why a Mobile Test Strategy is just Test StrategyWhy a Mobile Test Strategy is just Test Strategy
Why a Mobile Test Strategy is just Test Strategy
Ho Chi Minh City Software Testing Club
 
Continuous Delivery Presentation
Continuous Delivery PresentationContinuous Delivery Presentation
Continuous Delivery Presentation
Mauricio Ferreyra
 
What is DevOps?
What is DevOps?What is DevOps?
What is DevOps?
Mesut Güneş
 
Testing Frameworks And Methodologies
Testing Frameworks And MethodologiesTesting Frameworks And Methodologies
Testing Frameworks And MethodologiesSteven Cahill
 
How to Introduce Continuous Delivery
How to Introduce Continuous DeliveryHow to Introduce Continuous Delivery
How to Introduce Continuous Delivery
Dr. Alexander Schwartz
 
5 Key Metrics to Release Better Software Faster
5 Key Metrics to Release Better Software Faster5 Key Metrics to Release Better Software Faster
5 Key Metrics to Release Better Software Faster
Dynatrace
 
Declarative Network Configuration
Declarative Network Configuration Declarative Network Configuration
Declarative Network Configuration
Salesforce Engineering
 
Digital Assurance - Today & Tomorrow
Digital Assurance - Today & TomorrowDigital Assurance - Today & Tomorrow
Digital Assurance - Today & Tomorrow
Agile Testing Alliance
 
Continuous integration testing fundamentals
Continuous integration testing fundamentalsContinuous integration testing fundamentals
Continuous integration testing fundamentalsCygnet Infotech
 
Effective performance evaluation as part of a CI approach - Mission Impossible?
Effective performance evaluation as part of a CI approach - Mission Impossible?Effective performance evaluation as part of a CI approach - Mission Impossible?
Effective performance evaluation as part of a CI approach - Mission Impossible?
Intechnica
 
Gabriel carabat a healthy approach for test automation
Gabriel carabat   a healthy approach for test automationGabriel carabat   a healthy approach for test automation
Gabriel carabat a healthy approach for test automationRomania Testing
 
Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)
Leonard Fingerman
 
Automation testing in Agile project
Automation testing in Agile projectAutomation testing in Agile project
Automation testing in Agile project
Hien Nguyen
 
Introduction to Test Automation - Technology and Tools
Introduction to Test Automation - Technology and ToolsIntroduction to Test Automation - Technology and Tools
Introduction to Test Automation - Technology and Tools
KMS Technology
 
An Introduction to Iterative Software Development
An Introduction to Iterative Software DevelopmentAn Introduction to Iterative Software Development
An Introduction to Iterative Software DevelopmentGeoffrey Weglarz
 
Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
Lee Barnes
 
System analysis methodologies
System analysis methodologiesSystem analysis methodologies
System analysis methodologies
uppipisara
 
Automated Testing with Agile
Automated Testing with AgileAutomated Testing with Agile
Automated Testing with Agile
Ken McCorkell
 
How, why, and the roi kcdc '21
How, why, and the roi  kcdc '21How, why, and the roi  kcdc '21
How, why, and the roi kcdc '21
Marcus Merrell
 
Certified Professional Master Agile Testing information and highlights
Certified Professional Master Agile Testing  information and highlightsCertified Professional Master Agile Testing  information and highlights
Certified Professional Master Agile Testing information and highlights
Agile Testing Alliance
 

What's hot (20)

Why a Mobile Test Strategy is just Test Strategy
Why a Mobile Test Strategy is just Test StrategyWhy a Mobile Test Strategy is just Test Strategy
Why a Mobile Test Strategy is just Test Strategy
 
Continuous Delivery Presentation
Continuous Delivery PresentationContinuous Delivery Presentation
Continuous Delivery Presentation
 
What is DevOps?
What is DevOps?What is DevOps?
What is DevOps?
 
Testing Frameworks And Methodologies
Testing Frameworks And MethodologiesTesting Frameworks And Methodologies
Testing Frameworks And Methodologies
 
How to Introduce Continuous Delivery
How to Introduce Continuous DeliveryHow to Introduce Continuous Delivery
How to Introduce Continuous Delivery
 
5 Key Metrics to Release Better Software Faster
5 Key Metrics to Release Better Software Faster5 Key Metrics to Release Better Software Faster
5 Key Metrics to Release Better Software Faster
 
Declarative Network Configuration
Declarative Network Configuration Declarative Network Configuration
Declarative Network Configuration
 
Digital Assurance - Today & Tomorrow
Digital Assurance - Today & TomorrowDigital Assurance - Today & Tomorrow
Digital Assurance - Today & Tomorrow
 
Continuous integration testing fundamentals
Continuous integration testing fundamentalsContinuous integration testing fundamentals
Continuous integration testing fundamentals
 
Effective performance evaluation as part of a CI approach - Mission Impossible?
Effective performance evaluation as part of a CI approach - Mission Impossible?Effective performance evaluation as part of a CI approach - Mission Impossible?
Effective performance evaluation as part of a CI approach - Mission Impossible?
 
Gabriel carabat a healthy approach for test automation
Gabriel carabat   a healthy approach for test automationGabriel carabat   a healthy approach for test automation
Gabriel carabat a healthy approach for test automation
 
Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)Test Automation Best Practices (with SOA test approach)
Test Automation Best Practices (with SOA test approach)
 
Automation testing in Agile project
Automation testing in Agile projectAutomation testing in Agile project
Automation testing in Agile project
 
Introduction to Test Automation - Technology and Tools
Introduction to Test Automation - Technology and ToolsIntroduction to Test Automation - Technology and Tools
Introduction to Test Automation - Technology and Tools
 
An Introduction to Iterative Software Development
An Introduction to Iterative Software DevelopmentAn Introduction to Iterative Software Development
An Introduction to Iterative Software Development
 
Performance Testing in the Agile Lifecycle
Performance Testing in the Agile LifecyclePerformance Testing in the Agile Lifecycle
Performance Testing in the Agile Lifecycle
 
System analysis methodologies
System analysis methodologiesSystem analysis methodologies
System analysis methodologies
 
Automated Testing with Agile
Automated Testing with AgileAutomated Testing with Agile
Automated Testing with Agile
 
How, why, and the roi kcdc '21
How, why, and the roi  kcdc '21How, why, and the roi  kcdc '21
How, why, and the roi kcdc '21
 
Certified Professional Master Agile Testing information and highlights
Certified Professional Master Agile Testing  information and highlightsCertified Professional Master Agile Testing  information and highlights
Certified Professional Master Agile Testing information and highlights
 

Viewers also liked

Planning your OpenStack PoC
Planning your OpenStack PoCPlanning your OpenStack PoC
Planning your OpenStack PoC
openstackstl
 
An example of a successful proof of concept
An example of a successful proof of conceptAn example of a successful proof of concept
An example of a successful proof of concept
ETLSolutions
 
How to Build a Proof of Concept
How to Build a Proof of Concept How to Build a Proof of Concept
How to Build a Proof of Concept
Michael Hamilton
 
Ads and-poc
Ads and-pocAds and-poc
Ads and-pocebuc
 
HPE presentation at SAPPHIRE 2016 in SUSE Mini-theatre
HPE presentation at SAPPHIRE 2016 in SUSE Mini-theatreHPE presentation at SAPPHIRE 2016 in SUSE Mini-theatre
HPE presentation at SAPPHIRE 2016 in SUSE Mini-theatre
Mike Nelson
 
Tsg Poc Dx Introductory Overview March2009 D
Tsg Poc Dx Introductory Overview March2009 DTsg Poc Dx Introductory Overview March2009 D
Tsg Poc Dx Introductory Overview March2009 D
jerrychoate
 
Poc exadata pres_doag_2015
Poc exadata pres_doag_2015Poc exadata pres_doag_2015
Poc exadata pres_doag_2015
Jacques Kostic
 
NIH SciENcv Proof of Concept
NIH SciENcv Proof of ConceptNIH SciENcv Proof of Concept
NIH SciENcv Proof of Concept
ORCID, Inc
 
Story-based Strategy Model: Grassroots Organizing Meets Narrative Power
Story-based Strategy Model: Grassroots Organizing Meets Narrative PowerStory-based Strategy Model: Grassroots Organizing Meets Narrative Power
Story-based Strategy Model: Grassroots Organizing Meets Narrative Power
Center for Story-based Strategy
 
ATLAS Automation POC
ATLAS Automation POCATLAS Automation POC
ATLAS Automation POCaakashmc
 
Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...
Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...
Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...
eazynow
 
Agados POC Report to Build/Rebuild for ERP PKG
Agados POC Report to Build/Rebuild for ERP PKG Agados POC Report to Build/Rebuild for ERP PKG
Agados POC Report to Build/Rebuild for ERP PKG
Yongkyoo Park
 
PoC: Using a Group Communication System to improve MySQL Replication HA
PoC: Using a Group Communication System to improve MySQL Replication HAPoC: Using a Group Communication System to improve MySQL Replication HA
PoC: Using a Group Communication System to improve MySQL Replication HA
Ulf Wendel
 
MVPOC - Minimum Viable Proof of Concept
MVPOC - Minimum Viable Proof of ConceptMVPOC - Minimum Viable Proof of Concept
MVPOC - Minimum Viable Proof of Concept
Ray DeLaPena
 
HP & NFV POC at SDN World Congree
HP & NFV POC at SDN World CongreeHP & NFV POC at SDN World Congree
HP & NFV POC at SDN World Congree
Marie-Paule Odini
 
Proof Of Concept Presentation on Concept
Proof Of Concept Presentation on ConceptProof Of Concept Presentation on Concept
Proof Of Concept Presentation on ConceptUniversity of Limerick
 
Proof of Concept Workshop
Proof of Concept WorkshopProof of Concept Workshop
Proof of Concept Workshop
Danny Holtschke
 

Viewers also liked (20)

Planning your OpenStack PoC
Planning your OpenStack PoCPlanning your OpenStack PoC
Planning your OpenStack PoC
 
An example of a successful proof of concept
An example of a successful proof of conceptAn example of a successful proof of concept
An example of a successful proof of concept
 
test_automation_POC
test_automation_POCtest_automation_POC
test_automation_POC
 
How to Build a Proof of Concept
How to Build a Proof of Concept How to Build a Proof of Concept
How to Build a Proof of Concept
 
Ads and-poc
Ads and-pocAds and-poc
Ads and-poc
 
HPE presentation at SAPPHIRE 2016 in SUSE Mini-theatre
HPE presentation at SAPPHIRE 2016 in SUSE Mini-theatreHPE presentation at SAPPHIRE 2016 in SUSE Mini-theatre
HPE presentation at SAPPHIRE 2016 in SUSE Mini-theatre
 
Tsg Poc Dx Introductory Overview March2009 D
Tsg Poc Dx Introductory Overview March2009 DTsg Poc Dx Introductory Overview March2009 D
Tsg Poc Dx Introductory Overview March2009 D
 
Poc exadata pres_doag_2015
Poc exadata pres_doag_2015Poc exadata pres_doag_2015
Poc exadata pres_doag_2015
 
Youthway on the MBTA
Youthway on the MBTAYouthway on the MBTA
Youthway on the MBTA
 
NIH SciENcv Proof of Concept
NIH SciENcv Proof of ConceptNIH SciENcv Proof of Concept
NIH SciENcv Proof of Concept
 
Story-based Strategy Model: Grassroots Organizing Meets Narrative Power
Story-based Strategy Model: Grassroots Organizing Meets Narrative PowerStory-based Strategy Model: Grassroots Organizing Meets Narrative Power
Story-based Strategy Model: Grassroots Organizing Meets Narrative Power
 
Planning open stack-poc
Planning open stack-pocPlanning open stack-poc
Planning open stack-poc
 
ATLAS Automation POC
ATLAS Automation POCATLAS Automation POC
ATLAS Automation POC
 
Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...
Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...
Twiliocon Europe 2013: From PoC to Production, Lessons Learnt, by Erol Ziya &...
 
Agados POC Report to Build/Rebuild for ERP PKG
Agados POC Report to Build/Rebuild for ERP PKG Agados POC Report to Build/Rebuild for ERP PKG
Agados POC Report to Build/Rebuild for ERP PKG
 
PoC: Using a Group Communication System to improve MySQL Replication HA
PoC: Using a Group Communication System to improve MySQL Replication HAPoC: Using a Group Communication System to improve MySQL Replication HA
PoC: Using a Group Communication System to improve MySQL Replication HA
 
MVPOC - Minimum Viable Proof of Concept
MVPOC - Minimum Viable Proof of ConceptMVPOC - Minimum Viable Proof of Concept
MVPOC - Minimum Viable Proof of Concept
 
HP & NFV POC at SDN World Congree
HP & NFV POC at SDN World CongreeHP & NFV POC at SDN World Congree
HP & NFV POC at SDN World Congree
 
Proof Of Concept Presentation on Concept
Proof Of Concept Presentation on ConceptProof Of Concept Presentation on Concept
Proof Of Concept Presentation on Concept
 
Proof of Concept Workshop
Proof of Concept WorkshopProof of Concept Workshop
Proof of Concept Workshop
 

Similar to HPE | Network Virtualization | POC

Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation
HP Enterprise Italia
 
Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery AutomationIl paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation
HP Enterprise Italia
 
Hp discover 2012 managing the virtualization explosion
Hp discover 2012   managing the virtualization explosionHp discover 2012   managing the virtualization explosion
Hp discover 2012 managing the virtualization explosion
Stefan Bergstein
 
Vasudevan_Bhaskar.pdf
Vasudevan_Bhaskar.pdfVasudevan_Bhaskar.pdf
Vasudevan_Bhaskar.pdf
ssuser57cef8
 
Introducing AppPulse
Introducing AppPulse Introducing AppPulse
Introducing AppPulse
Globe Testing
 
Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...
Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...
Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...
Marcelo Galvão
 
Optimizing Global Application Delivery Webinar
Optimizing Global Application Delivery WebinarOptimizing Global Application Delivery Webinar
Optimizing Global Application Delivery Webinar
Ginny Au
 
Webinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devicesWebinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devicesPerfecto Mobile
 
Hp perfecto webinar - UFT Mobile
Hp perfecto webinar - UFT MobileHp perfecto webinar - UFT Mobile
Hp perfecto webinar - UFT MobilePerfecto Mobile
 
Delivering Network Innovation with SDN - Tom Nadeau
Delivering Network Innovation with SDN - Tom Nadeau Delivering Network Innovation with SDN - Tom Nadeau
Delivering Network Innovation with SDN - Tom Nadeau
scoopnewsgroup
 
Service Management excellence with operational intelligence
Service Management excellence with operational intelligenceService Management excellence with operational intelligence
Service Management excellence with operational intelligence
HP Enterprise Italia
 
Virtualization: Improve Speed and Increase Quality
Virtualization: Improve Speed and Increase QualityVirtualization: Improve Speed and Increase Quality
Virtualization: Improve Speed and Increase Quality
TechWell
 
iKariera 2015
iKariera 2015iKariera 2015
iKariera 2015
Tomáš Muchka
 
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
Eneko Jon Bilbao
 
HP Helion Webinar #4 - Open stack the magic pill
HP Helion Webinar #4 - Open stack the magic pillHP Helion Webinar #4 - Open stack the magic pill
HP Helion Webinar #4 - Open stack the magic pill
BeMyApp
 
Hp helion meetup_networking_sdn
Hp helion meetup_networking_sdnHp helion meetup_networking_sdn
Hp helion meetup_networking_sdn
Marton Kiss
 
Developing applications for the cloud
Developing applications for the cloudDeveloping applications for the cloud
Developing applications for the cloud
Bart Blommaerts
 
Cloud,beyond the hype, looking at the journey to Cloud
Cloud,beyond the hype, looking at the journey to CloudCloud,beyond the hype, looking at the journey to Cloud
Cloud,beyond the hype, looking at the journey to Cloud
Christian Verstraete
 

Similar to HPE | Network Virtualization | POC (20)

Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation
 
Il paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery AutomationIl paradigma DevOps e Continuous Delivery Automation
Il paradigma DevOps e Continuous Delivery Automation
 
Hp discover 2012 managing the virtualization explosion
Hp discover 2012   managing the virtualization explosionHp discover 2012   managing the virtualization explosion
Hp discover 2012 managing the virtualization explosion
 
Vasudevan_Bhaskar.pdf
Vasudevan_Bhaskar.pdfVasudevan_Bhaskar.pdf
Vasudevan_Bhaskar.pdf
 
Introducing AppPulse
Introducing AppPulse Introducing AppPulse
Introducing AppPulse
 
Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...
Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...
Como a virtualização de serviços e de rede podem ajudar no desenvolvimento e ...
 
Updated resume
Updated resumeUpdated resume
Updated resume
 
Optimizing Global Application Delivery Webinar
Optimizing Global Application Delivery WebinarOptimizing Global Application Delivery Webinar
Optimizing Global Application Delivery Webinar
 
Webinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devicesWebinar effective mobile performance testing using real devices
Webinar effective mobile performance testing using real devices
 
Hp perfecto webinar - UFT Mobile
Hp perfecto webinar - UFT MobileHp perfecto webinar - UFT Mobile
Hp perfecto webinar - UFT Mobile
 
Delivering Network Innovation with SDN - Tom Nadeau
Delivering Network Innovation with SDN - Tom Nadeau Delivering Network Innovation with SDN - Tom Nadeau
Delivering Network Innovation with SDN - Tom Nadeau
 
Service Management excellence with operational intelligence
Service Management excellence with operational intelligenceService Management excellence with operational intelligence
Service Management excellence with operational intelligence
 
Virtualization: Improve Speed and Increase Quality
Virtualization: Improve Speed and Increase QualityVirtualization: Improve Speed and Increase Quality
Virtualization: Improve Speed and Increase Quality
 
iKariera 2015
iKariera 2015iKariera 2015
iKariera 2015
 
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
2015 Mastering SAP Tech - Enterprise Mobility - Testing Lessons Learned
 
HP Helion Webinar #4 - Open stack the magic pill
HP Helion Webinar #4 - Open stack the magic pillHP Helion Webinar #4 - Open stack the magic pill
HP Helion Webinar #4 - Open stack the magic pill
 
Musharraf_Syed_Resume
Musharraf_Syed_ResumeMusharraf_Syed_Resume
Musharraf_Syed_Resume
 
Hp helion meetup_networking_sdn
Hp helion meetup_networking_sdnHp helion meetup_networking_sdn
Hp helion meetup_networking_sdn
 
Developing applications for the cloud
Developing applications for the cloudDeveloping applications for the cloud
Developing applications for the cloud
 
Cloud,beyond the hype, looking at the journey to Cloud
Cloud,beyond the hype, looking at the journey to CloudCloud,beyond the hype, looking at the journey to Cloud
Cloud,beyond the hype, looking at the journey to Cloud
 

More from Jeffrey Nunn

Hewlett Packard Entreprise | Stormrunner load | Game Changer
Hewlett Packard Entreprise | Stormrunner load | Game ChangerHewlett Packard Entreprise | Stormrunner load | Game Changer
Hewlett Packard Entreprise | Stormrunner load | Game Changer
Jeffrey Nunn
 
Application Performance Management 9.30 HPE whats new | 360 View
Application Performance Management 9.30 HPE whats new | 360 ViewApplication Performance Management 9.30 HPE whats new | 360 View
Application Performance Management 9.30 HPE whats new | 360 View
Jeffrey Nunn
 
Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)
Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)
Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)
Jeffrey Nunn
 
HPE ALM Octane | DevOps | Agile
HPE ALM Octane | DevOps | AgileHPE ALM Octane | DevOps | Agile
HPE ALM Octane | DevOps | Agile
Jeffrey Nunn
 
HPE performance analytics
HPE performance analyticsHPE performance analytics
HPE performance analytics
Jeffrey Nunn
 
HPE Agile Manager and ALM Overview
HPE Agile Manager and ALM OverviewHPE Agile Manager and ALM Overview
HPE Agile Manager and ALM Overview
Jeffrey Nunn
 
Hewlett Packard Enterprise (HPE) Service Virtualization (SV)
Hewlett Packard Enterprise (HPE) Service Virtualization (SV)Hewlett Packard Enterprise (HPE) Service Virtualization (SV)
Hewlett Packard Enterprise (HPE) Service Virtualization (SV)
Jeffrey Nunn
 
HPE Agile Manager and Project and Portfolio Management PPM overview
HPE Agile Manager and Project and Portfolio Management PPM overview HPE Agile Manager and Project and Portfolio Management PPM overview
HPE Agile Manager and Project and Portfolio Management PPM overview
Jeffrey Nunn
 
Hpe service virtualization 3.8 what's new chicago adm
Hpe service virtualization 3.8 what's new chicago admHpe service virtualization 3.8 what's new chicago adm
Hpe service virtualization 3.8 what's new chicago adm
Jeffrey Nunn
 
HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016
HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016
HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016
Jeffrey Nunn
 
Fuctional Testing value brief
Fuctional Testing value briefFuctional Testing value brief
Fuctional Testing value briefJeffrey Nunn
 
Performance Value Brief
Performance Value BriefPerformance Value Brief
Performance Value BriefJeffrey Nunn
 
Values Jeffrey Nunn
Values Jeffrey NunnValues Jeffrey Nunn
Values Jeffrey Nunn
Jeffrey Nunn
 
Disc Jeffrey Nunn
Disc Jeffrey NunnDisc Jeffrey Nunn
Disc Jeffrey Nunn
Jeffrey Nunn
 

More from Jeffrey Nunn (17)

Hewlett Packard Entreprise | Stormrunner load | Game Changer
Hewlett Packard Entreprise | Stormrunner load | Game ChangerHewlett Packard Entreprise | Stormrunner load | Game Changer
Hewlett Packard Entreprise | Stormrunner load | Game Changer
 
Application Performance Management 9.30 HPE whats new | 360 View
Application Performance Management 9.30 HPE whats new | 360 ViewApplication Performance Management 9.30 HPE whats new | 360 View
Application Performance Management 9.30 HPE whats new | 360 View
 
Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)
Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)
Black Friday Performance Testing with HPE's Stormrunnerload 2016 (1)
 
HPE ALM Octane | DevOps | Agile
HPE ALM Octane | DevOps | AgileHPE ALM Octane | DevOps | Agile
HPE ALM Octane | DevOps | Agile
 
HPE performance analytics
HPE performance analyticsHPE performance analytics
HPE performance analytics
 
HPE Agile Manager and ALM Overview
HPE Agile Manager and ALM OverviewHPE Agile Manager and ALM Overview
HPE Agile Manager and ALM Overview
 
Hewlett Packard Enterprise (HPE) Service Virtualization (SV)
Hewlett Packard Enterprise (HPE) Service Virtualization (SV)Hewlett Packard Enterprise (HPE) Service Virtualization (SV)
Hewlett Packard Enterprise (HPE) Service Virtualization (SV)
 
HPE Agile Manager and Project and Portfolio Management PPM overview
HPE Agile Manager and Project and Portfolio Management PPM overview HPE Agile Manager and Project and Portfolio Management PPM overview
HPE Agile Manager and Project and Portfolio Management PPM overview
 
Hpe service virtualization 3.8 what's new chicago adm
Hpe service virtualization 3.8 what's new chicago admHpe service virtualization 3.8 what's new chicago adm
Hpe service virtualization 3.8 what's new chicago adm
 
HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016
HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016
HPE | Agile Manager and Project and Portfolio Management (PPM) overview feb 2016
 
ALM Value Brief
ALM Value BriefALM Value Brief
ALM Value Brief
 
Fuctional Testing value brief
Fuctional Testing value briefFuctional Testing value brief
Fuctional Testing value brief
 
NV Value Brief
NV Value BriefNV Value Brief
NV Value Brief
 
Performance Value Brief
Performance Value BriefPerformance Value Brief
Performance Value Brief
 
SV Value Brief
SV Value BriefSV Value Brief
SV Value Brief
 
Values Jeffrey Nunn
Values Jeffrey NunnValues Jeffrey Nunn
Values Jeffrey Nunn
 
Disc Jeffrey Nunn
Disc Jeffrey NunnDisc Jeffrey Nunn
Disc Jeffrey Nunn
 

Recently uploaded

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Globus
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
informapgpstrackings
 
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, BetterWebinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
XfilesPro
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
Globus
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
Ortus Solutions, Corp
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
Juraj Vysvader
 
Lecture 1 Introduction to games development
Lecture 1 Introduction to games developmentLecture 1 Introduction to games development
Lecture 1 Introduction to games development
abdulrafaychaudhry
 
Why React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdfWhy React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdf
ayushiqss
 
Software Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdfSoftware Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdf
MayankTawar1
 
Strategies for Successful Data Migration Tools.pptx
Strategies for Successful Data Migration Tools.pptxStrategies for Successful Data Migration Tools.pptx
Strategies for Successful Data Migration Tools.pptx
varshanayak241
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Globus
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Natan Silnitsky
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
Ortus Solutions, Corp
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
Globus
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
vrstrong314
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2
 
Advanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowAdvanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should Know
Peter Caitens
 

Recently uploaded (20)

Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
 
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...
 
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, BetterWebinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
Webinar: Salesforce Document Management 2.0 - Smarter, Faster, Better
 
Enhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdfEnhancing Research Orchestration Capabilities at ORNL.pdf
Enhancing Research Orchestration Capabilities at ORNL.pdf
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 
BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...
 
Lecture 1 Introduction to games development
Lecture 1 Introduction to games developmentLecture 1 Introduction to games development
Lecture 1 Introduction to games development
 
Why React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdfWhy React Native as a Strategic Advantage for Startup Innovation.pdf
Why React Native as a Strategic Advantage for Startup Innovation.pdf
 
Software Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdfSoftware Testing Exam imp Ques Notes.pdf
Software Testing Exam imp Ques Notes.pdf
 
Strategies for Successful Data Migration Tools.pptx
Strategies for Successful Data Migration Tools.pptxStrategies for Successful Data Migration Tools.pptx
Strategies for Successful Data Migration Tools.pptx
 
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...
 
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.ILBeyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
Beyond Event Sourcing - Embracing CRUD for Wix Platform - Java.IL
 
Into the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdfInto the Box 2024 - Keynote Day 2 Slides.pdf
Into the Box 2024 - Keynote Day 2 Slides.pdf
 
Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024Globus Compute Introduction - GlobusWorld 2024
Globus Compute Introduction - GlobusWorld 2024
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
top nidhi software solution freedownload
top nidhi software solution freedownloadtop nidhi software solution freedownload
top nidhi software solution freedownload
 
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital TransformationWSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
WSO2Con2024 - WSO2's IAM Vision: Identity-Led Digital Transformation
 
Advanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should KnowAdvanced Flow Concepts Every Developer Should Know
Advanced Flow Concepts Every Developer Should Know
 

HPE | Network Virtualization | POC

  • 1. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Network Virtualization (NV) Enablement – How to POC Matthew Brady – EMEA NV Pre-Sales 15th May 2014
  • 2. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2 POC Overview Scoping (Discovery) Objectives Pre-Requisites How to Select Network Scenarios What to Cover How to Test How to Report Follow Up Agenda Content Intended as Aide Memoire/Checklist - Won’t Cover All Points - Questions/Feedback Throughout!
  • 3. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. POC Overview
  • 4. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4 Proof of Concept “A proof of concept (POC) … is a realization of a certain method or idea to demonstrate its feasibility, or a demonstration in principle, whose purpose is to verify that some concept or theory has the potential of being used. A proof of concept is usually small and may or may not be complete” - Wikipedia What is a POC? • Not a Trial (Self-Guided) • Not a Pilot (Actual Project Deliverables) • Limited in Scope and Length • Low number of vusers • 2-3 days total time • Single Application • Limited Journeys • Usually Assisted Give-Get? • Sponsor Required • Conditional Commitment/Access to Power • Final Results Presented to Power
  • 5. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. POC Scoping
  • 6. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6 Overall • What Is The Name And Description Of The Application To Be Covered By The POC? • Who Will Be Involved? • Where Will It Be Run, Using What Resources? Architecture/Technology • What Is The Outline Architecture Of The Application (Overview Only Is Required)? • Which Client Platforms Will Be Used (Windows/Android/Ios)? Which Client Types Should Be Included In The POC Test Activities? • How Will It Be Delivered To Users (Web Browser, Client Application, Mobile App Etc.) And Using Which Technologies (HTML5/JS/.Net/Java Etc.)? Scoping Questions
  • 7. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7 Geography/Connectivity • Where Will It Be Hosted When In Production? • Who Will Be The End-users? Where Will They Be Located? How Will They Be Connected? • Is The Client Communication Encrypted (Https)? • If Mobile Users, Which Operators And Connection Types (2.5g, 3g, 3.5g, 4g)? • If Web Users, Which Connection Types (Dialup, Dsl, Cable, Satellite, Other)? Types Of Testing/Tools • Single-user Functional And/Or Multi-user Non-functional • Manual, UFT, LR/PC, Sprinter And/Or SV • Desktop, Web And/Or On-device Scoping Questions
  • 8. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8 Scripting/Automation • Do LR Scripts Exist For The Application Already? If So, How Many Are There? What Protocols Do They Use? Which Ones Will Be Selected For The POC? • If Lr Scripts Do Not Exist, When Will They Be Available? Which Protocols Will They Use? • Which Lr Version Is Currently Installed? Will The POC Use This Environment Or Will A New Environment Be Setup For The POC? What OS Is Used For LR Controllers/Load Generators? Objectives • What Is The Test Objective? • 5000 Concurrent Users, 100k Transaction Per Hour, Functional Tests? • How Will Test Success Be Determined (E.g. Response Times, Server CPU Etc.)? • What Slas Or Other Requirements Exist? Scoping Questions
  • 9. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. POC Objectives
  • 10. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10 NV With LR/PC • Record/Sample Remote Network Conditions • Configure And Apply Network Conditions In Tests, Including Additional Reporting Capabilities • Analyse The Impact Of Remote Users On System Load • Analyse And Optimize The Application Flows For Performance Success Criteria Client Objectives – Multi User Testing Criterion Status after POC Install, license and configure HP NV software components Complete Demonstrate HP Network Catcher to profile/sample mobile network conditions Complete Execute automated tests using HP NV for HP to apply network conditions Complete Demonstrate additional HP NV reporting and metrics Complete Demonstrate using HP NV Analytics to optimize and improve performance of application flows Complete Provide comparison report and analyse comparative throughput of network virtualization in tests Complete
  • 11. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11 NV For Desktop/Mobile • Install, License And Deploy Of HP NV For Mobile • Record/Sample Mobile Network Conditions For UK Consumers • Apply Network Conditions, Time Transactions And Capture Traffic Flowing From On-device Tests • Analyse And Optimize The Application Flows For Performance Including HTTPS Decoding • Demonstrate Using API/Command Line To Control Network Conditions For Automation Success Criteria Client Objectives – Single User Testing Criterion Status after POC Install, license and configure HP NV software components Complete Demonstrate HP Network Catcher to profile/sample mobile network conditions Complete Execute manual and automated on-device tests using HP NV for Mobile to apply network conditions Complete Demonstrate additional HP NV reporting and metrics Complete Demonstrate using HP NV Analytics to optimize and improve performance of application flows Complete Provide comparison report, and analyse comparative throughput of network virtualization in tests Complete
  • 12. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12 1. Ensure Successful POC • Keep it Simple, Well-defined Scope • Ensure S/W Pre-installed • Check & Verify Objectives, Prerequisites • Limit Time/Effort but Provide Maximum Results 2. Demonstrate Maximum Value/Impact • Work Towards the Final Report – Not Client-Defined Schedule • Carefully Select Network Test Scenarios – Include Worst Cases • Opt For Poorer Network Conditions 3. Progress Sales Process • Agree Success Criteria Before POC • Demonstrate Compliance to Success Criteria • Provide Detailed Report With Executive Summary • Present Results to Executive Team HP Objectives?
  • 13. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. POC Prerequisites
  • 14. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14 Environment • Identify Testing Servers • Controller(s), Load-Gens • Scripting Workstation • Testing Workstation • Check Admin Rights • Check Server Specification/OS Version etc. Resources • When can the Application be tested? Will it remain unchanged through the POC? • Required Additional Software (Java, Silverlight) • Internet Connectivity (for NC Global Library etc.) • Software Download(s) • Demo Licenses & Extensions Prerequisites
  • 15. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15 LoadGen All Protocols Except Citrix NV for LR/PC Networ k NV-M LoadGen NV-M LoadGen NV-M NV-M (Controller Component ) NV-M (PC Server Component ) * 1 day minimum checkout period LM?? License NV Analytics NV Predictor (+MS Office) Controller PC Server Internet OR OR OR EITHER NC Global DB
  • 16. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16 Desktop & Web Apps, Manual or Automated Testing (UFT, Sprinter etc.) NV for Mobile (Desktop Mode) Networ k App Under TestNV-M License s LM? NV Analytics EITHER OR NV TM
  • 17. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17 On-Device Native, Hybrid & Web Apps NV for Mobile (Mobile Mode) Networ k Proxy New Wifi Hotspot On-Device Web Interface Desktop Web Console NV-M * 1 day minimum checkout period NV Analytics SSL Key
  • 18. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. How to Select Network Test Scenarios
  • 19. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Let’s Begin with a Bit of Theory
  • 20. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The Network Impact 20 Tokyo LondonNew York New York Data Center WAN Consider 3 Users - 1 in NY, 1 in London And 1 in Tokyo • All Users Access the Same Application in the NY Data Center • Will They All Experience the Application in the Same Way? • If Not, Why Not?
  • 21. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21 Network Impairments 21 Conditions which inhibit the flow of data across a network • Key Impairments are: • Network Delay (Latency) • Jitter (Delay variation, inter-packet gap variation) • Packet Loss (inverse of delivery rate) • Bandwidth Constraints • Bandwidth Utilization
  • 22. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22 Mobile Network Impairments 22 Mobile Networks Experience Extreme Conditions of the Same Net Impairments • More Network Delay (Last Mile + Backhaul, RRC latency) • More Jitter (Delay variation, inter-packet gap variation) • More Packet Loss (inverse of delivery rate) • More Bandwidth Constraints • More Bandwidth Utilization
  • 23. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23 Why is There Delay on a Terrestrial Network? 23 A Bt (msec) = d / (2/3)c Signal d Packet PacketPacket Packet Packet BUSY Lost Packet Network Delay • Propagation Delay • Processing / Insertion / Serialization Delay • Queueing Delay
  • 24. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24 New York London Frankfurt Tokyo Sydney Signal Speed - Geography 24 • Speed of Light = 299,792.458 km/s New York London Frankfurt Tokyo Sydney New York 5,585km 6,219km 10,871km 15,998km London 639km 9,585km 16,991km Frankfurt 9,357km 16,480km Tokyo 7,793km New York London Frankfurt Tokyo Sydney New York 28ms 31ms 54ms 80ms London 3ms 48ms 85ms Frankfurt 47ms 82ms Tokyo 39ms T = D / (2/3)c Distance Theoretical Minimum Signal Time (One Way)
  • 25. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25 25 Combination of: • Server Connect Latency (First Mile) • Back-Haul/Propagation Latency • Client Connect Latency (Last Mile) Total Latency First Mile Last Mile Propagation Latency
  • 26. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.26 Internet Consumer (Last Mile) Latency 26 Connection Type RTT Fiber 17ms Cable 28ms DSL 44ms http://www.fcc.gov/measuring-broadband-america Broadband Latency (FCC) • “Fiber-to-the-home services provided 17 ms round-trip latency on average, while cable-based services averaged 28 ms, and DSL-based services averaged 44 ms.”
  • 27. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.27 Mobile Latency Mobile Cell Tower Mobile Last Mile 50ms to 800ms Up to 50% Packet Loss BW 56Kbps to 20Mbps Origin Server Mobile Device
  • 28. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.28 Mobile Latency – 1st Data point 28 http://stakeholders.ofcom.org.uk/binaries/research/telecoms-research/bbspeeds2010/Mobile_BB_performance.pdf Ofcom study, UK Mobile Networks, May 2012 Static Tests: • “Latency… was an average of 117 milliseconds. Nearly two thirds of tests took more than 100 milliseconds” User Panel: • “Average latency was 192 milliseconds and around a third of tests delivered latency of higher than 200 milliseconds”
  • 29. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.29 Mobile Latency – 2nd Data point 29 Cisco Cloud Index, 2010-2015 “Global average fixed latency is 125ms and average mobile latency is 290ms… Global mobile consumer average latency is 307.3ms”
  • 30. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.30 Latency – Summary of Averages 30 0 50 100 150 200 250 300 350 Fiber Cable DSL Mobile (Ofcom Tests) Mobile (Ofcom Panel) Mobile (Cisco) Mobile Consumer (Cisco) Latency by Connec on Type
  • 31. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.31 Packet Loss 31 Random Early Detection Average queue monitored size & packets dropped based on statistical probabilities. If buffer is almost empty, all incoming packets are accepted. As queue grows, probability of dropping incoming packet grows. When buffer is full, probability reaches 1 and all incoming packets are dropped. Packets get dropped on an end to end link due to the following: • Network congestion causes queues to saturate • Network congestion triggers RED • Interference on wireless networks • Hardware errors • Firewall Policies Packet loss is the inverse of delivery rate 100% - packet loss rate = delivery rate
  • 32. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.32 Packet Loss 32 http://pam2012.ftw.at/papers/PAM2012paper6.pdf Packet Loss, Akamai/MIT Research • Tested 149 Million connections in 51 countries • Compared PL for mobile networks from non-mobile networks • Most connections have no packet loss (Median PL is 0%). • 75th percentile usually zero too • Average connection packet loss: 2.5% • Fixed network average is 2.2% • Implies connections are usually fine, but sometimes terrible • The average PL for the non-zero half is 5% • Suggests “small” connections have higher Packet Loss • HTTP Connections are often small
  • 33. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.33 Jitter 33 Jitter No Jitter Jitter-Sensitive Applications: • Video • VoIP Jitter = Variability over time of latency across a network • PDV – Packet Delay Variation Constant Latency = Zero Jitter
  • 34. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.34 Mobile Bandwidth in Theory 34 Protocol Max Downlink Max Uplink GPRS 56-115 Kbps CDMA200 (1X) 153.6 Kbps Edge 236.8 Kbps 3G Bar 2 Mbps stationary client 384 Kbps moving client EV-DO 2.5-4.9 Mbps 0.15-1.8 Mbps EDGE Evolution 1.6 Mbps 0.5 Mbps HSDPA 2 Mbps 200 Kbps HSPA+ 14.4 Mbps 4G Bar 1 Gbps stationary client 100 Mbps moving client HSPA+ Release 11 21-672 Mbps 5.8-168 Mbps LTE 100-300 Mbps 50-75 Mbps WiMax 37-365 Mbps 17-376 Mbps
  • 35. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.35 Mobile Bandwidth 35 You only get those speeds it if you’re this guy…
  • 36. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.36 Mobile Bandwidth 36 Source (PCWorld): http://bit.ly/JX7MQw PCWorld/Novarum 3G • Download: 0.59 – 3.84 Mbps • Upload: 0.56 – 1.44 Mbps 4G • Download: 2.81 – 9.12 Mbps • Upload: 0.97 – 5.86 Mbps Date: Apr, 2013
  • 37. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.37 Download BW Impact on Load Time DSL Cable FIBER Poor Cell Conn Avg Cell Conn Load Time (ms) Download BW (Kbps)
  • 38. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.38 Upload BW Impact on Load Time DSL Cable FIBER Poor Cell Conn Avg Cell Conn Load Time (ms) Upload BW (Kbps)
  • 39. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.39 Latency Impact DSL Cable FIBER Poor Cell Conn Avg Cell Conn Load Time (ms) Latency (ms)
  • 40. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.40 Network Profiles – Conclusions for Performance Upload BW doesn’t matter much until <200 kbps Download BW Matters at Low Values <1Mbps • Unless Transfer Volume (Data) is Large Latency matters the most Packet Loss is Important but causes variation in results • Only use in Worst Network Conditions
  • 41. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. How to Select Network Test Scenarios
  • 42. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.42 1. Determine the Full Range of Client Connection Types, for example: • Remote Office Tokyo, London • Consumer Web UK, Germany • Consumer Mobile 2.5G India, UK, Germany • Consumer Mobile 3G UK, Germany, Japan • Consumer Mobile 4G UK, Japan 2. Select Test Range of Network Conditions (Not All): • Remote Office London • Remote Office Tokyo • Consumer Web Europe Typical • Consumer Mobile 4G UK Best • Consumer Mobile 3G Europe Typical • Consumer Mobile 2.5G India Worst Selecting the Network Values for a POC
  • 43. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.43 1. Work Out One-way Propagation Delay Server To Clients • Calculate KM Distance/200,000 2. Attempt To Validate Using Ping Tests • Or Use Other Tools – Try To Exclude The Last Mile 3. Demonstrate Mechanisms To Capture Network Conditions … • NC Server/Agents • NC App • NC Global Library • Other Tools/Data But Do Not Use The Results In The POC Selecting the Network Values for a POC
  • 44. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.44 5. Select Test Range of Network Conditions (Not All): • Remote Office London • Remote Office Tokyo • Consumer Web Europe Typical • Consumer Mobile 4G UK Best • Consumer Mobile 3G Europe Typical • Consumer Mobile 2.5G India Worst 6. … Which Are: • Representative • Predictable • In Sequence Selecting the Network Values for a POC (cont.) Use: • Manual, Static Definitions • Stepped Latency & Bandwidth (Equally Spread?) • Static (Zero?) Packet Loss (Except Final Case)
  • 45. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.45 Example Network Test Profiles • Baseline 0ms 100Mbps/100Mbps 0% PL • Remote Office London 20ms 10Mbps/10Mbps 0% PL • Consumer Web Europe Typical 50ms 5Mbps/2Mbps 0% PL • Consumer Mobile 4G UK Best 80ms 10Mbps/2Mbps 0% PL • Remote Office Tokyo 100ms 10Mbps/10Mbps 0% PL • Consumer Mobile 3G Europe Typical 130ms 2Mbps/512kbps 0% PL • Consumer Mobile 2.5G India Worst 250ms 512kbps/128kbps 1% PL Notes: • Always One Way Latency in NV • Baseline Should Have BW Defined to Ensure Stability Between Tests • Network Profiles Should be Common Between Test Types (Functional/Non-functional) Selecting the Network Values for a POC (cont.) (Too Many?)
  • 46. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. What to Cover
  • 47. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.47 Capabilities Network Discovery • (NC Server/Agents – Avoid)? • NC Global DB • NC Mobile App • Other Tools Single User Test - Desktop • Define Network Conditions • Proxy (SSL/HTTPS)? • Transaction Manager • (Integration with UFT Etc. – Avoid?) Single User Test - Mobile • Define Network Conditions • Proxy (SSL/HTTPS) • Web Console/Mobile Web Multi User Test • Define Network Conditions • Runtime Metrics • LR Analysis • NV Predictor • (Packet Capture – NV Analytics – Avoid?) • (API Transaction Marking – Avoid?)
  • 48. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. How to Test
  • 49. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.49 Check SSL/HTTPS • Use Proxy (Always Needed for Mobile) • Verify Decryption Works (Doesn’t Always) Assess Test Steps • Analyse User Journeys using NV Transaction Manager • Use NV Analytics to Focus on Network-Impacted Steps • Create Modified (Focussed) TM Steps for Network Testing Execute Tests • Baseline • Network Models Test Process – Single User
  • 50. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.50 Remember • Save .Shunra files AND TM (Timing) Exports • Unless using Mobile Console • Good Naming Conventions! Gotchas … • Check the Adapter • Connect TM! • TM Marking (Before 1st/After Last Packet) • TM Times vs NV Analytics Times Test Process – Single User (cont.)
  • 51. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.51 Check SSL/HTTPS • If Yes – Avoid Packet Capture During Test • Use Manual Test? Check Script Timings • Script Think Time • Paced vs Real-World? • Ramp Design Test Scenario • Add/Define Groups to Align with NV Virtual Locations Execution Sequence: 1. Full Test w/o NV 2. Low-Volume Test with NV 3. Full Test with NV Test Process – Multi User (LR/PC) Will Network Make Any Difference to Capacity? • 0.1s Baseline Execution Time – 25s Think Time • 0.2s Baseline Execution Time – 300s Pacing
  • 52. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.52 Remember • Save Scenarios • Save Results & Analysis Files • Document Network Profiles (Export?) Gotchas • “Local User” – Required For LR/PC Analysis • Blank Virtual Location – For NV Predictor • BW per Virtual Location vs Per Vuser • Packet Capture • Transaction Marking Test Process – Multi User (LR/PC – cont.)
  • 53. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. How to Report
  • 54. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.54 Example Docs
  • 55. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Follow Up
  • 56. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.56 Presentation
  • 57. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Q&A

Editor's Notes

  1. Packet Loss: The term "packet loss" is used to describe the probability of dropping a packet at any point across the network link. It is often measured in percent, reflecting the loss probability. From the network perspective loss can be categorized as random loss or loss due to congestion. Random loss is the result of equipment malfunction, radio signals corruptions, bad cables, etc. It is always bad to have it. On the other hand, since IP doesn't provide any explicit way to indicate that the network is congested, routers use loss for that purpose. Such loss is a way of communicating back to the users the need to scale back offered load on the network. The average packet loss rate for a network connection gives an overall sense of the quality of the connection. A connection with less than 1 percent average packet loss is considered a decent connection. But average loss doesn't tell the whole story. We also need to know what type, or pattern, of packet loss we will encounter. There are at least two kinds of packet loss that we must consider: 'Random' loss and 'Burst' loss. To explain the difference between them, let's suppose we are trying to run 2 Voice over IP conversation over 2 links that have an average of 1 percent packet loss. Call A loses one in every 100 packets over the entire call (random loss) while Call B loses 100 packets in two incidents - at the beginning and the end of the call (burst loss). Which call would you rather have? That's why it is important to consider not just the average packet loss but also the type of loss and information on any bursts of packet loss over time.
  2. Bandwidth constraints: Network capacity, or bandwidth, is the number of bits a network connection or interface can carry in a given period of time. It is usually measured in bps (bits-per-second), Kbps, Mbps or Gbps. The greater the bandwidth, the greater the number of concurrent application sessions the link can serve (for a given transaction) and the greater the rate that each application session can consume from the network. Bandwidth Utilization: Bandwidth utilization is a measure of how much of the WAN link's maximum data rate is being used. Consider an intuitive notion of utilization - it may start by picturing the WAN circuit as a pipe of a certain diameter and then imagining that it is partly filled with something we call traffic. Bandwidth utilization is a factor of the number of concurrent application sessions across the link and the average rate used by each session. For example, if a T1 link (1544 Kbps) serves an average of 20 concurrent application sessions and each session uses 50 Kbps on average each way then we would say that the link is 64.7% utilized ((50 Kbps * 20 sessions)/1544 kbps = 64.7%). Starting at 70% BU, network performance starts to degrade, 80% badly degraded, and 90+% “Flooded”. Smaller pipes, are subject to easier flooding, which will cause significant increased latency and jitter
  3. Bandwidth constraints: Network capacity, or bandwidth, is the number of bits a network connection or interface can carry in a given period of time. It is usually measured in bps (bits-per-second), Kbps, Mbps or Gbps. The greater the bandwidth, the greater the number of concurrent application sessions the link can serve (for a given transaction) and the greater the rate that each application session can consume from the network. Bandwidth Utilization: Bandwidth utilization is a measure of how much of the WAN link's maximum data rate is being used. Consider an intuitive notion of utilization - it may start by picturing the WAN circuit as a pipe of a certain diameter and then imagining that it is partly filled with something we call traffic. Bandwidth utilization is a factor of the number of concurrent application sessions across the link and the average rate used by each session. For example, if a T1 link (1544 Kbps) serves an average of 20 concurrent application sessions and each session uses 50 Kbps on average each way then we would say that the link is 64.7% utilized ((50 Kbps * 20 sessions)/1544 kbps = 64.7%). Starting at 70% BU, network performance starts to degrade, 80% badly degraded, and 90+% “Flooded”. Smaller pipes, are subject to easier flooding, which will cause significant increased latency and jitter