So Your Boss Wants You to Performance Test Blackboard

So Your Boss Wants you to Performance Test the Blackboard Learn™ Platform
Steve Feldman

Quick Bio
Blackboard since 2003
Performance Engineering from the start
Platform Architecture in 2005
Security Engineering in 2010
“Love my job…love my team. If you email me, I will respond.”
Stephen.feldman@Blackboard.com
http://goo.gl/Z4Rq5

@seven_seconds

Problem Statement
So Your BossWants You to PerformanceTest

Expectation Setting

Expectation Setting
Why are you going through this exercise?
What do you expect to get out of it?
Who will be working this effort?
When will it be accomplished?
How much will it cost?
Where do we go next once we accomplish it?

Expectation Setting
Drive
Functional
Objectives
Mine and Analyze Data
Access
Right
Tools
Develop
Simulation
Scripts
Benchmarking  Figuring Out Your Gaps
Capture
Appropriate
Metrics
Analyze and Respond
to the
Data
Escalate without Confidence

Expectation Setting

Everything You Need to Know

Planning: Goals and Objectives
Goals should be measureable and traceable
Best goals align to the vision and direction of the business.
Performance requirements are preferred
Poor goals involve system utilization metrics as they don’t align to the business.

Attributes of a good performance/scale goal
Response time percentile
Throughput metric: bytes, hits, pages/transactions served or processed
Community/Population
Definition of a business transaction
Workload/Data Condition
Database transaction
Failure Rate
HTTP Error Codes

Analysis Tools

Planning: Scheduling
Defining Objectives
Analyzing Behavior
Analyzing System Data
Functional Script Definition (Coverage Model)
Scripting
Data Set Construction (Test Bed Data Set)
Planning: Test Process
Monitoring Setup
Infrastructure Setup
Sample Testing
Restore Process
Calibration (Testing and Tuning)
Scalability Testing
Analyzing Results
Presenting Results

Everything You Need to Do
Performance Scenario and Modeling
Conduct Functional Interviews
Functional Analysis (Review Use Cases)
Log Mining
Data Mining
User Experience and Expectations
Sequence, Order and Probability
Modeling Time
Time of day, year and universal behavior

Multiple techniques for creating test bed and data conditions
Combination of ContentExchange and Snapshot
Use of B2 APIs
Direct SQL
Two pitfalls to avoid
Avoid creating synthetic data with load test scripts.
Avoid trying to use real customer data for actual test conditions.

Two Recommended Synthetic Transactions: Production HTTP Drivers and True Browser Rendering.
HTTP Drivers: Interval-based simulators usually from external sources.
Regulate Frequency, Define Functional Paths and Verify Non-Functional Requirements
Browser Rendering: Execute full browser behavior.
Show full E2E and not just First to Last Byte from a Server Pespective

Record and Playback vs. HTTP Capture
R/P acts like a proxy capturing HTTP and allows playback like a video recorder.
HTTP Capture: HTTPLiveHeaders, Fiddler and Firebug

Partial Payloads vs. Full Payloads
Use Partial for Code Simplicity and Management
Emphasis on Server Side Request
Accelerate scripting delivery time
Use Full for Total Round Trip Time
Dynamic, but controlled content for simplicity purpose
Doesn’t get browser time
Nugget: Introduce automated Selenium or WebPage Test script(s) on sampled intervals during life of test for browser and end-2-end time.

Arrival Rates and Load Levels

Analytics: Study Both as a Transparent Lens

SLA’s  Acceptance Criteria on top of Performance and Scalability Requirements.

When You Can’t Do It Yourself

Lead the objective planning phase  Gather requirements and conduct functional interviews.
Establish relationship w/ PerfEng team at Blackboard
Product detailed test plan
Execute the benchmark lifecycle
Produce summary report
Short-term recommendations for configuration
Feedback to Blackboard
Long-term capacity planning guidance

How We Do It

How We Do It
Endurance
Tests
Aggregate Workload
Tests
Adoption Tests: S-Curve,
Normal Distribution
and Altruistic L-Curve
Tests
Calibration
Tests
User Abandonment
Tests
Performance Archetype (PAR)
Tests
Regression
Tests

Steve Feldman
@seven_seconds

Please provide feedback for this session by emailingDevConFeedback@blackboard.com.
The title of this session is:
So Your Boss Wants you to Performance Test the Blackboard Learn™ Platform Before you Go Live with the Next Release

http://sevenseconds.wordpress.com	660
url_unknown	10
http://twitter.com	5
http://translate.googleusercontent.com	2
https://twitter.com	1
http://webcache.googleusercontent.com	1
http://sevenseconds.wordpress.com&_=1323421010115 HTTP	1

So Your Boss Wants You to Performance Test Blackboard

by sfeldman13 on Jul 12, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

7 Embeds 680

Statistics

So Your Boss Wants You to Performance Test Blackboard — Presentation Transcript