1. A part of the Nordic IT group EVRY
Distributed Video Management and
Security Systems
by Alexey Ostapov
Testing Tips and Tricks

2. Topics to Discuss
DIFFICULTIES IN TESTING OF COMPLEX SYSTEMS
DESIGNING TESTS FOR SECURITY SYSTEMS2
BUG HUNT …3
… AND REPRODUCTION SCENARIOS
AUTOMATED TESTING
TIPS AND TRICKS
1
5
6
4

3. What I am doing here?
Know how to start testing
Know how others avoid rakes
Know how getting things done
Compare with personal experience
Get some Fun
Why this presentation is interesting?

4. Who is this guy?
Alexey Ostapov
5+ years in software testing
5 years in security systems testing
1 year as QA trainer
Certified tester and so on
Test lead
Just nice guy
http://www.linkedin.com/in/ostapov

5. Simple System Test Flow
1.Run Application
2.Do Some Steps
3.Get some error
4.Reproduce (do step 2)
5.???
6.PROFIT!

6. Video Security System Test Flow
1.Run Application
2.Do Some Steps
3.Get some error
4.Reproduce (do step 2)
5.???
6.Does not reproduced?

7. Big Bro SW
Complexity of “Big Brother Software”
Plugins
SDK
Plugins
Operating
System
Drivers
Frameworks
DirectX
Updates
Environment
N
e
t
w
o
r
k
Cameras
Sensors
Recorders
Monitors
Services
FirmwareEquipment
API

8. Big Bro From Inside

9. What Input for Test Design?
Specification Specification Specification
Standards
Existing
systems
Common
Sense

10. What Test Design Techniques We are Using?
Equivalence partitions
Bitrate (256 Kbps, 1 Mbps, 5 Mbps, etc)
Device type (SD, HD, Full / MPEG, .H264 cameras, etc)
Decision Tables and State Transition
Device state (sees motion, in recording, not connected, etc)
Network condition (high, low, pings / 100 Mbps, 1 Gbps, etc)
Settings setup (Administrator, User, Another User / Schedules, Quality)
Pairwise
OS (Windows family)
Device type
Network condition
All known suits!

11. What Strategy Use?

12. What Strategy We are Using?
Risk based strategy?
SW can be considered as safety-critical
Specification based strategy?
Requirement are not complete
Requirements changing frequently
Mixed strategy – YES
Why?

13. Our Testing Process
Prerelease
System Test
Exploratory
testing
Updating Test
Cases
Smoke
Testing
System
Regression
Test
Duration
testing
Risk based
testing

14. Main Targets for Bug Hunt
OS and UI
• Everything user can do must be tested:
• System Configuration
• System Usage
• System Breaking
Network
• Normal flow
• Heavy traffic
• Disconnections
Devices
• Default Setting
• Reconfiguration
• Interoperability

15. Hunting BUG
1.Keep Calm!
2.Remember what you did
3.Gather as much as possible info
 Settings
 Devices/System state
 Logs
 Screenshot
 Etc
What to do if you have found new one 1st time?

16. Reproducing BUG
1. Try to reproduce in own environment
2. Try to reproduce in another environment (ask your colleagues)
3. Try to reproduce same conditions
1. Network
2. Devices
3. Events
4. Use additional software to reproduce
1. Debugger
2. Sniffer
3. Performance Monitor
5. Prepare new environment from scratch
6. Do NOT give up

17. CTQ? What a hell?
• Setup defined environment
• Run software in continuous video displaying mode
• Run Control Tools
• Wait defined time (72 hours minimum)
• Check software characteristics
• Perform smoke testing
How we do Critical-to-Quality duration testing

18. Why We are Doing CTQ?
Test stability over time
Find UI issues
Find memory leaks
Find Crashes

19. Moments of Big Bro
 GDI object crash
 Application crashes randomly without any reasons
 Cause – GDI UI objects were never reused
 .h264 decoder crash
 Reproducible constantly but rare over some time
 Cause – camera sends bad frame
 2.5 Gb RAM instant memory leak
 Cause - parasite network traffic causes Win API alloc of huge memory
 Kill process crash
 Does not left any info about what happened
 Cause – internal crash in thread manager
 Video recorder frequent reboot
 Cause – FW used to work synchronously. Request were out of stack
Challenges make testing fun!

20. Questions?

21. Contact us!
Address:
03056,
24, Polyova Str.,
Kyiv, Ukraine
Phone:
+38 044 457-88-56
Email:
info@infopulse.com.ua
Alexey Ostapov
Email:
alexey.ostapov@
infopulse.com.ua
Thank you
for
Attention!