Simple Railroad Command Protocol


Published on

Presentation of Testing results of SRCP softwares & detailed result written on Master test specification.

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Simple Railroad Command Protocol

  1. 1. MASTER TEST SPECIFICATION FOR SIMPLE RAILROAD COMMAND PROTOCOL(SRCP) 28 JUNE 2010 Tester, Ankit Singh Masters High Integrity System University of Applied Sciences Frankfurt am Main
  2. 2. OVERVIEW <ul><li>SRCP Version 0.8.3 </li></ul><ul><li>SRCP is an internet protocol for controlling and programming (digital) model railway system. </li></ul><ul><li>Communication: TCP Network connection between Client & Server </li></ul><ul><li>SRCP Provides Three Modes of operations: </li></ul><ul><ul><li>Handshake Mode </li></ul></ul><ul><ul><li>Command Mode </li></ul></ul><ul><ul><li>Info Mode </li></ul></ul>
  4. 4. STRATEGIES: TO SELECT THE FINAL TEST CASES <ul><li>Important Network Establishment between a client and a server </li></ul><ul><li>1. Level of priority </li></ul><ul><ul><li>a. MUST/MUST NOT tagged requirements </li></ul></ul><ul><ul><li>b. SHALL/RECOMMEND tagged requirements </li></ul></ul><ul><ul><li>c. CAN/OPERATIONAL tagged requirements </li></ul></ul><ul><li>2. Valid/Invalid requirements </li></ul><ul><li>3. General flow of operations requirements </li></ul>
  5. 5. TEST METRICS USED Test metric Definition Purpose How to calculate Defect severity The severity level of a defect indicates the potential business impact for the end user (business impact = effect on the end user x frequency of occurrence). Provides indications about the quality of the product under test. High-severity defects means low product quality, and vice versa. At the end of this phase, this information is useful to make the release decision based on the number of defects and their severity levels. Every defect has severity levels attached to it. Broadly, these are Critical, Serious, Medium and Low. Test coverage Defined as the extent to which testing covers the products complete functionality. This metric is an indication of the completeness of the testing. It does not indicate anything about the effectiveness of the testing. This can be used as a criterion to stop testing. Coverage could be with respect to requirements, functional topic list, business flows, use cases, etc. It can be calculated based on the number of items that were covered vs. the total number of items.
  6. 6. BLACK BOX TESTING <ul><li>Equivalence partitioning </li></ul><ul><li>Boundary-value analysis </li></ul><ul><li>Error guessing </li></ul>
  7. 7. WRONG TEST SPECIFICATION (BLACK BOX TESTING) Test Case ID.No Test Area Command Expected Output Behavior Description Severity Level 3_1 Command set ConnEctionMode Srcp Command 410 ERROR unknown command Fail : <TimeStamp>202 OK CONNECTIONMODE Low 3_2 Feedback SET 1 FB 0x111111 0 100 INFO 1 FB 0x111111 0 Fail: <TimeStamp> 422 ERROR unsupported device group Critical 3_3 Session GET 0 SESSION 2 100 INFO 0 SESSION 2 Fail: <TimeStamp> 422 ERROR unsupported device group Critical 3_4 Server RESET 0 SERVER 101 INFO 0 SERVER Fail : <TimeStamp> 423 ERROR unsupported operation Crititcal 3_5 Time TERM 0 TIME 102 INFO 0 TIME Fail: <TimeStamp> 422 ERROR unsupported device group Critical
  8. 8. WHITE BOX TESTING <ul><li>Statement coverage </li></ul><ul><li>Decision coverage </li></ul><ul><li>Condition coverage </li></ul>
  9. 9. WHITE BOX TESTING REPORT SAFE CODING GUIDELINES Function Bug Suggestion enqueueInfoFB 1000, Message length hardcoded Should be passed any specified message, and calculate size dynamically queue_reset_fb usleep not checked for incorrect result check the result returned by usleep() infoFB msg overflow can occur, so message length should be dynamically checked Check for the overflow
  10. 10. FINDINGS <ul><li>TCP-port 4303 assigned by IANA but not mentioned in specification </li></ul><ul><li>Incomplete data in specification leads to confusion </li></ul><ul><li>Insufficient details in Specification </li></ul><ul><li>In some cases commands are not working on certain device groups </li></ul><ul><li>Differences between Specification & srcpd implementations. </li></ul><ul><li>Proper Return checks are not done in the source codes which can lead to crashing of the system. </li></ul>
  11. 11. IMPROVEMENTS <ul><li>Perform stress tests (eg. high number of simultaneous connections) </li></ul><ul><li>Increase code coverage </li></ul><ul><li>Increase control over environment (OS settings, real time debugging info) </li></ul><ul><li>Improve overall clarity about specification </li></ul><ul><li>Server Crash testing (Server crash and recovery time) </li></ul>
  12. 12. CONCLUSION <ul><li>Full code coverage using black-box testing isn’t feasible </li></ul><ul><li>White-box in combination with black-box testing considerably increases code coverage </li></ul><ul><li>Increased code coverage = Increased code safety </li></ul>
  13. 13. REFERENCES <ul><li>Lecture notes and slides of Dr. Shoenfelder. </li></ul><ul><li>The Art of Software Testing, Second Edition, Glenford J. Myers </li></ul><ul><li> </li></ul>
  14. 14. <ul><li>Thank You </li></ul><ul><li>For </li></ul><ul><li>Listening patiently </li></ul><ul><li>& </li></ul><ul><li>Asking No Questions !!! </li></ul><ul><li> </li></ul>