Mts srcp

585 views

Published on

Mater Test Specification for SImple Rail road command protocol

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
585
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Mts srcp

  1. 1. Master Test Specification Simple Rail Road Command Protocol (SRCP)Submitted by:Rishu SethHigh Integrity SystemsFachhochschule Frankfurt am MainMatriculation number-936161 1
  2. 2. Table of Contents1.Introduction ...................................................................................................... 4 1.1 Overview ............................................................................................... 4 1.2 Test Object ............................................................................................ 4 1.3 Test Environment .................................................................................. 5 1.4 Test Plan................................................................................................ 6 1.5 Test Organisation .................................................................................. 7 1.6 Test Resources ...................................................................................... 72. Test Planning ................................................................................................... 8 2.1 Test Areas ............................................................................................. 8 2.2 Test Metrics .......................................................................................... 9 2.2.1 Defect Severity ...................................................................... 10 2.2.2 Defect Priority ....................................................................... 10 2.3 Test Requirement ................................................................................ 12 2.3.1 Network Connection ............................................................. 12 2.3.2 Lexical Units ......................................................................... 12 2.3.3 Commands ............................................................................. 13 2.3.4 Replies ................................................................................... 13 2.3.5 State of Server ....................................................................... 13 2.3.6 Hand Shake Mode ................................................................. 14 2.3.7 Command Mode .................................................................... 153. Test Specification .......................................................................................... 16 3.1 Equivalence Classes ........................................................................... 16 3.1.1 Feedback Sensors .................................................................. 16 3.1.2 Generic Accessories .............................................................. 17 3.1.3 Generic Loco ......................................................................... 17 2
  3. 3. 3.2 Black Box Test Cases ......................................................................... 17 3.2.1 Hand Shake Area ................................................................... 18 3.2.2 Command Area ..................................................................... 19 3.2.3 Info Area................................................................................ 20 3.3 White Box Testing .............................................................................. 20 3.3.1 SRCPD Source ...................................................................... 20 3.3.2 Masking of Condition............................................................ 26 3.3.3 Cause Effect Graph ............................................................... 264. Findings .......................................................................................................... 305. Conclusion...................................................................................................... 30References .......................................................................................................... 30 3
  4. 4. 1. Introduction 1.1 OverviewThis document is Master Test Specification for Simple Railroad CommandProtocol (SRCP: Version 0.8.3). The initial focus of this document is to analyzeSRCP, its specification and provide test cases & scenarios, that willeventually lead to errors in the system and then after that also to look inside thecode and find anomalies and errors. Although to provide the full test coverageis not exactly possible. The main objectives of this document are:1. It provides a status report of the test object in comparison to the requirements. Is the product fit for release? Adding value to the product by finding defects.2. To Test the most prioritized elements of the test-object. The realistic use of the software should be considered.3. Testing should be unbiased and from different point of views considering the needs of different users.4. Test process is well structured. Is it repeatable and predictable?1.2 Test ObjectSRCP is an internet protocol for controlling and programming (digital) modelrailway systems. It produces client server communication. The server is thecomponent that is connected with the plant.The client can be run on the same computer as the server. The server has noclue about how the plant looks like. It is the server’s task to translate thecommands of SRCP into action.A SRCP server is unaware of the exact concrete railway system. And thetopology and even it is not being informed about the installed equipment. Its 4
  5. 5. task is to collect all available information as precisely as possible handlingknowledge about the possibilities and limits of the controlled plant. Its datahave always to be understood as the best knowledge about the state of therailway system.Limitation of SRCPSRCP does not fulfill any formal real time demands.Merits of SRCP 1. A SRCP server abstracts the control of the construction. 2. Paying attention to different safety strategies. 3. Scaling Capability.An SRCP server represents one central unit (CU). A CU provides at least onebus over which information is read or commands are executed. A client contactsthe server in order to send commands to the plant and to receive informationfrom there. So there are SRCP channels between client and server, where theclient sends commands to the server and SRCP channel between server and theplant, where the server has to execute the command on the plant. SRCPconnection is based on TCP (Transmission Control Protocol).The Test object will be the protocol version 0.8.3 of SRCP and the SRCPDaemon 2.0.12, which will emulate the SCRP environment.1.3 Test EnvironmentThe SRCP Test Environment consists of: 1. SRCPD - will emulate the SRCP server and a digital plant. 2. Server - A machine with OS Linux. SRCP Daemon will run on the server. 5
  6. 6. 3. Client - The Client represents a working station with a python written script, which will connect to the server. OS can be either Linux or Windows. 4. Java - will provide a client program that sends commands to the SRCP server. 5. Documentation – MS (Microsoft) Word, Excel, Latex.1.4 Test PlanThe overall test strategy consists of 6 main phases or levels .The levels aregiven belowTest-Planning • To Analyze the SCRP specification. • It provides test metrics to measure the quality of the test object. • It formulates the requirements of the test object.Test-Case-Generation • Use the test metrics & requirements to generate test cases for black box testing. Also divide the test cases in different test areas. • Provide a python written client program which will help us to execute the test cases on the test object. • Inspect the source code of SRCP Daemon and specify white box test cases.Test-Case-Execution • Execute test cases & log results.Analysis Test Result 6
  7. 7. • To analyze test result and compare it with desired result.Bugs Analysis • To find the bugs that means deviation from our previous stag.Test-Documentation • Visualize test results. • Defect Reporting.Final-Test-Result • Finalized the test result after Bugs Analysis.1.5 Test Organization • Developers: Dr. Schönfelder. • Testers: Students. • Execution of Tests: Workstations at the university.1.6 Test Resources • SCRPD 2.0.12 – source code. • Specification of SRCP 0.8.3 • Configurations file "srcpd.conf". • A client program written in Java will be used to connect to the server and execute test cases. 7
  8. 8. 2. Test Planning2.1 Test AreasThere are mainly three main test areas: • Hand Shake Area: On initial connection hand Shake Mode is activated. Then after this connection, its according to the demand of the client whether to activate Command Mode or Info Mode. • Command Area • Info AreaThere are some different Sub Test Areas also which are used in combinationwith these three Test Areas: 1. Server: A SRCP server represents exactly one central unit (CU) in the usual sense. Every CU disposes of at least one bus for addressing on which information is read and/or commands are executed. If a SRCP server can manage and provide several CU it is valid. 8
  9. 9. 2. Feed Back (FB): Feedback devices are encoders that signal an occurrence on construction 3. Generic Accessoire: A generic Accessoire generally indicates a decoder that can serve one or more ports under one address. Often these are switch decoders or signal decoders working as impulse decoders. • Generic Loco (GL): It indicates engine decoders in the engine address room of the hardware. • Locks: Devices locking other devices. The LOCK devices are devices providing a lock over another device. They are optional.2.2 Test MetricesIt is the measure of software quality and confidence of our test object. Testmetrices have various uses: 1. It can reduce working time - once we have a matrix for a specific issue, it will take our less time to test it or to think how to test it. 2. It is logical and testing challenging - It is a challenge to make our own matrix and to find common issues that are repeatable from project to project.For Testing of this project, used metrics will look at: • Functional areas • Requirements CheckFor Test coverage and consistency of Test EffortDuring Test Preparation this metric uses: • Number of Test requirements compared to Functional Area/ Requirements 9
  10. 10. • Number of Test Cases planned compared to Test Cases ready for executionDuring Test Execution this metric uses : • Number of Test Cases Passed, Failed or Blocked. • Number of Test Cases Executed compared to Test Cases Planned Severity: Grouping by potential danger caused.2.2.1 Defect SeverityIt is the measure of how lethal a defect can be and how strong is the effect offailure on software user. • Critical: It is the highest order of severity and results in the failure of complete software system of a subsystem or of a software unit within the system. • Major: It is almost relevant to “Critical” but there are still some chances of processing some acceptable alternatives to get desired result. • Minor: This is referred to as some common normal defects which are not as lethal as other two but somehow effect the normal working of a software.DEFECT SEVERITY: TEST CLASS DEFINITION STANDARD % PAS CRITERIA RATE CRITICAL ESSENTIAL 100 % MAJOR NECESSARY 90% MINOR NOT IMMEDIATE 70%2.2.2 Defect Priority 10
  11. 11. It is the measure of priority that is set in regard to addressing defects andaccording to its severity, to set minimum time within which it should be fixed. • High: This is the most prioritized area and defect falling in this category severely affect the system’s working and should be fixed as soon as possible. • Medium: This is having a bit less priority and the defects can be fixed in normal course of development activities. • Low: This is having the lowest priority and the defects can be fixed after the more severe n priortised defects are take care of.For example when a client connects to the SRCP server, the server generates awelcome message containing information about the server. The defect of notgenerating the welcome would have minor severity, but the process is repeatedevery time we connect to the server and that is the first thing we see. So thepriority for this defect would be high and this would be test cases we shoulddefinitely consider.Another example would be that all commands are case sensitive and have to bewritten in uppercase letters. In case some commands would be accepted withlower case letters that would result in a defect, which doesnt cause a failure anddoesnt really impair usability. So the defect would have a minor severity andminor priority. So a test case for this defect would not be relevant, consideringour matrices.DEFECT PRIORITY: SEVERITY LEVEL DEFINITION High High system crash data loss Medium Wrong result operation error Low Low minor problems 11
  12. 12. The table tells us that we will only consider relevant combinations for our testcases. We cannot provide full test coverage of the Test-Object, because thenumber of all possible test cases could be from high to astronomically high.Implementing all possible test cases would consume too much time, so that bythe time we are finished will all possible test cases, new versions of our Test-Object would be available, which also need to be tested. So for the test cases weonly focus only on the important ones with the help of our matricesA test-area from the main test areas (Chapter 2.1) has passed the test process ifit does not contain any defects of the severity level "Critical“. It is essential tothe product. Here need 100% pass rate.1. A test-area has passed the test process if it does not contain less than 90% pass rate. It is called important.2. A test-area has passed the test process if must not be less than 70% pass rate and it is called desirable test class.2.3 Test Requirements Test-Requirements tell us what requirements the Test-Object has to full fill, by analyzing the specification of the Test-Object. They will help us to derive test cases.2.3.1 Network Connection The SRPC is based on the data transfer technologies of TCP (Transmission control Protocol). One port is occupied. At present the port 12345 SHALL be used. Registering with IANA can change this for the future and give the port a name, too. The temporary name is srcp 12345/tcp2.3.2 Lexical Units 1. In this case, all character codes are described in their decimal form as # followed by the numeric value of the character. 12
  13. 13. • The entire communication consists of the characters of the 7bit ASCII character set in the range from #32 to #127 (including the borders). • Again the characters #9 (TAB) is used for space, #10 and #13 (CR, LF) for end of line are valid. • All further characters (esp. with a code value >127) are ignored in incoming data and removed. • In outgoing data they can be included, but is not used within SRCP. 2. The characters #9and #13 are seen as white space.2.3.3 CommandsCommands are send in the “Command Mode” and in “Hand Shake” from clientto server. The SRCP server processes the command and generates a responsethat is to be sent to the client. Command consists of: 1 Commands consist of a command word, followed by a command parameter list separated by white space. 2 The end of a command is the end of line. 3 The end of line always consists of the character #10(n, LF). 4 A prefixed #13(r, CR) is valid. The character #13 is ignored.2.3.4 RepliesA server HAS TO reply to every command of a client. That is to be sent to theclient on the same connection.A reply SHALL be single-line. With the sign (Backslash, #92) immediatelybefore the LF(resp CRLF) it is marked that the reply contains another line.A single line including the end of the line MUST NOT exceed the length of1000 characters.If the result of a command can be ascertained due to communication withconnected model railway devices this opportunity SHALL be used. In case thereare insufficient results for a task the answer is “416 ERROR no data”. 13
  14. 14. 2.3.5 State Of ServerSRCP server is always in one of three possible states towards the client: handshake, command mode or info mode.Hand Shake ModeIn this Mode, after establishing a connection and sending the welcome messagethe server waits for a command from the client. The server executes it and sendsa single line reply to the client. Then the server waits for the next command.That way information and commands are being exchanged. Some examplecommands used in this mode are:1 SET CONNECTIONMODE SRCP <MODE> The type of connection isagreed upon. Valid are INFO for the unidirectional information mode andCOMMAND for the bidirectional command mode. The server sends either 202OK or the error message 401 ERROR unsupported connection mode to theclient.2 SET PROTOCOL SRCP <VERSION> The desired protocol that is to be usedis arranged. If this command is missing the SRCP version given during 14
  15. 15. welcome is accepted. Every released version number from and including 0.8 isvalid as <VERSION>.2.3.6 Command ModeDuring command mode the server waits for commands from the client andexecutes these. As the result of the execution the server always creates a replydue to be sent to the client in the same TCP session. Subsequently the nextcommand is executed.The server‘s replies to the client in the command mode always consist of a timestamp at the beginning, a numerical answer code and further data. Thenumerical answer code is structured in groups. 100-199 marks information andresults, 200-299 includes receipts confirming the processing according to therules, 400-499 marks error while executing commands, 500-599 errors by theserver itself, 600-699 specific error codes in implementation.1xx INFO: Informations, results2xx OK: Command is accepted and brought to execution. Attention: This is noconfirmation for actually executing the command.4xx/5xx ERROR: An error condition has occurred. The command is ignoredand not executed.6xx ERROR: A server specific error has occurred. The command in question isnot executed.The significance of the time stamp arises from the kind of reply: For quittances(OK, ERROR) it specifies the point in time when the command is processed andthe receipt OK or ERROR is generated.Some conditions in command mode are : 1. Replies of the commands by the server are to be sent to the client in the same TCP session. 2. GET, SET, CHECK, WAIT, INIT, TERM, RESET, VERIFY are commands that are defined in command mode. 15
  16. 16. 3. Valid commands must always be executed. 4. It is not possible to go to info mode out of command mode.2.4 Result of Planning PhasePROJECT PLAN: APRIL MAY JUNETestplanningTest case generationTest caseexecutionAnalysis test resultdocumentationBugs analysisdocumentation(Repeat until desirelevel)Final result of testcasesdocumentation3. Test Specification3.1 Equivalence ClassEquivalence class method will help us to develop test cases in a structured wayand Assists finding test data to provoke a special expected behaviour.3.1.1 Feedback Sensors 1. Class of addresses allowed: 1 ≤ x ≤ 10. Representative element = 1. 2. Class of addresses not allowed: x < 1 && x > 10. Representative element = 11 16
  17. 17. 3.1.2 Generic Accessoires 1. Class of all possible addresses allowed: 0 ≤ x ≤ 324. Representative element = 1. 2. Class of addresses not allowed: x < 0 and x > 324. Representative element = 3253.1.3 Generic Loco 1. Class of all possible addresses allowed: 1 ≤ x ≤ 81. Representative element = 1. 2. Class of addresses not allowed: x < 1 and x > 81. Representative element = 823.2 Black Box Test CasesA python-written client program will run input/output driven tests on the SRCPserver. The client program will communicate with the server through a TCPbased connection. Input data will be provided through the specification ofSRCP.Because of the configuration of SRCPD we will be able to use bus 1 for most ofthe test cases and bus 0 for "SERVER", "SESSION", "LOCK","DESCRIPTION" or "TIME". 17
  18. 18. FIGURE: Server Client Communication 18
  19. 19. ID Command Description Output Severity/Priority1 Set Connection Enter command 202 OK Critical / High Mode SRCP Command mode in session Connection 1 Mode OK2 SET Set protocol 201 OK Critical / High Protocol PROTOCOL version for SRCP SRCP 0.8.3 server3 Set Connection Try enter 401 Error Critical / High Unsupported Mode SRCP invalid mode Connection Mode4 Set Connection Enter info mode 202 OK Critical / Major Mode SRCP INFO Connection Mode OK5 GO Complete Hand 200 OK GO Critical / High ShakeID Command Description Output Severity/Priority1 RESET 0 SERVER Reinitiate 200 OK Critical /High server2 INIT 1 POWER Initiate power 200 OK Critical /High supply on bus 13 SET 1 POWER Set power 200 OK Critical / High ON supply on4 GET 1 FB 1 Get info from INFO 1 FB Major /Medium updated FB 2 123 on bus 1 with address 15 SET 0 POWER Get server out 415 ERROR Major/ High 19
  20. 20. OFF of power forbidden supply (not allowed)ID Command Description Output Severity/Priority1 TERM 0 Close 100 INFO 0 Critical / High SERVER connection to SERVER server2 INIT 1 Initiate power 101 INFO 1 Critical/ Medium POWER supply on bus 1 POWER3 INIT 1 FB S88 Initialize a FB 101 INFO 1 Major /Medium 888 1 on bus 1 with FB address 14 INIT 1 GA 0 S Initialize a GA 101 INFO 1 Critical / Medium 11 on bus 1 with GA 0 1 1 address 0 over protocol S with port & value 15 TERM 1 GL 1 Set engine to 102 INFO 1 Major / Medium default state & GL 1 take it out of communication 50 45 40 35 30 25 Series1 20 15 10 5 0 Total Test Cases Pass FailFigure Test Case pass and fail ratio Total 45 Pass 18 Fail 27 20
  21. 21. 3.3 White Box Testing3.3.1 SRCPD SourceWhite Box Testing is a method of testing software that tests internal structuresor workings of an application, as opposed to its functionality. In white-boxtesting an internal perspective of the system, as well as programming skills, arerequired and used to design test cases. The tester chooses inputs to exercisepaths through the code and determine the appropriate outputs.Main Criteria in White Box Testing is code coverage, which means all thebranches in the control flow graph are going to be covered. The White boxTesting is going to be performed on the Specific modules of the SRCP server toensure safety of the tested models.First we need to look at the most important source files for out test process. Foreach module of SRCPD there are 2 different files. The *.h files contain thedefined functions and constants. The *.c files contain the exact definition of thefunctions which were defined in the corresponding *.h files.I have performed white box testing on the following module of the SRCP:Followings are the bugs findings performed by doing statement and logiccoverage white – box testing.MODULE: srcp-fb.c1. Function: initfbBug: Here bus number is not checked and ‘fb’ is invalid under under busnumber zero. 21
  22. 22. Suggestion: Bus number should be checked before initializing the ‘fb’.2. Function: queue_reset_fbBug: While on sleep it does not show any error message.Suggestion: It should return an error message or result in case an error occurs.3. Function: enqueueInfoFBBug: Here declared message length is constant i.e. 1000 and if anotherparameter is passed , it may crash.Suggestion: It should be made Dynamic.Problem FB: Take FB on bus 1 with address 1 out of communication.It is not running on bus 1.Suggestion: It should be running on bus 1.Problem: SET 2 FB 1 SET FB with too short parameter list. Address is notmentioned.Suggestion: Parameter list is checked and also address of bus should becheckedFlow graph:It is a representation of all paths that have been traversed by a program duringits execution, using graph notation. Using this flow chart, we can compute thenumber of independent paths through the code. In a control flow graph eachnode represents a basic block. Direct edges are used to represent jumps incontrol flow. There are two specially designated blocks:Entry block: Through which control enters into flow graph.Exit block: Through which all control flow leaves the graph.1. Function: initfbIts code part and corresponding flow graph is given below:CODE: 22
  23. 23. FLOW GRAPH:2. Function: queue_reset_fbIts code part and corresponding flow graph is given below: 23
  24. 24. CODE:FLOW GRAPH: 24
  25. 25. 3. Function: enqueueInfoFBIts code part and corresponding flow graph is given below: 25
  26. 26. CODE:FLOW GRAPH:3.3.2 Masking of Conditions:It is necessary for efficiently analyzing the test paths. In this case multi-condition decision is made separately to analyse the test paths efficiently.In my white box testing I have made a masking of condition graph for ‘init_fb’function in the module ‘srcp-fb.c’.Masking of condition: 26
  27. 27. 3.3.3 Cause Effect GraphThis is basically a hardware testing technique adapted to software testing. Itconsiders only the desired external behavior of a system. This is a testingtechnique that aids in selecting test cases that logically relate Causes (inputs)to Effects (outputs) to produce test cases.A “Cause” represents a distinct input condition that brings about aninternal change in the system. An “Effect” represents an outputSteps to design the cause effect GraphStep – 1: For a module, identify the input conditions (causes) and actions(effect). 27
  28. 28. Step – 2: Develop a cause-effect graph.Step – 3: Transform cause-effect graph into a decision table.Step – 4: Convert decision table rules to test cases. Each column of thedecision table represents a test case.ESTABLISHING A CONNECTION: CAUSE EFFECT(1) Client connects to the server by (E1) Welcome string sent by the serverTCP/IP connection(2) Client desired operation mode (E2) Server reply according to operation mode demand by client(3) Server – Client sent GO command (E3) Server confirmed GO commandCEG 1:CEG 2: 28
  29. 29. CEG 3: 29
  30. 30. DECISION TABLE:TEST CASE 1 2 3 4(1) Client connects 1 0 1 0to the server byTCP/IP connection(2) Client desired 1 1 1 0operation mode(3) Client sends GO 0 1 1 0command4. Findings • Delayed Response from server • Not proficient details in Specification • Time Out – Server gets stuck after certain period of time5. ConclusionIn our master test case specification, the Black box test cases are selectedaccording to the SRCP protocol and our assumptions stated earlier. White Boxtest case findings were done while analyzing code and it is very important to doboth Black Box and White Box testing together so as to find more bugs and bemore sure about working of the software.6. References[1] Class notes and slides of Dr. Schoenfelder[2] The Art of Software Testing , Second Edition, Glenford J. Myers 30

×