Image Based Testing-IndicThreads-Q11

776 views
679 views

Published on

Session Presented at 2nd IndicThreads.com Conference On Software Quality held on 25-26 March 2011 in Pune, India
WEB: http://Q11.IndicThreads.com

------------
Session Abstract

Image-based testing (IBT) is a new form of automated software testing that relies solely on evaluating and\or duplicating the user-experience through the on-screen display and user-input devices.

Image-based testing requires the test automation engine be able to acquire artifacts from the on-screen display or otherwise process snapshots of the on-screen display to perform the tasks assigned required for an automated test. It utilizes image recognition to perform mouse clicks and keyboard strokes just as a user would.

IBT is technology agnostic because it is only looking at what is on the screen. As a result, the application under test can be written in almost any language, such as Java, .Net, or Flash. Image-Based testing allows us to test virtually anything that can be displayed on the screen. It doesn’t matter what the underlying technology is, if it is visible on the screen then we can interact with it.

Tool manufacturers can hardly keep pace with the needs of cutting edge application testers. If you are tasked with testing newer or novel technologies then it is usually difficult to find a tool that can work in that environment early in the development and testing lifecycle. When the tools you have can’t support the technologies you need to test you need to turn to something else. This is a good time to consider Image-Based testing (IBT).

This session will look at:

What exactly IS Image-Based Testing?
Thinking and Testing Differently with IBT.
Developing IBT Tests and App Maps.
SAFS Image-Based Recognition Syntax.
Dealing with Variations in the IBT Environment.
Dealing with NLS Testing in an IBT Environment.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
776
On SlideShare
0
From Embeds
0
Number of Embeds
26
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Explain how IBT is all about graphics on screen and its all about images. There might be many other challenging UI technologies which are prevalent in market but do not have good tool support for testing. IBTs score a point here.
  • As automation inputs and outputs are produced and consumed on the OS level, the application technology becomes irrelevant and it can automate any GUI application which is displayed on screen.
  • Along with this, the ease associated with doing trivial operations like reading a text, verifying enabled/disabled state, selecting from a dropdown or grid etc. helps OBT score a good point when compared to IBT.  IBTs offer an alternative here since they are UI technology neutral.
  • When seeking a "window" mapping the entire screen is searched for this image. When seeking a "component" mapping the search area is limited to the area of interest found for the "window" mapping. The bounds of the area of interest can be expanded by using the optional ImageR and ImageB items described below.
  • It is important to note that images must be saved in a format that provides no-loss of pixel information.  Stored images must be able to match with 100% picture quality the image snapshots that will be retrieved from the screen.  While "BitTolerance" discussed above allows for some degree of comparison fuzziness, it will usually not be able to compensate for stored images that cannot reproduce 100% picture quality due to excessive compression or intentional loss of pixel information.
  • Images stored for a particular Display typically work for most or all screen resolutions on that Display. This is an issue that each Display is configured for different levels of data compression. Bitmaps stored for the Normal Display have no data compression and no loss of image information. The displayed image for the Remote displays is usually compressed--intentionally removing image information. Because of this, Normal Display images usually will not match Remote Display images. To compensate for this, it is highly recommended that recognition images always be captured in the display mode that will be used for runtime testing.  For example, if you know all testing will be done via Remote Desktop sessions, then it is best to have all recognition images captured and prepared during Remote Desktop sessions.
  • It is important to note that images must be saved in a format that provides no-loss of pixel information.  Stored images must be able to match with 100% picture quality the image snapshots that will be retrieved from the screen.  While "BitTolerance" discussed above allows for some degree of comparison fuzziness, it will usually not be able to compensate for stored images that cannot reproduce 100% picture quality due to excessive compression or intentional loss of pixel information.
  • It is important to note that images must be saved in a format that provides no-loss of pixel information.  Stored images must be able to match with 100% picture quality the image snapshots that will be retrieved from the screen.  While "BitTolerance" discussed above allows for some degree of comparison fuzziness, it will usually not be able to compensate for stored images that cannot reproduce 100% picture quality due to excessive compression or intentional loss of pixel information.
  • It is important to note that images must be saved in a format that provides no-loss of pixel information.  Stored images must be able to match with 100% picture quality the image snapshots that will be retrieved from the screen.  While "BitTolerance" discussed above allows for some degree of comparison fuzziness, it will usually not be able to compensate for stored images that cannot reproduce 100% picture quality due to excessive compression or intentional loss of pixel information.
  • imagepath  can be the full path to a single image or to a directory containing multiple images. Multiple images are necessary if the target image is different in different environments. For example, on different platforms, or different versions of the application or operating system. The framework will search the screen for each of the images in the directory until it finds the match.
  • Image Based Testing-IndicThreads-Q11

    1. 1. Image Based Testing- application technology independent automation Girish Kolapkar SAS R&D (India)
    2. 2. Agenda: <ul><ul><li>What exactly IS Image-Based Testing (IBT)? </li></ul></ul><ul><ul><li>OBT vs IBT </li></ul></ul><ul><ul><li>Thinking and Testing Differently with IBT </li></ul></ul><ul><ul><li>Developing IBT Tests and App Maps </li></ul></ul><ul><ul><li>SAFS Image-Based Recognition Syntax </li></ul></ul><ul><ul><li>How to automate using IBT </li></ul></ul><ul><ul><li>Image Manager Tool </li></ul></ul><ul><ul><li>Sample Application Map And Script </li></ul></ul>
    3. 3. Agenda: <ul><ul><li>Dealing with Variations in the IBT Environment </li></ul></ul><ul><ul><li>Dealing with NLS Testing in an IBT Environment </li></ul></ul><ul><ul><li>Challenges </li></ul></ul><ul><ul><li>Demo </li></ul></ul><ul><ul><li>Q & A </li></ul></ul>
    4. 4. What exactly IS Image-Based Testing? <ul><ul><li>Strictly “What You See is What You Get” (WYSIWYG) </li></ul></ul><ul><ul><li>Interaction with the visual display of the screen </li></ul></ul><ul><ul><li>Testing based on finding and interacting with graphics on the screen </li></ul></ul><ul><ul><li>Automation is driven through keyboard and pointer events </li></ul></ul><ul><ul><li>Test technologies partially or not supported by existing tools </li></ul></ul><ul><ul><ul><li>E.g Legacy Applications (Xterm application),Flash Applications, GUI on Unix </li></ul></ul></ul>
    5. 5. What exactly IS Image-Based Testing? <ul><ul><li>An alternative when items are not recognized at all by existing tools </li></ul></ul><ul><ul><li>Independent of any underlying application technology </li></ul></ul><ul><ul><li>The antonym or opposite of Object-Based Testing. </li></ul></ul>
    6. 6. What exactly IS Image-Based Testing? Image Based Testing Tool Operating System Application Under Test Display Buffer Mouse pointer events/ keyboard events queue
    7. 7. OBT vs IBT <ul><li>Object-Based Testing </li></ul><ul><ul><li>Ideal for automating PC hosts </li></ul></ul><ul><ul><li>Performance: slower than IBT </li></ul></ul><ul><ul><li>Can automate supported technologies only </li></ul></ul><ul><ul><li>Can produce reliable & robust scripts(time tested) </li></ul></ul><ul><ul><li>Wide range of functions available to use </li></ul></ul><ul><li>Image-Based Testing </li></ul><ul><ul><li>Ideal for Applications which OBT can’t support or limited support e.g. GUI on non PC hosts, legacy applications </li></ul></ul><ul><ul><li>Performance :faster than OBT </li></ul></ul><ul><ul><li>Can automate virtually anything that can be displayed on the screen </li></ul></ul><ul><ul><li>Needs to be provedimproved after extensive usage </li></ul></ul><ul><ul><li>Limited verifications </li></ul></ul>
    8. 8. Thinking and Testing Differently for IBT <ul><ul><li>Object-Based Testing deals with real components. </li></ul></ul><ul><ul><li>Image-Based Testing deals with images. </li></ul></ul><ul><ul><li>IBT doesn’t look for any object hierarchy </li></ul></ul><ul><ul><li>Application Map or Object Map is defined by images not by properties </li></ul></ul><ul><ul><li>All possible Image variations needs to be stored </li></ul></ul>
    9. 9. SAFS Image-Based Recognition Syntax <ul><ul><li>SAFS keeps concept of Components in Windows </li></ul></ul><ul><ul><li>A “component” can be: </li></ul></ul><ul><ul><ul><li>an image inside another image. an image inside the bounds of other images. </li></ul></ul></ul>
    10. 10. Component as an Image inside another Image <ul><li>Finding the right Arrow in a Sea of Matching Arrows </li></ul><ul><ul><li>Locate the Unique Anchor that contains the Target </li></ul></ul><ul><ul><li>Locate the Target within the Anchor </li></ul></ul><ul><li>Component Function call: </li></ul><ul><li>BulletsItem, ArrowButton, Click </li></ul><ul><ul><li>A “component” can be: </li></ul></ul><ul><ul><ul><li>an image inside another image. an image inside the bounds of other images. </li></ul></ul></ul>BulletsItem ArrowButton
    11. 11. Component as an Image inside the bounds defined by other Images <ul><li>Finding a Target inside the bounds from a single image </li></ul><ul><li>Component Function call: </li></ul><ul><li>OfficeWin, NewSlide, Click </li></ul><ul><ul><li>A “component” can be: </li></ul></ul><ul><ul><ul><li>an image inside another image. an image inside the bounds of other images. </li></ul></ul></ul>OfficeWin NewSlide
    12. 12. Component as an Image inside the bounds defined by other Images <ul><li>Find a Target inside the bounds defined by multiple images </li></ul><ul><li>Component Function call: </li></ul><ul><li>OfficeWin, CenterText, Click </li></ul><ul><ul><li>A “component” can be: </li></ul></ul><ul><ul><ul><li>an image inside another image. an image inside the bounds of other images. </li></ul></ul></ul>OfficeWin CenterText
    13. 13. SAFS Image-Based Recognition Syntax <ul><ul><li>Component is an image inside another image. </li></ul></ul><ul><ul><ul><li>[BulletsItem] BulletsItem =“Image=pathToBulletsItemImages” </li></ul></ul></ul><ul><ul><ul><li>ArrowButton=“Image=pathToArrowButtonImages” </li></ul></ul></ul><ul><ul><li>Component is inside bounds defined by other images. </li></ul></ul><ul><ul><ul><li>[OfficeWin] OfficeWin =“Image=pathToOfficeAnchorImages; _ ImageRight=pathToOfficeCloseIcons” </li></ul></ul></ul><ul><ul><ul><li>CenterText=“Image=pathToCenterTextImages” </li></ul></ul></ul>BulletsItem ArrowButton OfficeWin CenterText
    14. 14. How to automate using IBT <ul><ul><li>Test Development for IBT is the same </li></ul></ul><ul><ul><ul><li>Fewer Component Functions supported. </li></ul></ul></ul><ul><ul><ul><li>Tests and commands look the same as OBT. </li></ul></ul></ul><ul><ul><li>App Map development is mostly the same </li></ul></ul><ul><ul><ul><li>Recognition strings are VERY different. It’s all about images, not components. </li></ul></ul></ul><ul><li>e.g. </li></ul><ul><li>IExplorer App Map Entry:[IExplorer] IExplorer=&quot;Image=<imagepath>&quot; </li></ul>
    15. 15. How to automate using IBT <ul><ul><li>Capture images in supported formats </li></ul></ul><ul><ul><ul><li>BMP, GIF,JPEG,PNG,TIFF etc. </li></ul></ul></ul><ul><ul><li>Map logical names for the &quot;window&quot; and the &quot;component&quot; in the application map </li></ul></ul><ul><ul><li>Write test recordsscripts same as OBT </li></ul></ul><ul><ul><li>Execute records using SAFSDRIVER </li></ul></ul><ul><ul><li>Review the results </li></ul></ul>
    16. 16. Image Manager Tool <ul><ul><li>Tool to facilitate image captures </li></ul></ul><ul><ul><ul><li>Command to launch </li></ul></ul></ul><ul><ul><ul><ul><li>java org.safs.image.ImageManager </li></ul></ul></ul></ul>
    17. 17. Enhancements BitTolerance|BT= Optional. Specifies the integer percentage (1-100) of image bits or pixels that must match for an image to be considered a successful match. The default is, of course, 100. This means ALL pixels must match unless some other BitTolerance is specified. Samples: IExplorer=&quot;Image=<imagepath>;BitTolerance=70&quot; IExplorer=&quot;Image=<imagepath>;ImageR=<imagepath>;BT=75&quot;
    18. 18. Sample Application Map [SampleApplication] SampleApplication=&quot;Image=c:ImagesAnchorImage.bmp;ImageR=c:ImagesCloseIcon.bmp&quot; ButtonMinimize=&quot;Image=c:ImagesMinIcon.bmp&quot; ButtonMaximize=&quot;Image=c:ImagesMaxIcon.bmp&quot; ButtonClose=&quot;Image=c:ImagesCloseIcon.bmp&quot;
    19. 19. Sample Test Records C SetApplicationMap Demo.MAP C LaunchApplication SampleApplication &quot;c:safssamplesDotnetDotNetAppWinDemo.exe&quot; C WaitForGUI SampleApplication SampleApplication 15 T SampleApplication SampleApplication GetGUIImage c:OutputImage1.jpg T SampleApplication SampleApplication RightClick T SampleApp SampleApp InputKeys &quot;x&quot; T SampleApplication SampleApplication GetGUIImage c:OutputImage2.jpg T SampleApplication ButtonClose Click
    20. 20. Dealing with Variations in the IBT Environment <ul><ul><li>32-Bit, 24-Bit, 16-Bit Color Depth and Resolution </li></ul></ul><ul><ul><li>Themes and Schemes </li></ul></ul><ul><ul><li>Platform and Version Image and Icon Variations </li></ul></ul><ul><ul><li>  Multiple images are necessary if the target image is different in different environments </li></ul></ul><ul><ul><li>Identify and Store Item Variations in a Directory </li></ul></ul>
    21. 21. Dealing with NLS Testing in an IBT Environment <ul><ul><li>Same as any Recognition String handling for NLS: </li></ul></ul><ul><li>Item1=pathTo{^SHARED_DIR}targetImages Item2=pathTo{^LANG_DIR}targetImages </li></ul><ul><ul><li>Images will have to be captured and transferred if there is a language dependency </li></ul></ul>
    22. 22. Challenges <ul><ul><li>It’s all about images </li></ul></ul><ul><ul><ul><li>Require to capture all possible image variations </li></ul></ul></ul><ul><ul><ul><li>PreciseReliableUnique Images </li></ul></ul></ul><ul><ul><ul><li>Display configurations </li></ul></ul></ul><ul><ul><ul><li>Time consuming test development </li></ul></ul></ul><ul><ul><ul><li>Verification </li></ul></ul></ul><ul><ul><ul><ul><li>Image verification only </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Can’t verify component properties </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Can’t extract data from application under test </li></ul></ul></ul></ul><ul><ul><li>Themes and Schemes </li></ul></ul><ul><ul><li>Platform and Version Image and Icon Variations </li></ul></ul><ul><ul><li>Identify and Store Item Variations in a Directory </li></ul></ul>
    23. 23. Demo <ul><ul><li>What the IBT demo does </li></ul></ul><ul><ul><ul><li>Launch Sample Application </li></ul></ul></ul><ul><ul><ul><li>Perform few interactions </li></ul></ul></ul><ul><ul><ul><li>Close </li></ul></ul></ul><ul><ul><ul><li>Review the results </li></ul></ul></ul>
    24. 24. Q&A
    25. 25. Thanks  <ul><ul><li>References : </li></ul></ul><ul><ul><ul><li>SAFS Image-Based Recognition http://safsdev.sourceforge.net/sqabasic2000/SAFSImageBasedRecognition.htm </li></ul></ul></ul>

    ×