Sybase Unwired Platform Test Tool


Published on

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Sybase Unwired Platform Test Tool

  1. 1. SAP How-to GuideMobilitySybase Unwired PlatformHow To ... Test the Sybase Unwired Platform:Available Test Tools A Branded Service provided by SAP Customer Solution AdoptionApplicable Releases:Sybase Unwired Platform 2.1Version 1.0March 2012
  2. 2. All other product and ser vice names mentioned ar e the tr ademarks of their resp ectiv e co mpani es. Dat a cont ained in this document serves information al purposes only. N ational product specifications may vary. The information in this document is proprietar y to SAP. No part of this document may b e rep roduced, cop ied, or tr ans mitted in an y form or for any purpos e without the express prior writ ten permis sion of SAP AG. This document is a preli minary v ersion and not subject to your licens e agr eement or any other agr eement with SAP. This document contains only int ended str at egies, d evelop ments , and functionalities of the SAP® product and is not intended to be binding upon SAP to an y parti cular course of business , product str ategy, and /or dev elopment. Pl eas e note that this docu ment is subject to ch ange and may be ch anged by S AP at any ti me without notice. SAP assu mes no r esponsibility for errors or omissions in this document. S AP does not w arr ant the accur acy or compl et eness of the information, text, graphics, links , or other items contained within this mat erial . This document is provided without a warrant y of any kind, either expr ess or i mplied, including but not limited to the i mplied w arr anties of mer chant ability, fitness for a particular purpose, or non-infringement. SAP sh all hav e no liability for damages of any kind including without limit ation direct , special, indir ect , or consequ ential d amages that may r esult from the us e of these materi als. Thi s limit ation shall not apply in cases of intent or gross negligence. The statutor y liability for p ersonal injur y and defectiv e products is not affected. SAP has no control over the information that you may access th rough the use of hot links contained in thes e materials and does not endors e your use of third-party Web pag es nor provide an y war ranty whatso ever rel ating to third-part y Web pages .© Copyright 2012 SAP AG. All rights reserved.No part of this publication may be r eproduced or tran smitt ed in any form or for any purpose wi thout the expr ess p er mission of SAP AG. Th e information cont ained her ein may b e changed wi thout prior notice.Some softw ar e products mar ket ed by S AP AG and its dist ributors contain propri et ary softw ar e component s of other softwar e vendor s.Microsoft, Windows , Excel, Outlook, and PowerPoint ar e r egistered tr ademarks of Microsoft Corporation.IBM , DB2, D B2 Univer sal Datab ase, Syst em i, System i5, S ystem p, S yst em p5, S yst em x , Syst em z, S ystem z 10, Syst em z9, z 10, z 9, iS eries, pS eri es, x Seri es, z Series , eSer ver , z/VM, z /O S, i5/O S, S /390 , OS /390, O S/4 00, AS /400, S /390 Parall el Enterp rise Ser ver, Po wer VM, Pow er Archit ectur e, P OWE R6+ , PO WER6 , PO WER5 +, PO WE R5, PO WER, OpenPow er , PowerPC, Bat chPipes, Bl adeC enter, S ystem Stor age, G PFS, H ACMP, RET AIN, D B2 Connect, RACF, Redbooks, O S/2 , Parall el Sysplex , MVS/E SA, AIX, Intelligent Miner, WebSph er e, N etfinity, Tivoli and Infor mix are tr ademarks or regist er ed trad emar ks of IBM Corporation.Linux is the registered tr ademark of Linus Torvalds in the U. S. and other countries. SAP “How-to” Guides are intended to simplify the product implement-Adobe, the Adobe logo, Acrobat, PostS cript, and Reader ar e eith er tr ademarks or regist ered t rad emar ks of Adobe S ystems In corporated in the United Stat es and /or other countries.Oracl e is a regist er ed trad emar k of Oracle Corporation.UNIX, X/Op en, OSF /1, and Motif ar e regist er ed tr ademar ks of the Open Group. tation. While specific product features and procedures typically areCitrix, ICA, Program N eighborhood, MetaFr ame, WinFrame, VideoFr ame, and MultiWin are tr ad emarks or regist er ed trad emar ks of Citrix Systems, Inc.HTML, XML, XH TML and W3C are t rad emar ks or r egister ed tr ademarks of W3C®, World Wid e Web Consortium, M ass achus etts Institut e of Technology.Jav a is a r egistered tr ademark of Sun Micro syst ems, Inc.Jav aScript is a r egistered tr ademark of Sun Microsyst ems, Inc., used under licen se for technology invented and i mplement ed by N etscape.SAP , R/3, S AP N etWeaver, Duet, Partn erEdge, ByDesign, S AP BusinessObjects Explor er, Str eamWork, and other S AP product s and se rvices mentioned herein as w ell as thei r r espective logos are tr ademarks or regist er ed trad emar ks of SAP AG in Germany and other co untries.Business Objects and the Business Objects logo, BusinessObjects, explained in a practical business context, it is not implied that thoseCrystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other features and procedures are the only approach in solving a specificBusiness Objects products and services mentioned herein as well as their business problem using SAP NetWeaver. Should you wish to receiverespective logos are trademarks or registered trademarks of Business additional information, clarification or support, please refer to SAPObjects Software Ltd. Business Objects is an SAP company. Consulting.Sybase and Adapti ve S erv er , iAnywh er e, S ybas e 365, SQL Anywhere, and other S ybas e products and services mentioned herein as well as thei r respective logos are tr ademarks or regist ered t rad emar ks of Sybase, Inc. S ybase is an SAP comp any. Any software coding and/or code lines / strings (“Code”) included in this documentation are only examples and are not intended to be used in a productive system environment. The Code is only intended better explain and visualize the syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, except if such damages were caused by SAP intentionally or grossly negligent. Disclaimer Some components of this product are based on Java™. Any code change in these components may cause unpredictable and severe malfunctions and is therefore expressively prohibited, as is any decompilation of these components. Any Java™ Source Code delivered with this product is only to be used by SAP’s Support Services and may not be modified or altered in any way.
  3. 3. Document HistoryDocument Version Authored By Description1.00 Patrick Kelleher First release of this guide
  4. 4. Typographic Conventions IconsType Style Description Icon DescriptionExample Text Words or characters quoted Caution from the screen. These include field names, screen Note or Important titles, pushbuttons labels, Example menu names, menu paths, and menu options. Recommendation or Tip Cross-references to other documentationExample text Emphasized words or phrases in body text, graphic titles, and table titlesExample text File and directory names and their paths, messages, names of variables and parameters, source text, and names of installation, upgrade and database tools.Example text User entry texts. These are words or characters that you enter in the system exactly as they appear in the documentation.<Example Variable user entry. Angletext> brackets indicate that you replace these words and characters with appropriate entries to make entries in the system.EXAMPLE TEXT Keys on the keyboard, for example, F2 or ENTER.
  5. 5. Table of Contents1. Business Scenario .....................................................................................................................12. Background Information ..........................................................................................................13. Prerequisites ............................................................................................................................ 24. Available Test Tools for testing SUP Mobile Apps ................................................................. 3 4.1 Functional Testing Tools ................................................................................................. 4 4.2 Load Testing Tools ........................................................................................................... 5 4.3 Performance Measurement Tools .................................................................................. 75. Appendix .................................................................................................................................. 9
  6. 6. How To... Test the Sybase Unwired Platform: Available Test Tools1. Business ScenarioThis document is intended to explain the current tools available for testing mobile applications builtusing the Sybase Unwired Platform (SUP). The intent is not show how to test mobile apps butrather describe which tools can be used and what types of testing can be performed based on thetype of mobile application.The focus of the testing will be on the performance and reliability of the SUP, not the clientapplication running on the mobile device.2. Background InformationThere are several different test methodologies, each have their own goals, procedures, and tools.In this document we will look at three areas of testing: functional testing, load testing, andperformance measurement.Functional testingThe goal of functional testing is to ensure the application or program works according to the userrequirements. For our purposes we will include unit testing, integration testing, and end-usertesting.Functional testing usually starts with a test case. In unit testing this the test class the developerwrites to test a unit of code. In higher-level functional testing, the testers have to follow a test casethat describes the steps they must follow in the user interface of the application and what theoutcome should be.The Integrated Development Environment (IDE) used by the development team usually haveframeworks for writing unit tests. Other functional testing is usually performed manually; howeverthere exist test management software, such as SAP Solution Manager, that manages test casesand tracks errors.Load TestingThe goal of load testing is to see if the application or system will fail under load. It is similar toperformance testing in that you want to see how the system performs, and it is similar to functionaltesting in that you are looking to see if there are bugs or poor implementation aspects that causethe system to fail at certain loads.In preparation for load testing it is important to collect appropriate test data, in terms of bothcontent and volume. Where possible, it is best to use production data to source the test scripts.Analysis then has to be performed on the number of users who are expected to use the system orapplication, their geographical location, and the expected volumes the system will have to handleduring its peak usage intervals.The result of the preparation should be a set of test scripts that will simulate multiple concurrentusers running different test cases and varying volumes of data. Each script might be writtenmanually by a tester; or it might be generated by recording the data and input entered when atester runs though a functional test case. LoadRunner is an example of a test tool that allows you torecord user input into a web form or a SAP GUI interface in this manner.Load testing also requires a test framework that allows you to run multiple scripts at once on atintervals, thus controlling the number and type of simulated concurrent tasks performed on theApril 2012 1
  7. 7. How To... Test the Sybase Unwired Platform: Available Test Toolssystem. There are many commercial and public test frameworks available that perform thisfunction.Performance MeasurementThe goal of performance measurement is to measure the system throughput while under load.Thus, performance measurement has the same requirements as load testing only the focus is onmeasurement and not functionality. The performance criteria that are measured duringperformance measurement are called metrics.The preparation of the test scripts should follow from the load tests. In addition, agents will need tobe installed and configured to measure the performance metrics. These agents might be part ofthe operation system, such as PerfMon; or they might be from a 3 rd party tool, such as WilyIntroscope.Characteristic Functional Load Performance Testing Testing MeasurementGoal Ensure the Ensure that the system Measure the Program/Application/ works under stress performance of the System works as intended system under stress and determine performance bottlenecksProcedure Write and execute unit Prepare test scripts and Configure monitoring tests and test cases execute tests using load agents prior to running testing framework load testsTools  Testing and  Recording tools for  Agents to measure debugging tools or generating test performance the IDE. scripts metrics  SAP Solution  Parsing tool to  Framework to Manager to manage parameterize test manage results test cases script  Analysis tools to  Test framework interpret results that simulates load and identify and collects results bottlenecks and points of failure.Table 2-1: Overview of the types of testing methods and associated tools3. PrerequisitesAs the intent of this document is to provide information on the tools available for testing MobileApplication developed with SUP, there are no hardware or software requirements as such.However, the reader should be familiar with the architecture of the Sybase Unwired Platform(, and should understand how performance ismeasured in SUP ( 2012 2
  8. 8. How To... Test the Sybase Unwired Platform: Available Test Tools4. Available Test Tools for testing SUP Mobile AppsBefore we look at how to test mobile applications built with Sybase Unwire Platform, we need tounderstand architectural types of SUP-based mobile apps. The SUP documentation differentiatesbetween two basic arch-types: mostly online apps and mostly offline apps. Mostly offline ordisconnected apps are defined by their cache. This cache is in sync with the SUP cache (CDB)which needs to stay in sync with the EIS source system.For our purposes we will define three types of mostly offline apps: apps that use MBS (MessageBased Synchronization), apps that use the DOE (Data Orchestration Engine) and apps that useRBS (Replication Based Synchronization). Technically DOE based apps use MBS but their serverside cache resides in the DOE and not in the CDB of SUP.For mostly online apps we’ll assume Sybase Workflow and Hybird Web Container (HWC) apps areessentially the same. We will also include OData based apps in this group. Mostly Offline (Cache Based) Mobile Mostly Online Mobile Applications ApplicationsTechnology RBS MBS DOE HWC ODataSynchronization Yes NoMobile Data Sybase UltraLite SQL Lite SQL Lite (XML) Native App PersistenceInner Protocol MobiLink iMO-TravellerCompression ZLibEncryption RSA AES 128 – AES 1024Wire Protocol Plain HTTP Encapsulated Binary over chunked HTTPSSL Yes NoTable 4-1: Mobile Application types and their associated technologies11 Copied from the „SUP Development Best Practice“ whitepaper by Martin AugstApril 2012 3
  9. 9. How To... Test the Sybase Unwired Platform: Available Test Tools4.1... Functional Testing ToolsFunctional testing usually takes place on the device. The only exception is unit testing anddebugging which is usually done in the IDE.Device IDE SimulatorWin32, Windows Mobile Microsoft Visual StudioiOS (iPhone, iPad) XCode (Mac only) Built-inAndroid Android SDK (eclipse based) Built-in (eclipse based)Table 4-1-1: IDE’s for popular mobile platformsAdditional tools for monitoring data transfer, debugging HTML/JavaScript, testing OData channels,and viewing cache data are also available.Purpose ToolSQL Lite Firefox Add-onSQL Anywhere/Advantage Sybase Mobile SDKDBWeinre HTML5/JavaScript debuggerFirebug Firefox Add-on (HTML5 debugger)Wireshark Protocol analyzer RESTful web service tester 4-1-2: Additional test tools for mobile application developmentThe Sybase Mobile SDK (eclipse based, formerly called the Unwired Workspace) offers tools to testconnections to various back-end EIS systems and view the results of test calls. It also providesdrivers for the Advantage and SQL anywhere database systems so you can view the contents of theMessage Server queue and the CDB.The ABAP workbench provides an IDE for testing and debugging OData Channel development inNetWeaver Gateway, and GET operations can be tested with any browser with a connection to theGateway server. Other REST operations can be tested with a REST client such as the RESTClientFirefox Add-on.HTML5 based OData clients can be debugged using Firebug or Weinre. These can also be includedin HWC based libraries for debugging (follow this link to see how to include Weinre can be includedApril 2012 4
  10. 10. How To... Test the Sybase Unwired Platform: Available Test Toolsin a HWC app: Load Testing ToolsIn order to load test mobile applications you would need to control multiple instances of theapplication running at once. There are several options:  Run the app manually on multiple mobile devices simultaneously (Gymnasium method)  Run the app on multiple virtual devices simultaneously  Emulate the app and run multiple instances on multiple virtual or emulated devices  Emulate the data (of multiple app instances running on multiple devices simultaneously) being sent to, and record the data being received from, the mobile platform (SUP) The first two options are only relevant if the expected peak usage of the app is low (< 200).Another consideration is: is really necessary to perform load tests on the device? Devicecharacteristics might affect the app performance on the device and thus influence the frequencyand duration of the connections and the amount of data carried across each connection (we’ll referto this as traffic).Nevertheless, this variance can be profiled in functional testing. The device itself is mostly a singleuser device; therefore, load testing the device is not required.If we omit the device from the testing then it should be enough to emulate the load on the SUP. Thelast two options propose two ways of emulating the test data: 1. Write test scripts that emulate the app (app centric). 2. Write test scripts that emulate the data traffic (data centric)The app centric approach is simpler but more effort. This requires writing a functional testprogram that works according to the business rules which govern the mobile application. The testprogram may be the app code or a self-contained section of the app code however it must conformto the test harness.The data centric approach requires a more complex approach. The wire protocol used by themobile app must be understood. A tool is required that is able to record or capture the data of asample test (execution of a particular use case of the app). The recording is then used to replay thewire protocol with other parameterized data, both consecutively and concurrently as determinedby the test harness.Both approaches require a test harness or framework. The test harness must be able to start andmanage multiple instances of the test program/script, collect log entries, manage connections tothe SUP server, and signal errors when they arise. It must also be able to control the data set usedfor testing and be able to control how data is fed to the various test instances.The following table summarizes the steps required for each approach: App Centric Data CentricSteps 1. Set-up and configure test 1. Set-up and configure test harness harness 2. Collect realistic test data (from a 2. Collect realistic test data (from a productive environment if productive environment if feasible) based on the number of feasible) based on the number ofApril 2012 5
  11. 11. How To... Test the Sybase Unwired Platform: Available Test Tools concurrent tests you plan to run. concurrent tests you plan to run. 3. Generate code stubs 4. Determine the use case for each 3. Determine the use case for each test test 5. Set log levels to log as much as 4. Set log levels to log as much as possible possible 5. Start recording/capturing all the data sent to the server 6. Run the use case for each test on 6. Run the use case for each test on a test device with the actual app a test device with the actual app 7. Use the use case, the business 7. Create scripts that parameterize process description and the fields in the recorded data. (The information collected from the test harness will substitute test logs to code the test scripts data for the parameterized fields.) 8. Replicate the device conditions on the test platform 9. Functional test the test scripts 8. Functional test the test scripts 10. Create a baseline for each test 9. Create a baseline for each test 11. Ramp-up the tests 10. Ramp-up the testsIssues Implementation specific Testing restricted to data not app Requires custom coding Need to disable encryption and compression Difficult to maintain if the app changes Test could break with new SUP version as it uses unpublished internal protocols Steep learning curve Requires Sybase development to enable a scripting interface Test traffic may differ for actual Currently not practical for cache- application based mobile applications Requires Sybase to expose method for Requires test framework to retrieve, setting the device id of the test client store and replay surrogate keys instance Both approaches require a concept to handle push notifications (DCN) Both approaches require a test user management utility to automate SUP device connection administrationLoad (or stress) testing tools currently available for testing SUP vary in their applicability to whatthey can test and where they can be used. Sybase has internal tools for load testing and a testharness developed for Sybase Professional Services to use for Sybase customers.April 2012 6
  12. 12. How To... Test the Sybase Unwired Platform: Available Test Tools Description Tested Technology Commercial UseHP High-performance test At least one RBS SAP has specialLoadRunner framework for load customer project MBS licensing for LR but this testing enterprise has used this to does not cover the .Net OData applications test SUP test license HWCJMeter Open source test No information to App Centric Commercially available framework for testing date approach only enterprise applicationsZAP-fix Commercial mobile No Information App Centric Commercially available testing toolPerfecto Commercial mobile No Information App Centric Commercially availableMobileCloud testing toolFloodgate Test Framework Used in several App Centric but Currently only available developed by Sybase SUP projects could be adapted via SPS for Sybase Professional to Data Centric Services (Consulting)Benchmark Test Framework Used internally App Centric used Not commerciallyApplication developed by Sybase primarily for RBS availableFramework for internal and MBS.(BAF) performance testing Not tested with and measurement OData or HWCMobiLink Command line program Used Internally RBS only Not bundled with SUPreplay utility used to “replay” RBS but available as a(mlreplay) data transfers. download from SybaseHP LoadRunnerLoad Runner is licensed through SAP for testing ABAP based systems and Web (HTML) interfaces.Currently there are only two methods for testing SUP with LoadRunner: 1. Custom C# program that consumes the MBO objects as described in the App Centric approach described above. This requires a .Net license for LoadRunner that is not part of the licensing agreement sold by SAP. SAP is working with HP to develop a licensing agreement for testing SUP. 2. Citrix client: this does not work for every app type but for MS Windows based apps Citrix can be used to record and parameterize the user interface of the app and replay the virtualized app for each test instance. This method works but requires a lot of hardware to simulate more than 100 client devices. The use of LoadRunner with the Citrix client is part of the SAP license.4.3... Performance Measurement ToolsPerformance measurement tools might be part of the operating system, part of the platform ormight use hooks and or plug-ins into the software/platform in order to allow the tools to measureApril 2012 7
  13. 13. How To... Test the Sybase Unwired Platform: Available Test Toolsperformance metrics of the software being tested. The following is a list of test tools that measureperformance and performance measuring tools:Purpose ToolWily Introscope Pure performance measurement tool that uses code injection (Aspect oriented programing) in Java to hook into the SUP to measure response times and other metrics.HP Load Runner Measurement instrumentation built into the test tool to monitor response timesBAF Internal reports provide monitoring information on:  OS Level  SUP Server internals  User-defined statisiticsMLMon Part of MobiLink, monitoring tool for the database replication that takes place during RBS synchronization.PerfMon Microsoft Windows operating system level monitoring for performance usage metrics of the system such as:  Network usage  CPU usage  Memory usage  File I/OApril 2012 8
  14. 14. How To... Test the Sybase Unwired Platform: Available Test Tools5. AppendixReferences:Tool ReferenceMicrosoft Visual Studio Mobile id=9263XCode SDK Anywhere/ DBWeinre LoadRunner product.html?compURI=tcm:245-935779JMeter MobileCloud real_handsets.htmlMobiLink replay utility .12.0.1/mlserver/ml-utilities.htmlWily Introscope HIER&HEADER=Y&FUNCTIONBAR=N&EVENT=TREE&NE=NAVIGATE& ENR=01200314690900003513&V=INST&TA=ACTUAL&PAGE=SEARCHMobiLink monitor 05.0155/doc/html/aba1270578277892.htmlutility (mlmon)April 2012 9
  15. 15.