More Related Content
What's hot
What's hot (20)
Similar to S313352 optimizing java device testing with automatic feature discovering
Similar to S313352 optimizing java device testing with automatic feature discovering (20)
Recently uploaded
Recently uploaded (20)
S313352 optimizing java device testing with automatic feature discovering
- 2. Optimizing Java Device Testing with Automatic Feature Discover y Fyodor Romanov, Software Engineer
- 5. Approach how to optimize and shorten device test-cycle using Diagnostic tests to explore device capabilities.
- 6. Demo showing Diagnostic tests usage and benefits
- 9. Before a manufacturer can make a device commercially available, they must perform a significant amount of testing to ensure a high-quality, reliable product that delivers customer satisfaction.
- 10. Java Device quality testing helps Java technology wireless device manufacturers and network operators maximize product quality and minimize time to market.
- 11. Problem description There are a lot of (more then 30) Java Platform, Micro Edition (Java ME platform) technologies (and combinations):
- 13. Up to 3 weeks for guru to run all test on one device.
- 14. Challenge of managing and, if possible, lowering quality testing costs.
- 15. The most challenging task for test execution is to configure tests for particular device.
- 16. Express testing: evaluate device quality in 4 hours.
- 18. Problem statement Device manufacture knows, but testing house doesn't.
- 19. Before Automatic feature discovery.
- 22. Solution implementation Implementation example will be based on JDTF test framework however it should be applicable to any Java device testing framework. The Java Device Test Framework (JDTF) is a test framework based on Oracle‘ s commercial Java Device Test Suite (JDTS) product. JDTF is a general purpose, fully-featured, flexible, and configurable test framework suited to assess various aspects of Java Platform, Micro Edition (Java ME) device implementation quality .
- 23. JDTF overall tests structure JDTF organizes tests into hierarchical structure s called test packs , a collection of tests that are functionally related and have common setup requirements :
- 24. JDTF testing structure Harness is using tests and different configuration files to execute test on device. Test execution on device is controlled by MicroAgent which starts/stops test and communicates with Harness (sends results, logs, etc.)
- 27. Move them into special dedicated test pack.
- 28. Writing diagnostic test. Changes in MicroAgent. Diagnostic tests find out whether the device supports an optional feature and export a property by using Logger class as follows. public class Logger { ... public void exportProperty(String name, String value) { LogRecord rec = new LogRecord (...); rec.setProperty(EXPORTED_PROPERTY_PREFIX + name, value); ... } }
- 30. Test should be marked with ‘diagnostic’ keyword
- 31. Test should try to use optional feature and call Logger.exportProperty(String name, String value)
- 32. If feature is supported, return PASSED and if feature is not supported return FAILED. This helps to visualize test result.
- 33. Do nothing else
- 34. Writing diagnostic test. Code Sample. public class ConnectionTest implements AutomatedTest { ... public boolean testHTTP() { Connection c = null; try { ... c = Connector.open(url); logger.exportProperty("HTTPSupport", "true"); return true; } catch (IllegalArgumentException iae) { return false; } ... }
- 36. Adding Template Builders to TestPacks
- 37. Changes in FW
- 38. Final touch. Adjusting Relevance Filters Relevance filter should filter out not relevant test providing right reason. Code sample: public String LapiDependencyProvider.isRelevant(final TestId id) throws TestDependencyProviderException { final boolean createDeleteLmsSupported = getBoolean(TestPackProperty.CreatingAndDeletingLandma rkStoresSupported, id); if (!createDeleteLmsSupported) { if (fqn.endsWith("#addLandmarkScenario2") || fqn.endsWith("#deleteLandmarkFromAnotherLMS") { return "Test not relevant because Landmark Store create/delete operations configured to be not supported."; } }
- 40. Reduces 1 - 2 days work for one person (depending of person's experience) to 30 minutes.
- 41. Before Automatic Feature Discovery feature customers were taking a complete unknown device and painstakingly reading logs to find out what it supports .
- 42. DEMO
- 44. JDTS home page with API detailed description
- 47. How to process Diagnostic tests output in testing framework JDTF test framework was used for demonstration <Insert Picture Here>
- 48. Q&A Q&A
Editor's Notes
- The growing complexity and diversity of devicebswith their varying operating systems, processors, and memory configurationbsincreases the need for thorough testing. At the same time, service providers and manufacturers face the challenge of managing and, if possible, lowering in-house costs. The Java Device testing helps device manufacturers and service providers ensure their reputation for quality, while building customer satisfaction and loyalty.
- Java ME technology was originally created in order to deal with the constraints associated with building applications for small devices. For this purpose Sun defined the basics for Java ME technology to fit such a limited environment and make it possible to create Java applications running on small devices with limited memory, display and power capacity. Java ME platform is a collection of technologies and specifications that can be combined to construct a complete Java runtime environment specifically to fit the requirements of a particular device or market. The Java ME technology is based on three elements; * a configuration provides the most basic set of libraries and virtual machine capabilities for a broad range of devices, * a profile is a set of APIs that support a narrower range of devices, and * an optional package is a set of technology-specific APIs.
- Failture analisys becomes complicated because even if a technology is supported, most of them have optional features which also can be unsupported. For example, JSR293 Location API 2.0 declares following optional features: Creating and deleting landmark stores, including private landmark stores Adding, removing and renaming landmark categories Adding and removing LandmarkStoreListener
- Usually there are no proper available documentation for device and it’s manual and error-prone task to discover what technologies and optional features device supports and update special configuration file (template) accordingly.
- Identifying what technologies device supports includes writing so-called Diagnostic tests that check particular optional feature by trying to use it and then analyze tests logs. Based on that user should set properties in device configuration file (template). I t is time-consuming to have to read through the log file and then interpret and w rite values generated there into configuration file. T hings that make it cumbersome are : Value key names may not match correctly: example Media Format names like MP4 Values are in different places, i.e. not grouped by feature/function/JSR.
- Use Diagnostic tests to explore a device’s capabilities, such as : the presence of optional APIs, the presence of JSRs using Class.forName() canvas size, a number of colors ,… etc and report back to the harness. After running the automated diagnostic tests on a device, AUTOMATICALLY create a template that is customized for the device. In the generated template tests that cannot pass because of a missing feature, are filtered out. Using such a template saves testing time by running only tests that are relevant for a device.
- When configuration is finished and test execution is not started, JDTF harness calls an optional test pack method for each test case. This method inspects the configuration and returns null if the test is relevant or a list of reasons describing why the test is irrelevant. An irrelevant test is one that is pointless to run because o ne or more properties in the configuration, which essentially represents the capabilities of the test device, makes a test irrelevant. For example, a property representing the presence of an optional device capability is set to false.
- * Represents an entity that can convert device profile properties * to the test pack configuration values. * This interface intended to be implemented by the API clients. * This method maps Device Profile properties to test pack properties which * are put in the device profile template. * @param diagnosticsProperties a map which binds a test. pack id with a map * consisting of device profile properties and their values. * @return a map of test pack properties and their values.
- Demo includes running JDTF with Automatic Feature Discovery feature implemented, running couple of diagnostic tests, generating template and demonstrating that appropriate tests are filtered out with proper reason.