Mobile usability testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Mobile usability testing

  1. 1. Mobile usability testing Problems and solutions The report at the conference "Quality Assurance: Management & Technologies" November 1-2, 2007 BugHuntress QA Lab 2008 1 (c) 2008 BugHuntress QA Lab
  2. 2. Mobile usability testing Problems and solutions The report at the conference "Quality Assurance: Management & Technologies" (QAMT Ukraine 2007), November 1-2, 2007 Nowadays the software market is quite words, software developers concentrate saturated. A certain task can be solved by on valid execution of functionality while dozens of competing programs with al- user interfaces are elaborated last. most identical functionality. Ultimately, the choice for one or another application is Therefore, the issue of usability is quite increasingly based on non-functional fea- acute. tures. And a user interface which provides This report covers the following items: easy and effective work with a software product can play one of the crucial roles • Usability, what it means and provides when making such a choice. • Usability standards BugHuntress QA Lab Company was • Typical mobile peculiarities founded in 2001. Our 6-year experience in • Mobile usability problems independent software testing shows that high-quality user interfaces occur in about • Successful solutions 70% of desktop applications. At the same • Usability testing automation time a similar number for mobile applica- • Independent usability testing and its role tions is merely about 40%. In other What usability is So what is usability? chances that a user will buy it are poor. If a software product didn't pass ISO standard defines usability as "the ex- such face-control and was not bought, tent to which a product can be used by all the rest including functionality sim- specified users to achieve specified goals ply loses meaning. with effectiveness, efficiency and satisfac- tion in a specified context of use". • Simplicity and intuitive intelligibility of the interface (respectively, quick The key words here are the last three - learning capability). Nobody would like effectiveness, efficiency and satisfac- the idea of sitting for several hours in tion while working with a program. order to learn and memorize one and What is usability composed of and what only operation. If there are other pro- does it give to a user? There are several grams that give the same result and interpretations. We will not go into theo- can be mastered on the first attempt, retical details here. Let's emphasize the one would prefer this more simple and points which are crucial for product suc- user-friendly system. cess in the market. • Ease and forethought of control ele- So an interface is "usable" if it has: ments, menu, dialog windows order, hot keys and so on which provide high • Aesthetic appeal and gives satisfaction speed of work with the program. when using the product. Due to this the program can be used If the product is not considered aes- with minimum time and effort con- thetically appealing, up-to-date and sumption. For instance, in business fancy (of course, it depends on a applications this means higher labor specified context of use and is not the efficiency of user. same for games and office suite), the 2 (c) 2008 BugHuntress QA Lab
  3. 3. • Taking into account scientific recom- • Friendliness and user support (and, mendations will provide little user fati- consequently, minor user errors). gability when working with the pro- This point is tightly related to the pre- gram. Again, this means high labor ef- vious one. It is vital in such cases as ficiency of the user within long time. medical systems, systems of produc- Such advantage of the system is fun- tion and transport control, accounting damental for many corporate custom- and analytical business applications, as ers. well as many others, where the price of correcting a mistake can be too ex- At last (but not least), pensive. Basic standards of the user interface How are qualitative user interfaces cre- • Microsoft* Developer Network (MSDN) ated? recommendations. There exists a range of guides and stan- ISO basic standard defines the term "us- dards providing a set of recommendations ability" and describes definitions related to on the development and testing of user it. It expands the general principles of us- interfaces. ability and also contains particular rec- ommendations. Here are some of the usability standards and guidelines: For Internet-applications there exists 508 Standard. It describes methods of building • ISO 9241-11:1998 Ergonomic re- static and dynamic web-pages. In MSDN quirements for office work with visual one can get acquainted with the rules of display terminals (VDTs); building user interfaces according to Mi- • Section 508 standard - Web-based crosoft ideology (for both Microsoft Win- Intranet and Internet Information and dows and Pocket PC / Windows Mobile Applications (Rehabilitation Act, platforms). § 1194.22); 3 (c) 2008 BugHuntress QA Lab
  4. 4. * All trademarks and product names mentioned in this report are the property of their owners. Mobile standards Currently mobile technologies are devel- • Symbian S60 Platform Visualization oping very quickly. Handhelds have their and Graphic Design Guideline; peculiarities crucial to ensure usability. • UIQ Style Guide; This is the reason why special guides and • Palm OS User Interface Guidelines. recommendations on usability for mobile platforms have been created. Here are To illustrate "usable" handhelds distinc- some of them: tions let's consider the following exam- ples. Left-handed mode A somewhat large share of users (and re- Indeed, a lot of mobile applications (e.g., spectively market share) belongs to left- games) imply interaction by means of sty- handed people. For usual desktop applica- lus. Mobile devices have small screens, tions left hand setting happens on the and it is necessary to design the graphic level of peripherals. For mobile applica- interface for both right-handed and left- tions it is already a question of interface handed people. If it is done, the screen is setting. not covered by the player's hand holding stylus while playing. Speed or satisfaction Small sizes of handheld screens lead to ance and speed of work, on the other the fact that mobile software developers hand. have to search for the compromise be- tween ease of use and subjective satisfac- It is worthy to note that a lot of optimal tion (in particular, reasonable density of solutions of this kind of problems are buttons, icons and other controls on the found at the stage of testing. screen), on the one hand, and perform- Landscape mode One more example of mobile specificity is grades. But for handhelds this is done a Landscape mode. Desktop computers do rather widely. And in this case the applica- not need this because their monitors are tion should support both portrait and rarely used by rotating them by 90 landscape screen formats. 4 (c) 2008 BugHuntress QA Lab
  5. 5. General problems of usability Here are some of issues which influence the quality of usability in the mobile software industry: • No established mobile software usability culture; • Lack of trained specialists; • Direct usage of PC-approaches; • Direct porting between different mobile and embedded platforms; • Fast-paced software market environment. 1. First, many programmers who cur- many specialized colleges and univer- rently work on creation of the mobile sities. Some of them propose a course programs were trained to develop PC "Ergonomics of Web-site". But this is software. During decades of PC soft- extremely insufficient for development ware development practice there of successful interfaces for desktop, emerged programs and interface li- mobile or embedded applications. braries that became a standard de facto. Even programmers who are not 4. The next problem is "direct" porting quite experienced in what concerns between different mobile platforms. In usability can borrow such standard in- the struggle for the market share, terfaces, "automatically" creating fairly software developers aim at making usable programs. their programs support diverse mobile platforms and diverse models of hand- In mobile industry the situation differs helds. However, since the platforms a lot. Usability culture of this kind does are technologically different the pro- not exist here yet. Conventional pro- grams are to be re-developed for al- grams and libraries are few so far. most each of them. Accordingly, their Moreover, for each platform there is its interface must take into account archi- own standard, the platforms are tech- tecture and phone model, too. Thus, to nologically diverse, and they look as be successful, a programmer has to well as operate differently. acquire a wide range of knowledge. Such professionals are few, and very 2. For this reason, approaches accepted often programs migrate among plat- for PC are applied to mobile software. forms without changes in user inter- Developers directly transfer an ideol- face which affects usability dramati- ogy of PC development to the devel- cally. opment of programs for handhelds. As a result, there emerge programs for 5. Currently software development pro- cell phones and PDA with interfaces jects are generally time-restricted. "To rather satisfactory if run on PC but make faster" is the motto of our time. never on a handheld. On a handheld However, "faster" and "thought-out" they are inconvenient, bulky and so- do not always combine in one project. phisticated. In the long run one has to sacrifice something. High-quality interface de- 3. Lack of trained specialists. This prob- sign is often the very thing that is sac- lem in Ukraine has its specificity. A rificed to a tight schedule. subject about user interface develop- ment is absent in the curriculum of 5 (c) 2008 BugHuntress QA Lab
  6. 6. Successful solutions Nevertheless, the problems we talked cessfully adapts the content of the pages above are solvable. search en- displayed to sizes and other peculiarities gine is a bright example of this. This inno- of mobile devices and cell phones. vative system is built on the basis of Web 2.0 and mobile technologies. It can be Recommendations of testers, working in used on both desktops and handhelds. At our company, also contributed to reaching present a beta-version of the system is this result. released. However, it takes into account differences of diverse handhelds and suc- Automation of usability testing At present automation tools are actively erate Web application screenshots on dif- used for software testing. Usability testing ferent systems with various system set- has its peculiarities here. tings (resolution, brightness, etc.). enables to significantly In fact, the concept of usability is tightly reduce the testing time. associated with the subjective evaluation of a user. As a result, it defies algorithmi- service is an exam- zation and respectively automation. ple of automation tools intended for mo- bile applications testing. For this reason the tools for testing auto- mation are scarce and cover only separate In a similar way some other products can activities, but not the testing process as a be used. They don't perform usability test- whole. Among them we can mention ing in full but allow making it easier and service. It helps to gen- faster. 6 (c) 2008 BugHuntress QA Lab
  7. 7. Independent usability testing What can be recommended to the compa- also give professional recommendations nies which want to create qualitative soft- as to interface improvement. ware, also in terms of usability? The factor of testing independence itself is In many cases an acceptable and effective becoming of greater importance. In par- solution can be cooperation with an inde- ticular, independent testing allows obtain- pendent testing company. ing an unbiased attitude to the software quality. In our practice we had the cases Indeed, it immediately resolves the prob- when an internal testing team was pushed lem of training specialists' scarcity. Sec- to rather conceal bugs than detect them. ondly, the personnel of such companies In such situation independent testing is are able to not only detect problems, but the best way to avoid problems when re- leasing the product to the market. Conclusion The users have always been and will be Testing companies play a significant role interested in the fact that the program is in usability improvement. It is they who convenient and effective in use, as well as guard interests of the end user. Also, they satisfactory to work with. On the other are ready to propose their experience and hand, high-quality usability is a weighty resources to software producers to create competitive advantage of software solu- programs which would be highly appealing tions in the present-day oversaturated for a user and successful in the market. software market. BugHuntress QA Lab is a Ukrainian testing company founded in 2001. BugHunt- ress QA Lab offers outsourcing software testing & QA services, dedicated testing teams, and independent verification and validation (IV&V) services. The core expertise of BugHuntress QA Lab is testing of software for mobile and wireless devices; system software; embedded systems, multimedia hardware- software complexes; WAP, Web 2.0, e-Business applications. BugHuntress QA Lab performs automated and manual testing of functionality, usability, load, perform- ance, localization, and other types of testing. For more information contact All rights to trademarks, product names, and brands mentioned in this document belong to their owners. 7 (c) 2008 BugHuntress QA Lab