914 IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010machine. There are several works to apply the hapticinterface to games. Yuichiro Sekiguchi applied the hapticeffect to a game, proposing a device that gives a user theillusion of a virtual object inside the device when shaking itusing accelerators and actuators . Wanjoo Park et al.proposed a brickout game using a haptic dial interface andsome haptic effects for the game . Recently a remote control manufacture announced auniversal remote control with tactile feedback and touchscreen. When the user touches the button, the remote generatesimple vibration. III. IMPLEMENTATION OF THE CONTROL Fig. 3. System architecture of the haptic dial interface. In this section, we explain in detail the hardwareconfiguration of the haptic dial knob. A block diagram of the IV. GRAPHICAL USER INTERFACE (GUI)system is shown in Fig. 2. GUI for the universal remote control consists of main (home) menu, TV channels, room temperature, and room lighting. The main menu has three icons that represent three sub menus to control home appliances. The user can select one of them by touching an icon. He/she can go back to the main menu by pressing the right button (we called it ‘home button’) anytime and select other home devices (Fig. 4(a)). (a) Motor, Control unit PCB and Touch screen module (a) Main menu (b) TV channel (b) Haptic remote control Fig. 2. Implementation of the universal haptic remote control. MCU (Micro Controller Unit) computes the torqueamount at a given angular position and sends the commandto a DC motor via a DAC (Digital to Analog Convertor).The DC motor executes the command and generates various (c) Room temperature (d) Room lightinghaptic patterns. We use a gear box of 5:1 ratio to change theangle of rotation axis. The dial knob is installed on the Fig. 4. Graphical user interface for the universal remote control. Mainmotor gear box, allowing the user to rotate the knob and to menu (a), TV channel (b), room temperature (c) and room lighting (d)feel various haptic effects which are programmed along theangular position. An encoder measures the angular position TV channel menu shows the channel list and informationwith 1000 pulse per turn and an encoder counter has a 24- about current selected channel. The channel numbers arebits quadrature counter and 25MHz count frequency. A displayed on the rounded band to go well with rotationalTouch LCD Module is used to display visual information action of the dial knob. Some numbers has the heart mark thatand to allow touch input. indicates popular channels or favorite channels programmed
L. Kim et al.: A Universal Remote Control with Haptic Interface for Customer Electronic Devices 915by the user. When the user selects the channel with the heart The torque profile for the detent effect is shown in Fig. 5.mark, he/she feels different tactile feedback. It helps the user By modulating A and b, the amount and frequency of torqueto find intuitively the channel that he/she may want to see (Fig. can be changed. We set A to a bigger value for the popular or4(b)). favorite channels having a heart mark in Fig. 4(b) so that the In room temperature menu, the user can select cooling or user can identify the channels intuitively. For instance,heating function by touching the screen and control the room channel 7 is a favorite channel and channel 5, 6, and 8 are nontemperature by spinning the dial knob. Yellow point indicates popular channels (Fig. 5).the current temperature the user select. The user spins the dialeasily in the proper temperature range. But he/she feels strong B. Room lighting controltactile feedback out of the range (Fig. 4(c)). Friction haptic effect is used to adjust the room brightness Room lighting menu shows a light bulb to visualize room level by rotating the dial knob. The friction effects generatelighting. As the user rotates the dial knob, the brightness of resistant torque opposite to the direction of movement as athe bulb is changed to indicate the room lighting level. The movement-based effect. This is implemented based on theuser feels the increasing/decreasing friction when he/she spins friction con model . The friction torque can be calculatedthe dial clockwise/counterclockwise. It gives very institutive by eq. (2).interface visually and haptically (Fig. 4(d)). The red pointindicates the current brightness level on the bulb. Lf exp(Pnow (n) / S f 1 ) V. HAPTIC EFFECTS Pcurr _ f (n) Pprev (n 1) ( Pnow (n) Ppre (n 1)) S f 2 In this section, we describe how to implement various Pdiff (n) ( Pnow (n) Pcurr _ f (n)) L fhaptic effects for the universal remote control. We designed (2) Pprev (n) Pcurr _ f (n)various haptic effects which are defined by adjusting torqueprofiles along the angular position and time. Haptic effects if Pdiff (n) Tf _ max then Tf (n) Tf _ maxinclude detent, friction, hard stop, and a combination of theseeffects. These haptic effects help the user to browse and find else if Pdiff (n) Tf _ min then Tf (n) Tf _ mininformation easily and intuitively. else Tf (n) Pdiff (n) A. TV channel control In order to give a feeling to select a channel to the user, we where Pcurr_f is the current position, Pnow is the angularuse the detent effect. The detent effect simulates notches with position of the dial knob, Ppre is the previous position, Sf1 anddifferent height along the angular position using sine Sf2 are the scaling factors, Pdiff is the difference of position, Lffunctions. Feeling a notch confirms each channel selection. Eq. is the friction level, Tf is the friction torque.(1) shows the detent effect which is defined by a sine function. Lf (friction level) increases exponentially as the brightness level increases since the exponential increasing make much better feeling than the linear motion (Fig. 6). We designed the fd ( ) A sin(b ) (1) friction profile depending on the dial knob’s movement. When the dial knob is rotated slowly, the rotational friction is low. where, A is the amplitude, b is the number of notch per On the other hand, when the user spins the dial fast, he/sheturn, is the rotation angle of the dial knob. feels more string friction. Fig. 7 shows the profile of the friction according to the knob movement. Haptic Effect - Channel Interaction 2 Haptic Effect - Dimming Interaction 30 1.5 25 1 0.5 20 Friction Level Torque 0 15 -0.5 10 -1 -1.5 5 -2 0 5 5.5 6 6.5 7 7.5 8 8.5 0 50 100 150 200 250 300 350 400 Channel Angle Fig. 6 Friction level by angle position Fig. 5 Haptic profile by channel changing
916 IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010 3.5 Haptic Effect - Dimming Interaction where, T is temperature, Sb is scaling factor, AT is the amplitude, b is the number of notch per turn, is the rotation 3 angle of the dial knob. level 0.1 2.5 level 1 level 5 level 15 VI. APPLY TO GAME CONTROLLER 2 level 25 Torque We applied the universal remote controller to a game controller 1.5 that gives the user haptic feedback during the game. For this, we developed a new brickout game in which conventional bricks are 1 replaced by banana bricks, a game ball is replaced by a monkey, 0.5 and a paddle is replaced by people holding up boards. In addition, a new item is added, a cloth wrapper. 0 0 10 20 30 40 50 60 70 Movement of Dial Fig. 7 Torque profile comparison of levels The friction effect help the user feel the brightness Bricksintuitively along with the visual effect. C. Room temperature control In order to give a haptic feedback for room temperature Ballcontrol, we used a combination of detent, friction, and hardstop effect. This effect is designed to guide the user to feel the Paddleproper temperature range with tactile feedback. The propertemperature for cooling ranges from 23 to 26 centigrade and Fig. 9 Development of a new brickout gameone for heating ranges from 19 to 22 centigrade. In the proper temperature range, the user can feel onlydetent effect and adjust the temperature easily. But if the usertries to set the temperature out of the proper range, he/shefeels the additional friction force proportionally to distancefrom the range. In the temperature range lower 16 and above30 centigrade, the user feels very strong resistance force (hardstop) in Eq. (3). Haptic Effect - Cooling/Heating Interaction 8 6 4 Torque / Friction Level 2 Fig. 10 Playing a brickout game using the universal remote control 0 The banana bricks work the same as general bricks and the -2 cloth wrapper contains haptic items. The game ball bounces off the top or side of the wall depending on the collision -4 conditions between the ball and the paddle. When the ball hits -6 Torque profile a cloth wrapper, the haptic item falls down. If the user catches Friction level of Cooling the item by moving the paddle, a predefined haptic effect is Friction level of Heating -8 felt that lasts a few seconds. This haptic feedback can make 14 16 18 20 22 24 26 28 30 32 Temperature the game easier or more difficult. Fig. 8 Haptic profile and friction level by temperature VII. USER STUDY Fig. 8 shows the torque profile with regards to the We performed a user study to evaluate usability of thetemperature. universal remote control. A. Usability factors fT (T ) (T 16) 2 / Sb , (T 16) For the user study, we followed a guideline suggested by AT sin(b T ), (16 T 30) (3) Jinwoo Kim  who defined usability factors with a (T 30) 2 / Sb , (30 T ) hierarchical structure. We choose appropriate factors to measure the usability of the universal remote control (Fig. 11).
L. Kim et al.: A Universal Remote Control with Haptic Interface for Customer Electronic Devices 917 Efficiency Responsiveness 4. TV Channel control: Now, TV shows some advertises. You Basic domain change the TV channel to number 17 for searching another Minimal action program. Accuracy Error prevention After completing the tasks, subjects were asked to answer a Error recovery questionnaire. The questionnaire for the user study is divided Meaningfulness Understandability into three sections such as general control (touch screen), TV Concomitant domain channel control and temperature control. The light control Consistency Predictability section is excluded because it is not popular way to control by Familiarity a remote control. Fig. 11 Evaluation factors of usability suggested by Jinwoo Kim. There are five questions excepting responsiveness and recovery factor in general section. For channel and temperature section, there are eleven and ten questions each Evaluation factors in the first level consist of basic and section with seven factors. Questions on the questionnaireconcomitant domains. In the second level, the basic domain were 7-point (-3 to 3) Likert scales.has two factors; efficiency and accuracy factor. Efficiency C. Resultfactor is about how effectively users can achieve given tasksin terms of responsiveness and minimal action. Fig. 12 shows results for each factor. For ‘Efficiency’Responsiveness is a factor to measure the response time and dimension, both of ‘Responsiveness’ and ‘Minimal Action’minimal action is about how simply users can deal with given got the high scores (2.50 and 2.08 respectively). It means thattasks. Accuracy factor is about user’s mistakes while using the the user can use the universal remote control effectively. Thecontrol and has error prevention and error recovery. Error subjects said that the haptic dial knob is very intuitiveprevention is about how well the remote control prevent user’s interface to control TV channel and temperature. The touchmistake and error recovery is about how easily users can screen also provides easy-to-use interface to select menus.correct the mistakes. For ‘Accuracy’ factor, both of ‘Error prevention’ and ‘Error Similarly, the concomitant domain has meaningfulness and recovery’ got the medium scores. ‘Error prevention’ factor gotconsistency factors. Meaningfulness is a factor to measure medium score (1.39). The score for ‘Error prevention’ ofhow well the remote control provides information and General (main screen) was not high (0.86) because if it is notfunctions which users want to use. Meaningfulness has correct position on touch screen, the system operates anotherunderstandability that is about how well users can understand program. In the case of ‘Error Recovery’ factor, temperaturesystem status via given information. Consistency is to measure control got high score (2.27). As the results of interview,how similar a function of the remote control is to other subjects recognized it is useful that friction is increased asconventional ones. Consistency has predictability and deviate degree from proper temperature.familiarity as detailed factors. Predictability is about how wellusers can predict instructions of the remote control throughexperience of using other similar ones. Familiarity is abouthow easily users can know instructions of a system throughexperience of real world. B. Method Subjects are 20-30’s thirty people who are familiar with theconventional remote controls. Before starting the survey, wegave information about the universal remote control and allowsubjects to use it freely for 5 minutes. Subjects did not knowthe task scenario and are asked to do tasks one by one usingthe universal remote control. These tasks are below: Fig. 12 The results of user-study 1. Temperature control: You came back home and it is too cold, - 5 °C outside. You are sitting down a sofa and control indoor For ‘Meaningfulness’ factor, ‘Understandability’ got high temperature to 28°C. score (2.09) and especially for the general section got the 2. TV Channel control: Now, you want to watch TV. The default channel is number 2 and it is boring to you so you change the highest score (2.68) because touch screen display shows TV channel to your favorite channel, number 11. various information according to context such as current 3. Temperature control: you feel it is too warm. You control temperature and channel information. indoor temperature to proper temperature in winter season, In case of ‘Consistency’ factor, ‘Predictability’ got high 24°C. score (2.42). It means the universal remote control has very
918 IEEE Transactions on Consumer Electronics, Vol. 56, No. 2, May 2010simple and stable interface. However ‘Familiarity’ has very  Mircea Badescu, Charles Wampler and Constantinos Mavroidis, "Rotary Haptic Knob for Vehicular Instrument Controls", Haptics02,low score (-0.32) because our universal remote control has a Haptic Interfaces For Virtual Envir & Teleoperator Systs, pp. 342-343,dial knob with haptic feedback and a touch screen unlike 2002conventional remote controls.  Laehyun Kim, Manchul Han, Sang Kyun Shin, Se Hyung Park, "A Haptic Dial System for Multimodal Prototyping", 18th International Conference on Artificial Reality and Telexistence (ICAT 2008), 2008 VIII. CONCLUSION  Yuichiro Sekiguchi, Koichi Hirota and Michitaka Hirose, “The Design Today people need to keep several remote controls to and Implementation of Ubiquitous Haptic Device”, Eurohaptics Conference, 2005 and Symposium on Haptic Interfaces for Virtualcontrol home appliances in their living room. Universal Environment and Teleoperator Systems, 2005. World Haptics 2005.remote control allows people to control multiple devices using First Joint , pp. 527 – 528, 2005a single remote control. However conventional universal  Wanjoo Park, Laehyun Kim, Hyunchul Cho and Sehung Park, “Designremote controls still have problems. Many buttons make the of Haptic Interface for Brickout Game”, IEEE International Workshop on Haptic Audio Visual Environments and Games (HAVE 2009), pp. 64-user spend time to browse and find the right one. In many case, 68, 2009people need to press a button several times, for instance  N. Melder and W. S. Harwin, “Extending the Friction Cone Algorithmvolume control and TV channel selection. In addition, the user for Arbitrary Polygon Based Haptic Objects”, Proceedings of the 12th International Symposium on Haptic Interfaces for Virtual Environmentneeds to read and remember the manual before he/she use it. and Teleoperator Systems (HAPTICS’04), pp 234 – 241, 2004 In this paper, we introduce a new universal remote control  Jinwoo Kim, et al, Introduce to Human Computer Interaction, Ahnto address these limitations. It has a simple interface which Graphics Press: 2005, pp. 195-235consists of a touch screen, two buttons, and a dial knob withhaptic feedback. Touch screen can display various information BIOGRAPHIESand be used an input device to select home device the userwant to control. Two buttons is used for the power on/off and Laehyun Kim (M’09) is a senior research scientist inreturning to the home menu. Haptic dial knob provides a very Intelligent Systems Technology Division at Korea Institute of Science and Technology. His research interests includeintuitive input and output interface. The user can change the haptics, computer graphics, and virtual reality. Kimstatus by rotating the dial knob and at the same time, he/she Laehyun received his BS from Hanyang University inreceives tactile feedbacks depending on the situation. Material Engineering, an MS from Yonsei University in Our universal remote control allows the user to control TV Computer Science, and a PhD in Computer Science from the University of Southern California.channel, room temperature, and room light brightness. Inaddition, we applied the device to a game controller for a Wanjoo Park received his B.S. in electronics engineeringbrickout game. It makes the game more fun and more from the Korea Aerospace University, Korea, in 2006, and his M.S. in electronics engineering from Sogang University,immersive. The result of a user study to evaluate the usability Korea, in 2008. Since 2008, he has been a researcher withshows that our universal remote control is very efficient and the Intelligence & Interaction Research Center of the Koreaintuitive device to control home appliances. Institute of Science and Technology. His research interests include machine learning, pattern recognition, haptics, and In the future we will design new haptic effects to control HCI.other devices and make the universal remote control smallerand lighter for better usability. Hyunchul Cho is a research scientist in the Intelligence and Interaction Research Center at Korea Institute of Science and Technology. He received his B.S. in Computer Science REFERENCES and Engineering from Korea University, Seoul, Korea in Karon E. MacLean, "Designing with Haptic Feedback" Symposium on 2003, and an M.S. in Computer Science and Engineering Haptic Feedback in the Proc. of IEEE Robotics and Automation (ICRA from Korea University, Seoul, Korea in 2005. 2000), pp.22-28, 2000 Vincent Hayward and Karon E. Maclean, “Do It Yourself Haptics: Part 1”, IEEE Robotics & Automation Magazine December 2007, pp. 88-104, 2007 Sehyung Park is a principal researcher in Intelligent Karon E. MacLean, “Foundations of Transparency in Tactile Systems Technology Division at Korea Institute of Science Information Design”, IEEE Transaction on Haptics, Vol. 1, No. 2, pp. Technology. His research interests include the study of 84-95, 2008 geometric modeling, the human-computer interface, reverse engineering, and NC programming. He received his BS El Saddik A., et al., “A Novel Biometric System for Identification and from Seoul National University in Mechanical Design and Verification of Haptic Users”, IEEE Transactions On Instrumentation and Measurement, Vol. 56, No. 3, pp.895-906 Production Engineering, an MS from Cornell Unversity in Mechanical Engineering, and a PhD in Mechnical Scott S. Snibbe, et al., “Haptic Techniques for Media Control”, Engineering from Korea Advanced Institute of Science and Technology. Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (UIST 2001) , pp.199-208, 2001