Kiyokawa, K., Yamamoto, S., Shibata, N., Yasumoto, K., Ito, M.: UbiREMOTE: Framework for Remotely Controlling Networked Appliances through Interaction with 3D Virtual Space, Proc. of ACM Multimedia Systems 2010 (MMSys2010), pp.271-280, DOI:10.1145/1730836.1730870 (Feb. 2010).
http://ito-lab.naist.jp/mediawiki/images/6/60/100223mmsys.pdf
In this paper, we propose a framework named “UbiREMOTE”for controlling information appliances connected to a home network with a unified and intuitive user interface from a remote place. The UbiREMOTE framework provides users with a way to control appliances in a home through a virtual space drawn on a mobile terminal screen which reflects the latest conditions of the real appliances and the rooms in the home. With UbiREMOTE, a user controls appliances by (1) moving to the front of an appliance, (2) choosing the appliance to control and (3) pushing buttons on the virtual remote controller which imitates the real remote controller for the appliance or the real console. In this paper, we propose a method to improve the drawing speed of 3D virtual space on mobile terminals and a method for automatically reflecting condition changes of the real space in the virtual space. We implemented the methods and evaluated the performance. The results showed that the proposed methods can be practically used on small mobile terminals.
(Slides) UbiREMOTE: Framework for Remotely Controlling Networked Appliances through Interaction with 3D Virtual Space
1. Kohta Kiyokawa , Shinya Yamamoto †, Naoki Shibata ‡, Keiichi Yasumoto and Minoru Ito Nara Institute of Science and Technology , † Tokyo University of Science Yamaguchi, ‡ Shiga University February 23, MMSys2010
2.
3.
4.
5.
6.
7.
8. Challenges and Approaches Smooth operation on mobile terminals Intuitive monitoring of target spaces Intuitive operation
9.
10.
11.
12. Intuitive UI based on 3D virtual space Appliance (lamp) in real space is turned on Monitor real space conditions Move viewpoint 25.0℃ Select target appliance (lamp), and turn it on
13.
14.
15.
16.
17.
18.
19.
20. Message sequence for cube update viewpoint movement Viewpoint movement Server Client New center pos for cube BG images for new cube User going out of sphere Rendering BG images Updating cube Updating cube User going out of sphere RendereingBG images
21.
22.
23. Reflecting state change in real space into virtual space Home server Wireless communication Communication Virtual space living room Temp:25C Hum:60 % Sensors at each room detect temperature change, movement of appliance/ furniture, etc Deploy sensors at each room, and update information when there is a big change in sensor values
24.
25.
26.
27.
28. Exp2 : Response time (system configuration) Mobile Terminal ( Tablet PC ) Home Server UPnP TCP/IP 3G cellular network / WiFi Electric Fan Light Heater infrared Remote Controller
In the first Experiment, we used tablet PC with this specification. We compared the proposed method with the method that renders all objects. These are the number of objects inside the cube. This is the measured frame rates. Our method has 250 times better frame rates. From this result, we believe that the proposed technique will achieve Practical rendering speed on mobile terminal.
In the Experiment 2 , we measured response time. We connected tablet pc to home server through 3G network or wifi. The home Server is also connected to these three appliances. However, these appliances are not UPnP ready, so we used infrared remote controller Between the server and each appliance like this figure.
This is the result of experiment 2. When we use wireless LAN, average the response time was less than 1second. On the other hand, when we use cellular network, the response time was about 3 second. We believe that this is reasonable response time for practical use.
In the 3 rd experiment, we conducted questionnaire for 6 testees of graduate students. Each testee [operated 4 appliances by the Remote controller with text-based interface like this figure and UbiREMOTE] We compared text-based controller with UbiREMOTE.
This is the result. For the question 1, [ Is operation comfortable? ] The ratio of good or very good answers was improved from 71% to 96%. For the question 2, [ Can monitor conditions of appliances? ] , The ratio was improved to a great extent, from 21% to 100%. These results suggest that [ It is easy to monitor appliances/conditions with UbiREMOTE ].
The third question “Did you feel something wrong in UbiREMOTE?” is about the look-and-feel of cubic virtual space of UbiREMOTE. As a result, no testee answered “feel strange”. However, there were the following opinions. So, we confirmed that the proposed LW 3D rendering technique Does not spoil user feeling and comfort in operating appliances.