Kohta Kiyokawa ,  Shinya Yamamoto †,  Naoki Shibata ‡,  Keiichi Yasumoto and  Minoru Ito         Nara Institute of Science...
<ul><li>Propose framework  “ UbiREMOTE ” for remotely controlling appliances with 3D virtual space I/F </li></ul><ul><ul><...
Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul...
Background <ul><li>Several frameworks for controlling appliances through network are available </li></ul><ul><ul><li>UPnP,...
<ul><li>Several existing services </li></ul><ul><ul><li>E.g., “JIG multi remocon” by jig.jp </li></ul></ul><ul><ul><li>Ope...
Our goals <ul><li>Remote controller service  that allows users to remotely control networked appliances in their home by <...
Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul...
Challenges and Approaches Smooth operation on mobile terminals Intuitive monitoring of target spaces Intuitive operation
<ul><ul><li>Reproduce building with many networked appliances (target space) in 3D virtual space </li></ul></ul><ul><ul><u...
<ul><ul><li>Reproduce building with many networked appliances (target space) in 3D virtual space </li></ul></ul>Challenges...
<ul><ul><li>Reproduce building with many networked appliances (target space) in 3D virtual space </li></ul></ul>Challenges...
Intuitive UI based on 3D virtual space Appliance (lamp) in real  space is turned on Monitor real space conditions Move vie...
Target environment Home server Remote controller terminal Sensors Control <ul><li>Reflect information in virtual space </l...
Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul...
Why we need LW 3D rendering? <ul><li>Motivation  </li></ul><ul><ul><li>Render 3D objects on mobile terminals </li></ul></u...
Straightforward approach <ul><li>Thin Client Method </li></ul><ul><ul><li>A powerful server renders 3D image, captures a s...
Tradeoff for 3D rendering short Client renders all objects Thin Client Method Response time 3D rendering heavy light long ...
<ul><li>Surround viewpoint with a  cube ,   where 6 still images are captured for virtual space and put on its faces </li>...
Updating cube when viewpoint moves <ul><li>Set a sphere of radius  R  with the same center as cube </li></ul><ul><li>While...
Message sequence for cube update viewpoint movement Viewpoint movement Server Client New center pos for cube BG images for...
Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul...
<ul><li>Difference between virtual and real spaces causes the  following situations </li></ul><ul><ul><ul><li>A user remot...
Reflecting state change in real space into virtual space Home server Wireless communication Communication Virtual space li...
Detection/reflection of appliance positions <ul><li>Attach to each appliance a sensor node equipped with </li></ul><ul><ul...
Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul...
Experimental evaluation <ul><li>Implemented UbiREMOTE prototype on tablet PC  </li></ul><ul><li>Evaluated performance </li...
Exp1 : Rendering speed will achieve practical rendering speed on mobile terminal Environment : Tablet PC <ul><ul><li>OS:  ...
Exp2 : Response time (system configuration) Mobile Terminal ( Tablet PC ) Home Server UPnP TCP/IP 3G cellular network / Wi...
Exp2 : Response time (Result) <ul><li>Reasonable response time for practical use  </li></ul>UbiREMOTE appliances (a)WLAN (...
Exp3:Usability evaluation by questionnaire <ul><li>6 testees (graduate students) </li></ul><ul><ul><li>operated 4 applianc...
<ul><li>It is easy to monitor appliances/conditions with UbiREMOTE </li></ul>Exp3 : Result of usability evaluation by ques...
<ul><li>Q3 : Did you feel something wrong in UbiREMOTE? </li></ul>Exp3 : Result of usability evaluation by questionnaire(2...
Video Demonstration
Conclusion and Future work <ul><li>Conclusion </li></ul><ul><ul><li>UbiREMOTE framework </li></ul></ul><ul><ul><ul><li>Int...
(Slides) UbiREMOTE: Framework for Remotely Controlling Networked Appliances through Interaction with 3D Virtual Space
Upcoming SlideShare
Loading in...5
×

(Slides) UbiREMOTE: Framework for Remotely Controlling Networked Appliances through Interaction with 3D Virtual Space

933

Published on

Kiyokawa, K., Yamamoto, S., Shibata, N., Yasumoto, K., Ito, M.: UbiREMOTE: Framework for Remotely Controlling Networked Appliances through Interaction with 3D Virtual Space, Proc. of ACM Multimedia Systems 2010 (MMSys2010), pp.271-280, DOI:10.1145/1730836.1730870 (Feb. 2010).

http://ito-lab.naist.jp/mediawiki/images/6/60/100223mmsys.pdf

In this paper, we propose a framework named “UbiREMOTE”for controlling information appliances connected to a home network with a unified and intuitive user interface from a remote place. The UbiREMOTE framework provides users with a way to control appliances in a home through a virtual space drawn on a mobile terminal screen which reflects the latest conditions of the real appliances and the rooms in the home. With UbiREMOTE, a user controls appliances by (1) moving to the front of an appliance, (2) choosing the appliance to control and (3) pushing buttons on the virtual remote controller which imitates the real remote controller for the appliance or the real console. In this paper, we propose a method to improve the drawing speed of 3D virtual space on mobile terminals and a method for automatically reflecting condition changes of the real space in the virtual space. We implemented the methods and evaluated the performance. The results showed that the proposed methods can be practically used on small mobile terminals.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
933
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • In the first Experiment, we used tablet PC with this specification. We compared the proposed method with the method that renders all objects. These are the number of objects inside the cube. This is the measured frame rates. Our method has 250 times better frame rates. From this result, we believe that the proposed technique will achieve Practical rendering speed on mobile terminal.
  • In the Experiment 2 , we measured response time. We connected tablet pc to home server through 3G network or wifi. The home Server is also connected to these three appliances. However, these appliances are not UPnP ready, so we used infrared remote controller Between the server and each appliance like this figure.
  • This is the result of experiment 2. When we use wireless LAN, average the response time was less than 1second. On the other hand, when we use cellular network, the response time was about 3 second. We believe that this is reasonable response time for practical use.
  • In the 3 rd experiment, we conducted questionnaire for 6 testees of graduate students. Each testee [operated 4 appliances by the Remote controller with text-based interface like this figure and UbiREMOTE] We compared text-based controller with UbiREMOTE.
  • This is the result. For the question 1, [ Is operation comfortable? ] The ratio of good or very good answers was improved from 71% to 96%. For the question 2, [ Can monitor conditions of appliances? ] , The ratio was improved to a great extent, from 21% to 100%. These results suggest that [ It is easy to monitor appliances/conditions with UbiREMOTE ].
  • The third question “Did you feel something wrong in UbiREMOTE?” is about the look-and-feel of cubic virtual space of UbiREMOTE. As a result, no testee answered “feel strange”. However, there were the following opinions. So, we confirmed that the proposed LW 3D rendering technique Does not spoil user feeling and comfort in operating appliances.
  • (Slides) UbiREMOTE: Framework for Remotely Controlling Networked Appliances through Interaction with 3D Virtual Space

    1. 1. Kohta Kiyokawa , Shinya Yamamoto †, Naoki Shibata ‡, Keiichi Yasumoto and Minoru Ito        Nara Institute of Science and Technology , † Tokyo University of Science Yamaguchi, ‡ Shiga University   February 23, MMSys2010
    2. 2. <ul><li>Propose framework “ UbiREMOTE ” for remotely controlling appliances with 3D virtual space I/F </li></ul><ul><ul><li>Intuitive operation for anyone </li></ul></ul><ul><ul><li>Intuitive monitoring of target spaces where appliances exist </li></ul></ul><ul><ul><li>Smooth operation on mobile terminals </li></ul></ul><ul><li>Implement and evaluate the framework </li></ul><ul><ul><li>Runs smoothly on mobile terminals </li></ul></ul><ul><ul><li>Has better usability than text-based remote controller </li></ul></ul>Overview
    3. 3. Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions and Future Works </li></ul>
    4. 4. Background <ul><li>Several frameworks for controlling appliances through network are available </li></ul><ul><ul><li>UPnP, DLNA, OSGi, ECHONET, etc </li></ul></ul><ul><li>These frameworks enable remote control of appliances </li></ul><ul><ul><li>Examples </li></ul></ul><ul><ul><ul><li>warm/cool the living room before arriving home </li></ul></ul></ul><ul><ul><ul><li>check if he/she switched off an appliance (e.g., heater) and turn it off remotely </li></ul></ul></ul>
    5. 5. <ul><li>Several existing services </li></ul><ul><ul><li>E.g., “JIG multi remocon” by jig.jp </li></ul></ul><ul><ul><li>Operate multiple appliances through a cell phone </li></ul></ul><ul><ul><li>User I/F is based on figures/icons and texts </li></ul></ul><ul><li>Drawbacks </li></ul><ul><ul><li>Usability rapidly reduces when operating many similar kind of appliances at different places (rooms) </li></ul></ul><ul><ul><ul><li>Difficult to find target appliance from many figures/texts on screen </li></ul></ul></ul><ul><ul><ul><li>UI is different from dedicated remote controllers </li></ul></ul></ul>Existing services and drawbacks © jig.jp co.ltd http://remocon.jig.jp/
    6. 6. Our goals <ul><li>Remote controller service that allows users to remotely control networked appliances in their home by </li></ul><ul><ul><li>Intuitive operation from a remote place </li></ul></ul><ul><ul><li>Intuitive monitoring of target spaces </li></ul></ul><ul><ul><ul><li>Current states of appliances (on/off, …etc) </li></ul></ul></ul><ul><ul><ul><li>Conditions (temperature, humidity, …etc) of rooms </li></ul></ul></ul><ul><ul><li>Smooth operation by mobile terminals </li></ul></ul>
    7. 7. Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions and Future Works </li></ul>
    8. 8. Challenges and Approaches Smooth operation on mobile terminals Intuitive monitoring of target spaces Intuitive operation
    9. 9. <ul><ul><li>Reproduce building with many networked appliances (target space) in 3D virtual space </li></ul></ul><ul><ul><ul><li>Imitate the real remote controller for each appliance by using control window </li></ul></ul></ul><ul><ul><ul><li>Show states of appliances by visual effects </li></ul></ul></ul>Challenges and Approaches Intuitive operation
    10. 10. <ul><ul><li>Reproduce building with many networked appliances (target space) in 3D virtual space </li></ul></ul>Challenges and Approaches <ul><ul><li>There are too many 3D objects in building </li></ul></ul><ul><ul><li>Poor 3D capability on mobile device </li></ul></ul><ul><ul><li>We introduce lightweight 3D rendering mechanism </li></ul></ul>Smooth operation on mobile terminals Intuitive operation
    11. 11. <ul><ul><li>Reproduce building with many networked appliances (target space) in 3D virtual space </li></ul></ul>Challenges and Approaches <ul><ul><li>Gather data by sensors deployed in each room </li></ul></ul><ul><ul><li>Detection/reflection of appliance positions </li></ul></ul><ul><ul><li>Lightweight 3D rendering mechanism </li></ul></ul>Smooth operation on mobile terminals Intuitive monitoring of target spaces Intuitive operation
    12. 12. Intuitive UI based on 3D virtual space Appliance (lamp) in real space is turned on Monitor real space conditions Move viewpoint 25.0℃ Select target appliance (lamp), and turn it on
    13. 13. Target environment Home server Remote controller terminal Sensors Control <ul><li>Reflect information in virtual space </li></ul><ul><li>Allow intuitive operations for appliances </li></ul>Appliances User Wireless communication 3D Virtual Space (UI) Operation Synchronization Target Real Space Communication
    14. 14. Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul><ul><ul><li>Lightweight (LW) 3D Rendering </li></ul></ul><ul><ul><li>Reflecting Real Space Info into Virtual Space </li></ul></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions and Future Works </li></ul>
    15. 15. Why we need LW 3D rendering? <ul><li>Motivation </li></ul><ul><ul><li>Render 3D objects on mobile terminals </li></ul></ul><ul><ul><li>However, there are so many 3D objects in home, and mobile device’s 3D rendering capability is not sufficient </li></ul></ul><ul><li>Requirements </li></ul><ul><ul><li>Smooth viewpoint movement </li></ul></ul><ul><ul><li>Quick response when operating an appliance </li></ul></ul><ul><ul><li>Sufficient quality of rendered image </li></ul></ul>
    16. 16. Straightforward approach <ul><li>Thin Client Method </li></ul><ul><ul><li>A powerful server renders 3D image, captures a still image from a specified viewpoint, and sends the image to a client </li></ul></ul><ul><li>Drawback </li></ul><ul><ul><li>Large operation delay (response time) is produced </li></ul></ul>4. Drawing image 2. Rendering and capturing still image Server Client 3. Sending captured image 1. Operation Some time lag produced
    17. 17. Tradeoff for 3D rendering short Client renders all objects Thin Client Method Response time 3D rendering heavy light long <ul><li>Our approach </li></ul><ul><ul><li>Propose compromise between two methods </li></ul></ul><ul><ul><li>Utilize a fact that users are likely to operate appliances near viewpoint </li></ul></ul>
    18. 18. <ul><li>Surround viewpoint with a cube , where 6 still images are captured for virtual space and put on its faces </li></ul><ul><li>Client renders only objects inside the cube </li></ul>Proposed LW 3D rendering technique These objects are drawn as background images (rendered by server) These objects inside the cube are rendered by client Viewpoint VP Cube
    19. 19. Updating cube when viewpoint moves <ul><li>Set a sphere of radius R with the same center as cube </li></ul><ul><li>While viewpoint (VP) stays within the sphere </li></ul><ul><ul><li> No update of cube </li></ul></ul><ul><li>When VP goes out of sphere </li></ul><ul><ul><li>Prefetch BG images for </li></ul></ul><ul><ul><li>new cube, and replace </li></ul></ul><ul><ul><li>current cube by new one </li></ul></ul>R BG image
    20. 20. Message sequence for cube update viewpoint movement Viewpoint movement Server Client New center pos for cube BG images for new cube User going out of sphere Rendering BG images Updating cube Updating cube User going out of sphere RendereingBG images
    21. 21. Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul><ul><ul><li>Lightweight (LW) 3D Rendering </li></ul></ul><ul><ul><li>Reflecting Real Space Info into Virtual Space </li></ul></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions and Future Works </li></ul>
    22. 22. <ul><li>Difference between virtual and real spaces causes the following situations </li></ul><ul><ul><ul><li>A user remotely operated a heater in living room (on the screen), but it actually had been moved to children room </li></ul></ul></ul><ul><li>Approach to avoid above situations </li></ul><ul><ul><li>Show some real space information in virtual space </li></ul></ul><ul><ul><ul><li>Temperature, humidity, etc </li></ul></ul></ul><ul><ul><ul><li>Position of appliances / furniture </li></ul></ul></ul>Why need to reflect real space info?
    23. 23. Reflecting state change in real space into virtual space Home server Wireless communication Communication Virtual space living room Temp:25C Hum:60 % Sensors at each room detect temperature change, movement of appliance/ furniture, etc Deploy sensors at each room, and update information when there is a big change in sensor values
    24. 24. Detection/reflection of appliance positions <ul><li>Attach to each appliance a sensor node equipped with </li></ul><ul><ul><li>Acceleration sensors for detecting movement </li></ul></ul><ul><ul><li>ZigBee communication capability for estimation of new position </li></ul></ul><ul><li>Use existing ZigBee based localization techniques to estimate new position of appliance </li></ul>
    25. 25. Outline <ul><li>Background and Goals </li></ul><ul><li>UbiREMOTE Framework </li></ul><ul><li>Proposed Techniques </li></ul><ul><li>Evaluation </li></ul><ul><li>Conclusions and Future Works </li></ul>
    26. 26. Experimental evaluation <ul><li>Implemented UbiREMOTE prototype on tablet PC </li></ul><ul><li>Evaluated performance </li></ul><ul><ul><li>Smoothness : Frame rates and response time </li></ul></ul><ul><ul><li>Usability : Questionnaire for User satisfaction in remote operation </li></ul></ul>TV Electric Fan Ceiling Light Stand Light CD Player Floor plan and position of viewpoint Controller of viewpoint Controller for an appliance
    27. 27. Exp1 : Rendering speed will achieve practical rendering speed on mobile terminal Environment : Tablet PC <ul><ul><li>OS: Windows XP tablet PC edition </li></ul></ul><ul><ul><li>CPU: Pentium M 1.2GHz </li></ul></ul><ul><ul><li>Memory: 512MB </li></ul></ul><ul><ul><li>Graphics Chip: Mobile Intel ®915GM </li></ul></ul># objects (# objects inside cube) # polygons Frame rate (fps) Render all objects 20 200000 1 Proposed 20 (1) 108 276 Proposed 20 (2) 2100 262 Proposed 20 (3) 4100 250
    28. 28. Exp2 : Response time (system configuration) Mobile Terminal ( Tablet PC ) Home Server UPnP TCP/IP 3G cellular network / WiFi Electric Fan Light Heater infrared Remote Controller
    29. 29. Exp2 : Response time (Result) <ul><li>Reasonable response time for practical use </li></ul>UbiREMOTE appliances (a)WLAN (b)Cellular Electric fan ON 0.94 3.22 OFF 0.92 3.14 Light ON 0.73 2.98 OFF 0.61 3.13 Heater ON 0.93 3.18 OFF 0.84 3.30 average 0.828 3.141
    30. 30. Exp3:Usability evaluation by questionnaire <ul><li>6 testees (graduate students) </li></ul><ul><ul><li>operated 4 appliances by Remote controller that has text-based interface and UbiREMOTE, </li></ul></ul><ul><ul><li>Return questionnaires. </li></ul></ul>Compared Text-based remote controller
    31. 31. <ul><li>It is easy to monitor appliances/conditions with UbiREMOTE </li></ul>Exp3 : Result of usability evaluation by questionnaire(1) サンプル数: 24 サンプル数: 24 ■ Very Good ■ Good ■ Not good ■ Bad Q1: Is operation comfortable? Q2:Can monitor conditions of appliances? Text-based 6 testees ×4 appliances = 24samples
    32. 32. <ul><li>Q3 : Did you feel something wrong in UbiREMOTE? </li></ul>Exp3 : Result of usability evaluation by questionnaire(2) Proposed LW 3D rendering does not spoil user feeling and comfort in operation
    33. 33. Video Demonstration
    34. 34. Conclusion and Future work <ul><li>Conclusion </li></ul><ul><ul><li>UbiREMOTE framework </li></ul></ul><ul><ul><ul><li>Intuitive and unified GUI for remotely controlling appliances </li></ul></ul></ul><ul><ul><li>Techniques to realize UbiREMOTE </li></ul></ul><ul><ul><ul><li>LW 3D rendering based on cubic virtual space </li></ul></ul></ul><ul><ul><ul><li>Automatic reflection of real space information </li></ul></ul></ul><ul><ul><li>Performance evaluation </li></ul></ul><ul><ul><ul><li>Achieved great improvement of 3D rendering performance </li></ul></ul></ul><ul><ul><ul><li>Usability was much better than Text-based interface </li></ul></ul></ul><ul><li>Future work </li></ul><ul><ul><li>Implementation on iPhone and Android </li></ul></ul><ul><ul><li>Usability evaluation by more detailed scenario </li></ul></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×