Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PDR for lab-forming field and its benchmarking


Published on

PDR for lab-forming field and its benchmarking in ISMAR 2016 Workshop: Standards for Mixed and Augmented Reality

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

PDR for lab-forming field and its benchmarking

  1. 1. PDR for lab-forming field and its benchmarking Takeshi Kurata12, Ryosuke Ichikari1, Tatsuro Myokan12, Masakatsu Kourogi1, Katsuhiko Kaji3, and Nobuo Kawaguchi4 1AIST, 2Univ. of Tsukuba, 3Aichi institute of technology, 4Nagoya Univ. ISMAR 2016 Workshop: Standards for Mixed and Augmented Reality (2016/9/23)
  2. 2. Lab-forming Field & Field-forming Lab • Borrowing from “Terraforming” • Lab-forming Field: Transforming a real field into a lab-like place. • Field-forming Lab: Transforming a laboratory into a field-like place. 2
  3. 3. Big Data on Result/Behavior/Environment for Lab-forming Field 3
  4. 4. Case study in Japanese Restaurant “Ganko” • Objectives 1. (for AIST) to test the CSQCC (Computer-Supported QCC) suites in a real service field. 2. (for the restaurant) to observe effects of process improvement planned by CSQCC. • Place – Japanese cuisine restaurant GANKO Ginza 4-chome (Tokyo) • Term – 1st term • January 12 to 18, 2011 – 2nd term • February 3 to 9, 2011 4 Dining area Course dishes 1st term (Jan. 12-18, 2011) for observing ordinary operations QC circle for making improvement plans 2nd term (Feb. 3-9, 2011) for observing improved operations
  5. 5. 5 B2 B1 Dinning Area Kitchen Office room Pantry
  6. 6. During Discussion in CSQCC 6 Trajectory of a wait staff in lunch time: 12:00-14:00 Fact: Going in and out of the kitchen/office to no small extent. Possible result: Difficulty in concentrating on guest service. Cause: Cell phone everywhere, but reservation book only in the office room. Possible improvement: e-reservation book Dinning Area Kitchen Office room T. Fukuhara, R. Tenmoku, T. Okuma, R. Ueoka, M. Takehara, and T. Kurata, "Improving Service Processes based on Visualization of Human-behavior and POS data: a Case Study in a Japanese Restaurant“, ICServ2013, pp.1-8.
  7. 7. PDR(Pedestrian Dead-Reckoning) Estimates velocity vector, relative altitude, and action type by measurements from a wearable sensor module.  Wearing a sensor module on waist (2D SHS (Steps and Heading Systems) PDR)  Easy to wear and maintain  Easy to measure data for action recognition  Relatively easily apply for handheld setting compared to shoe-mounted PDR (3D-INS (Inertial Navigation System) PDR) 7 Handheld PDR From PDR to PDRplus 10-axis sensors • Accelerometers • Magnetic sensors • Gyro sensors • Barometer Shoe-mounted PDR Waist-worn PDR
  8. 8. PDR Business Launched in Japan Collaboration among NTT Docomo, ZDC, AKM, and AIST. A PDR driven Indoor navigation mobile app was released in Apr, 2015. (As of Sep, 2016, 440 areas including subways and underground shopping arcades all across Japan) Collaboration between MegaChips and AIST. Sensor Hub LSI “Frizz” was released in Jan, 2015 and MegaChips plans to sell 10 million Frizz products in 2015. (Power-aware motion coprocessor optimized for PDR) 8 • M. Kourogi and T. Kurata, Personal positioning based on walking locomotion analysis with self-contained sensors and a wearable camera, ISMAR2003, pp. 103–112, (2003).
  9. 9. AR by PDR + Image registration (1999-2003) Panorama-based Annotation: IWAR1999, ISWC2001, ISMAR2003 G Environmental map A B C D E A B C F Input frames Position at which a panorama is taken Position Direction 235 [deg] 5 [deg] From the user’s camera Located Orientated 9
  10. 10. Frontier of PDR: Walking direction estimation 10 • Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.
  11. 11. Frontier of PDR: Walking direction estimation 11 • Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015. • Long Paper: Christophe Combettes, Valerie Renaudin, Comparison of Misalignment Estimation Techniques Between Handheld Device and Walking Directions, IPIN 2015. • FIS was proposed by Kourogi and Kurata in PLANS 2014. “Globally, the FIS method provides better results than the other two methods.” Frequency analysis of Inertial Signals Forward and Lateral Acc. Modeling Principal Component Analysis
  12. 12. Overview: History of our PDR 12 ISWC2001 IWAR1999 ISMAR2003 PLANS2014 PLANS2010 ICServ2013 Docomo map navi Image registration + Gyro Panorama-based annotation (Image-registration-based positioning) Image registration + PDR PDRplus (PDR + Action recognition) Handheld PDR (Walking-direction estimation) 2015- 2015- PDR module 2011- Academia Industry Before PDR ICAT2006 PDR + GPS + RFID
  13. 13. In the year of 2010 • iPhone 4: the first popular consumer mobile device equipped with 9-axis sensors including accelerometers, magnetic sensors, and gyro sensors 13 G-spatial EXPO 2010: Handheld PDR on iPhone 4 (World’s first-ever live demo) PLANS2010, PLANS2014
  14. 14. Global Trend on PDR PDR R&D players have rapidly indicated their presence all over the world on and after 2010. Movea (France) Sensor Platforms (USA) CSR (UK) TRX (USA) Trusted Positioning (Canada) 14 Acquired by QualcommAcquired by InvenSenseAcquired by InvenSense Acquired by Audience (USA) SFO
  15. 15. Standardization on PDR Benchmarking • PDR related R&D is highly active worldwide: Necessity for sharing common measures. • Description of the performance should be unified in spec sheets and scientific papers. • Different measures from absolute positioning methods such as GNSS, Wi-Fi, and BLE are required for PDR, which is a method of relative positioning. • PDR Benchmark Standardization Committee was established in 2014 as a platform of the grassroots activity. 15
  16. 16. Support Organizations • Asahi Kasei Corporation, Asia Air Survey Co., Ltd. (Y. Minami), INTEC Inc., MTI Ltd., KDDI R&D Laboratories, Inc., KOKUSAI KOGYO CO., LTD., SHIBUYA KOGYO CO., LTD., Koozyt, Inc., SITESENSING, inc., Sharp Corporation, Sugihara Software and Electron Industry Co., Ltd. (SSEI), Information Services International-Dentsu, Ltd. (ISID), Hitachi, Ltd., Frameworx, Inc. (S. Watanabe), MULTISOUP CO.,LTD., Milldea, LLC, Murata Manufacturing Co., Ltd., MegaChips Corporation, Recruit Lifestyle Co., Ltd. (K. Ushida), RICOH COMPANY, LTD., Rei-Frontier Inc., • Aichi Institute of Technology (K. Kaji), Akashi National College of Technology (I. Arai), Kanagawa Institute of Technology (H. Tanaka), Keio University (S. Haruyama, N. Kohtake, M. Nakajima), University of Tsukuba (T. Kurata), Tokyo Institute of Technology (S. Okada), Nagoya University (N. Kawaguchi), Niigata University (H. Makino), Ritsumeikan University (N. Nishio), National Institute of Advanced Industrial Science and Technology (AIST) (T. Kurata, M. Kourogi), Human Activity Sensing Consortium (HASC), Location Information Service Research Agency (LISRA) • 33 organizations in Japan as of April, 2016 16
  17. 17. Activity Examples in Japan 17 1st meeting of the committee (2014/05/22) 1st seminar in LBJ (2014/06/11) 2nd meeting of the committee (2015/05/19) 2nd seminar in LBJ (2015/06/13) Organized session in HCG symposium (2014/12/17) 3rd seminar in LBJ (2016/06/10) LBJ: Location Business Japan HCG: Human Communication Group Organized session in HCG symposium (2015/12/16) 3rd meeting of the committee (2015/05/19)
  18. 18. Scene in data collection 19
  19. 19. Multi-Algorithm On-Site Evaluation System • Evaluates the accuracy of each PDR algorithm automatically as often as sensor data is uploaded to the server • Provides trajectory images so that participants can compare their PDR • algorithms in real time. 20 K. Kaji, K. Kanagu, K. Murao, N. Nishio, K. Urano, H. Iida, N. Kawaguch, Multi-Algorithm On- Site Evaluation System for PDR Challenge, ICMU2016, (to appear)
  20. 20. UbiComp/ISWC 2015 PDR Challenge Corpus • Is now open to the public. ( 21 Routes 5 Devices 7 Subjects 93 # of pedestrian sensing data 241 # of pedestrian sensing data with calibration data 230 # of pedestrian sensing data with LIDAR data 10 Avg. of walking time [sec] 101 Avg. of moving distance [m] 115 Avg. of angular change [°] 606 K. Kaji, M. Abe, W. Wang, K. Hiroi, and N. Kawaguchi, UbiComp/ISWC 2015 PDR challenge corpus, HASCA2016 (UbiComp2016 Proceedings: Adjunct), pp.696-704 Statistics of the corpus Detailed route statistics of pedestrian sensing data with calibration data
  21. 21. Open Data Contest in Logistics & PDR Challenge in Warehouse • Open data contest in logistics by Frameworx – Submission: 2016/4/18-2016/7/18 – Award ceremony: 2016/9/12 • PDR Challenge in Warehouse – Now planning – Will be held as a Japanese domestic contest in 2016 – Will be held as an international contest in IPIN 2017 22
  22. 22. PDR Challenge Series • Ubicomp/ISWC 2015 PDR Challenge – Scenario: Indoor Navigation – On-site – Continuous walking while keeping watching the navigation screen by holding the smartphone – Several minutes per trial • 2016/2017 PDR Challenge in Warehouse (tentative) – Scenario: Picking work in a warehouse – Off-site – Not only walking but various actions including picking and carrying – Several hours per trial – Will be held in IPIN 2017 23
  23. 23. Examples of picking workers’ trajectories estimated by PDR + WMS (Warehouse Management System) 24
  24. 24. How to design benchmark Indicators? • Other aspects to be considered – Reliability: Different measures from absolute positioning methods are required for PDR – Efficiency: Power consumption – Repeatability: Temperature Hysteresis, Magnetic field, etc. – Representativeness: How to hold, Route shape, etc. 25 Benchmark indicators of vision-based spatial registration and tracking for MAR (ISO/IEC WD 18520)
  25. 25. How to compare and visualize? 26 Easy Difficult Method 1 Easy Difficult Method 2
  26. 26. How to compare and visualize? 27 Easy Difficult Method1Method2
  27. 27. Competitions: IPIN and the others (cf. EvAAL presentation in IPIN 2105 etc.) 28 IPIN year EvAAL, IPSN, UbiComp/ISWC Zurich, Switzerland 2010 universAAL is launched Guimaraes, Portugal 2011 EvAAL: indoor localization Sidney, Australia 2012 EvAAL: + activity recognition Montbeliard, France 2013 EvAAL: same as 2012 Busan, Korea 1st IPIN competition 2014 EvAAL: 3 floors, smartphone IPSN: infrastruc. based + free Banff, Canada EvAAL-ETRI comp. 2015 EvAAL-ETRI: 6 floors, on/off-site IPSN: infrastruc. based + free UbiComp/ISWC: 2 floors, smartphone PDR, 90 subjects Madrid, Spain Indoor Localization Competition 2016 IPIN: smartphone (on/off-site), PDR, Robot IPSN: infrastruc. based + free, 2D/3D
  28. 28. IPIN2017 29 ISMAR2015
  29. 29. Thank you! • AIST is now hiring for Tenure-track, Postdoc, and RA (PhD) positions at Tsukuba, Japan. • Target research fields are ↓ ↓ ↓ 30