Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lab-Forming Fields and Field-Forming Labs

present the concept of Lab-Forming Fields (LFF) and Field-Forming Labs (FFL). LFF is to transform real service fields into lab-like places for bringing research methodologies in laboratories to real fields with IoT. FFL is to transform laboratories into real-field-like places for getting subjects’ behavior and experimental results closer and closer to the ones which are supposed to be obtained in the real service fields with VR. Next, I introduce indoor positioning technologies such as PDR (Pedestrian Dead Reckoning) as a key technology for human behavior sensing. Then I conclude this talk by briefly reporting on case studies of service kaizen in a restaurant and a warehouse respectively.

  • Be the first to comment

  • Be the first to like this

Lab-Forming Fields and Field-Forming Labs

  1. 1. 国立研究開発法人 Lab-Forming Fields (LFF) and Field-Forming Labs (FFL) Takeshi Kurata1, 2 1 Human Informatics Research Institute, AIST, Japan 2University of Tsukuba, Japan E-mail: 1
  2. 2. Takeshi Kurata, Ph.D. • Position:  – Research Group Leader, Service Sensing, Assimilation, and Modeling  Research Group, Human Informatics Research Institute, AIST – Professor (Cooperative Graduate School Program), Faculty of  Engineering, Information and Systems, University of Tsukuba • Professional Experience: – 2011‐2014 Doctoral co‐supervisor, Joseph Fourier University, UJF‐ Grenoble 1, France – 2012‐ ISO/IEC JTC 1/SC 24 Member – 2003‐2005 Visiting Scholar, HIT Lab, University of Washington • Education: – 2007 Ph.D. (Eng.) from Doctoral Program in Graduate School of  Systems and Information Engineering, University of Tsukuba – 1996 M.E. from Doctoral Program in Engineering, University of Tsukuba • Research Interests: – Service Research, Assistive technology, Wearable/Pervasive Computing,  Mixed and Augmented Reality, Computer Vision 2
  3. 3. AIST President Dr. Ryoji Chubachi AIST, Tsukuba 1 h drive from Tokyo National Institute of Advanced Industrial Science and Technology • One of the largest national institute in Japan – The independent agency of the Ministry of Economy, Trade and Industry – The mission of AIST is advanced research and development for industry – Over 2,300 permanent researchers – Over 50 research units cover various research fields 3
  4. 4. Research Fields and Staffs of AIST 4
  5. 5. Human Informatics Research Inst. • History – Established in April, 2015 – Main department is located in Tsukuba • Organization – 85 permanent researchers – 10 research groups • Brain science • Human factors engineering • Digital human modeling • Service engineering 5
  6. 6. Framework of Human Informatics Human and Society Service Presentation Analysis Understanding Sensing Participation Social cognition Healthcare Wellness Safety Comfort Deep Data (High Quality Reference Data) Big Data IT IoT/Cyber-Physical 6
  7. 7. 国立研究開発法人 Comb Data: Big + Deep in LFF & FFL 7 SFS Dollhouse VR CCE Lite Wearable RGB-D sensing PDR Handheld AR
  8. 8. Result/Behavior/Environment and LFF/FFL 8
  9. 9. 国立研究開発法人 Lab-Forming Fields & Field-Forming Labs • Borrowing from “Terraforming” • Lab-forming Field: Transforming a real field into a lab-like place. (IoT/G-IoT) • Field-forming Lab: Transforming a laboratory into a field-like place. (VR) 9
  10. 10. 国立研究開発法人 Service design loop 10
  11. 11. 国立研究開発法人 11 測って 図る Hakatte Hakaru Measure Weigh Survey Plan Design Attempt
  12. 12. 国立研究開発法人 測って図る Hakatte Hakaru 12
  13. 13. ASPR Technologies for Multi-Stakeholders 13
  14. 14. 国立研究開発法人 Efficient interactive label attaching for supervised Service Operation Estimation 14
  15. 15. So many kinds of positioning methods 15
  16. 16. PDR(Pedestrian Dead-Reckoning) Estimates velocity vector, relative altitude, and action type by measurements from a wearable sensor module.  Wearing a sensor module on waist (2D SHS (Steps and Heading Systems) PDR)  Easy to wear and maintain  Easy to measure data for action recognition  Relatively easily apply for handheld setting compared to shoe-mounted PDR (3D-INS (Inertial Navigation System) PDR) 16 Handheld PDR From PDR to PDRplus 10-axis sensors • Accelerometers • Magnetic sensors • Gyro sensors • Barometer Shoe-mounted PDR Waist-worn PDR
  17. 17. AR by PDR + Image registration (1999-2003) Panorama-based Annotation: IWAR1999, ISWC2001, ISMAR2003 G Environmental map A B C D E A B C F Input frames Position at which a panorama is taken Position Direction 235 [deg] 5 [deg] From the user’s camera Located Orientated 17
  18. 18. Frontier of PDR: Walking direction estimation 18 • Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015.
  19. 19. Frontier of PDR: Walking direction estimation 19 • Tutorial: Personal Navigation with Handheld Devices by Valerie Renaudin, IPIN 2015. • Long Paper: Christophe Combettes, Valerie Renaudin, Comparison of Misalignment Estimation Techniques Between Handheld Device and Walking Directions, IPIN 2015. • FIS was proposed by Kourogi and Kurata in PLANS 2014. “Globally, the FIS method provides better results than the other two methods.” Frequency analysis of Inertial Signals Forward and Lateral Acc. Modeling Principal Component Analysis
  20. 20. Overview: History of our PDR 20 ISWC2001 IWAR1999 ISMAR2003 PLANS2014 PLANS2010 ICServ2013 Docomo map navi (500 areas as of March, 2017)) Image registration + Gyro Panorama-based annotation (Image-registration-based positioning) Image registration + PDR PDRplus (PDR + Action recognition) Handheld PDR (Walking-direction estimation) 2015- 2015- PDR module 2011- Academia Industry Before PDR ICAT2006 PDR + GPS + RFID
  21. 21. Global Trend on PDR PDR R&D players have rapidly indicated their presence all over the world on and after 2010. Movea (France) Sensor Platforms (USA) CSR (UK) TRX (USA) Trusted Positioning (Canada) 21 Acquired by QualcommAcquired by InvenSenseAcquired by InvenSense Acquired by Audience (USA) SFO
  22. 22. Standardization on PDR Benchmarking • PDR related R&D is highly active worldwide: Necessity for sharing common measures. • Description of the performance should be unified in spec sheets and scientific papers. • Different measures from absolute positioning methods such as GNSS, Wi-Fi, and BLE are required for PDR, which is a method of relative positioning. • PDR Benchmark Standardization Committee was established in 2014 as a platform of the grassroots activity. 22
  23. 23. Support Organizations • Asahi Kasei Corporation, Asia Air Survey Co., Ltd. (Y. Minami), INTEC Inc., MTI Ltd., KDDI R&D Laboratories, Inc., KOKUSAI KOGYO CO., LTD., SHIBUYA KOGYO CO., LTD., Koozyt, Inc., GOV Co., Ltd., SITESENSING, inc., Sharp Corporation, Sugihara Software and Electron Industry Co., Ltd. (SSEI), ZENRIN DataCom CO., LTD., Information Services International- Dentsu, Ltd. (ISID), Hitachi, Ltd., IBM Japan, Ltd., Frameworx, Inc. (S. Watanabe), MULTISOUP CO.,LTD., Milldea, LLC, Murata Manufacturing Co., Ltd., MegaChips Corporation, Recruit Lifestyle Co., Ltd. (K. Ushida), RICOH COMPANY, LTD., Rei-Frontier Inc., • Aichi Institute of Technology (K. Kaji), NARA Institute of Science and Technology (NAIST) (I. Arai), Kanagawa Institute of Technology (H. Tanaka), Keio University (S. Haruyama, N. Kohtake, M. Nakajima), University of Tsukuba (T. Kurata), Tokyo Institute of Technology (S. Okada), Nagoya University (N. Kawaguchi), Niigata University (H. Makino), Ritsumeikan University (N. Nishio), National Institute of Advanced Industrial Science and Technology (AIST) (T. Kurata, M. Kourogi), Human Activity Sensing Consortium (HASC), Location Information Service Research Agency (LISRA) • 36 organizations in Japan as of March, 2017 23
  24. 24. Scene in data collection 25
  25. 25. PDR Challenge Series • Ubicomp/ISWC 2015 PDR Challenge – Scenario: Indoor Navigation – On-site – Continuous walking while keeping watching the navigation screen by holding the smartphone – Several minutes per trial • IPIN 2017 PDR Challenge in Warehouse Picking – Scenario: Picking work in a warehouse – Off-site – Not only walking but various actions including picking and carrying – Several hours per trial 26
  26. 26. IPIN2017 27 • User Requirements • Hybrid IMU Pedestrian Navigation & Foot Mounted Navigation • Human Motion Monitoring • High Sensitivity GNSS, Indoor GNSS, Pseudolites • RTK GNSS with handheld devices • Mitigating GNSS errors prior to moving indoors • Self-contained sensors • Signal Strength Based Methods, Fingerprinting • UWB (Ultra-wideband) • Passive & Active RFID • Optical Systems • Ultrasound Systems • TOF, TDOA based Localization • Localization, Algorithms for Wireless Sensor Networks • Frameworks for Hybrid Positioning • Industrial Metrology & Geodetic Systems, iGPS • Radar Systems • Mapping, SLAM • Indoor Spatial Data Model & Indoor Mobile Mapping • Novel uses of maps and 3D building models • Magnetic Localization • Innovative Systems • Location Privacy • Applications of Location Awareness & Context Detection • Health and Wellness ApplicationsRegular Papers Due: April 30, 2017
  27. 27. Integrated Positioning (SDF) 28
  28. 28. Sub-meter indoor positioning: Visible Light Communication (VLC) & PDR • Less density of infrastructure installation by SDF combining VLC and PDR • Reduction of initial/running cost of sensing by Replacement demand of lighting 29 Collaboration with Panasonic
  29. 29. RGBD (Depth) sensor & PDR • Error compensation of PDR with precise trajectories obtained from surveillance (RGBD) cameras • Coverage compensation of surveillance cameras with continuous measurement of PDR 30
  30. 30. 国立研究開発法人 VDR (Vehicle/Vibration-based DR) 31
  31. 31. 国立研究開発法人 Whole-body posture estimation and precise positioning 32 Many sensors for heterogeneous and more precise real-world capturing (position, orientation, posture, physical load, etc.) and deep-data gathering
  32. 32. CSQCC (Computer-supported QC Circle) 33 Staying-time rate at each dinning area per personSales at each dinning area per employee Visualization tool combining human-behavioral and accounting history Employee taking order while cleaning up the guest room Icons showing the number of customers at each table POS data log Service Characteristics 1. Intangible 2. Heterogeneous 3. Inseparable 4. Perishable Alleviate the issues due to IHIP
  33. 33. QCC in manufacturing industry Purpose: Productivity improvement Conventional QCC in service industry Purpose: Productivity improvement Subjective QCC in service industry Purpose: Improvement of CS/ES w/ reasonable ways to gather objective data in plants In 1980s, applying QCC for service industry w/o reasonable ways to gather objective data in service fields In 1990s, Service industry lost interest in QCC In 2000 QCC in the Service Industry in Japan 34 Computer-supported QCC (CSQCC) Purpose: Productivity improvement In 2010 CSQCC in the future Productivity improvement Improvement of CS/ES w/ reasonable ways to gather subjective data continuouslyw/ reasonable ways to gather objective data in service fields 1950~ Deming Award
  34. 34. 3rd CSQCC for newly open 35 Yamano Aiko-tei, Shinjuku: Mansion style restaurant (2014.12.23) 2014.10.18 (Sat) 2014.11.08 (Sat)
  35. 35. Case study in Japanese Restaurant “Ganko” • Objectives 1. (for AIST) to test the CSQCC (Computer-Supported QCC) suites in a real service field. 2. (for the restaurant) to observe effects of process improvement planned by CSQCC. • Place – Japanese cuisine restaurant GANKO Ginza 4-chome (Tokyo) • Term – 1st term • January 12 to 18, 2011 – 2nd term • February 3 to 9, 2011 36 Dining area Course dishes 1st term (Jan. 12-18, 2011) for observing ordinary operations QC circle for making improvement plans 2nd term (Feb. 3-9, 2011) for observing improved operations
  36. 36. 37 B2 B1 Dinning Area Kitchen Office room Pantry
  37. 37. During Discussion in CSQCC 38 Trajectory of a wait staff in lunch time: 12:00-14:00 Fact: Going in and out of the kitchen/office to no small extent. Possible result: Difficulty in concentrating on guest service. Cause: Cell phone everywhere, but reservation book only in the office room. Possible improvement: e-reservation book Dinning Area Kitchen Office room T. Fukuhara, R. Tenmoku, T. Okuma, R. Ueoka, M. Takehara, and T. Kurata, "Improving Service Processes based on Visualization of Human-behavior and POS data: a Case Study in a Japanese Restaurant“, ICServ2013, pp.1-8.
  38. 38. Summary of 1st CSQCC for Wait Staff 39 Grasp of actual condition Shorter stay in dinning area than the manager assumed Kaizen plan development (1) Re-composition of service processes (SP) (2) Thoroughly obeying each division’s roll, (3) Guts Direct effect Stay ratio in dinning area at dinner time: UP ↑ Spillover effect Number of additional orders at dinner time: UP ↑ Side effect (Trade-off) (1) Work load (walking distance): No difference → (2) Number of additional orders at 3pm: No difference → Stay ratio in dinning areas 30% 35% 40% 45% 50% 55% 11 12 13 14 15 16 17 18 19 20 21 22 Walking Distance [m] 1,000 1,500 2,000 2,500 11 12 13 14 15 16 17 18 19 20 21 22 Num. of additional orders per customer 0.0 0.4 0.8 1.2 11 12 13 14 15 16 17 18 19 20 21 22Hour Hour Hour Before After Down: Due to SP re- comp. for preparation of dinner/party UP: Much more than time decreased in Tea hour No diff.: Due to no SP re-comp. No diff.: Despite SP re-comp. for preparation of dinner/party UP: due to reduction of opportunity loss No diff. on workload Lunch Tea Dinner Lunch Tea Dinner Lunch Tea Dinner
  39. 39. Walk distance of waiting staff per customer (meters / hour / person) 40 *** * p < .05, ** p < .01, *** p < .001 ****** They were able to reduce walking distance while not reducing staying time in the dining area!
  40. 40. Indicators for position keeping 41 B2 B1 Zone Dedication Rate=Orange/Red Zone Order Defense Rate =Orange/Blue All of orders in the staffʼs zone # of accepted orders by a staff in the staffʼs zone The total # of accepted orders by the staff
  41. 41. Relation between skill level and Zone Defense/Dedication 42 IV. Expert They take all of orders in their zone while taking orders in other zone for helping others. II. Fully occupied They take orders in his/her own zone but it is not enough for covering the zone. Support by other staffs is needed. III. Well organized They take all of orders in his/her zone, but they don’t help other zones. I. Purposeless They fail to take orders in his/her zone and take orders in other zones. Training is required. Zoneorderdefenseratio(ZOD): Theratioof#ofacceptedordersbyastaffin his/herownzoneoutofallofordersinthezone Zone dedication ratio (ZD): The ratio of # of accepted orders by a staff in his/her own zone out of the total # of accepted orders by the staff Precision individual skill Teamwork performance
  42. 42. Before 43 Precision
  43. 43. After 44 Improved coverage of each zone by each staff Less need for helping other staffs (zones) Precision
  44. 44. Pre-evaluation of Kaizen Plan Considering Efficiency and Employee Satisfaction by Simulation Using Data Assimilation 45 Sensing Modeling Picking work model of employee Action HT WMS Simulation ・Analyze ・Visualize EmployeeCart Receiver VL with ID VLC Evaluation Kaizen Support Framework Simulator Planning ・Man-hour Productivity ・Worktime ・Time to spare ・Evenness of work rate traffic line Subject Extraction
  45. 45. Kaizen of the kaizen activity 46 Simulation To-Be Pre-evaluation Sensing analyze visualize As-Is Understand current status Kaizen Plan C Kaizen Plan B Kaizen Plan A Action It is possible to quantitatively decide Kaizen plan and to apply KSF to several warehouses
  46. 46. Overview of the measurement field 25m 50 meters 54 meters D A BC 47 Wide passage Narrow passage
  47. 47. 25m 48 When many employees conduct picking work, Zone A become crowded. Items in A zone were picked frequently. Overview of the measurement field
  48. 48. P1 P2 P3 HT Measurement method :Warehouse Management System (WMS) WMS manages items and provides information. When employees pick an ordered item, they are required to scan a barcode with a hand terminal. We can estimate positions from scan data 49
  49. 49. Measurement method : Visible Light Communication System Receiver 50 Measure and record the positions of the employee and carts that are equipped with a receiver
  50. 50. P1 P2 P3 HT Measurement method :Warehouse Management System and Visible Light Communication System Receiver Estimate route during pickings for combination with timestamp 51
  51. 51. Picking work model constructed and verification of reproduction A B C Order HT 52 Cart Employee confirm the orders using hand terminal Move toward shelf and into the wide passage with cart Place cart near shelf on wide passage and leave and enter narrow passage Pick up items and read a barcode with hand terminal Return to cart and place picked items on the cart In this model essentially handles plural orders by repeating these steps
  52. 52. Sorting place 2. Distribute Shelves equally for every zone 1. Divide one floor with some zones Zone Picking 3. Employee takes charge of only one zone 5. Processes a zone package  and brings it to the sorting  place 4. Created by a combination  of the same zone’s  sub‐orders. 53
  53. 53. 54 Simulated trajectories Actual method (Single picking) Kaizen plan (Zone picking) Trajectory distinguished by color for each employee
  54. 54. Actual method (Single picking) Kaizen plan (Zone picking) # of zone - # of employees N - 7 3 - 4 4 - 4 5 - 5 6 - 7 7 - 7 EF Man-hour productivity M H H H H H Work time as a team M H M M L L ES Time to spare L L L L M M Evenness of workload M M L L M L The result of the best combinations of efficiency and employee satisfaction EF (Efficiency), ES (Employee Satisfaction) 55
  55. 55. Interview with FPV Passage of Time + Over 50% cost reduction on labor cost and preparation time compared with existing time studies + Consideration of customer privacy by not using cameras + FPV with less motion sickness + Effective in episodic memory retrieval for retrospective interviews considering bounded rationality Worker’s trajectory 3D model built from a set of photos First-person view (FPV) CCE (Cognitive Chrono-Ethnography) Lite Japanese-style hotel at Kinosaki Onsen (hot spring) 56
  56. 56. 国立研究開発法人 キャビンアテンダントのおもてなし分析 57 東⼤・ANA総研の共同研究、及び東⼤・産総研の共同研究の事例⽇経情報ストラテジー 12⽉号 • 飛行中の機内でCAの動線を計測 • PDR+BLE+マップマッチング • BLEは機内持ち込み荷物の中でラピッド設置・撤去
  57. 57. 国立研究開発法人 PDR+BLEを⽤いた達⼈CAと新⼈CAの⽐較 58 CA1 業務内容 CA2 業務内容 行き 帰り 計 達人CA 15分 6分 21分 新人CA 17分 3分 20分 • ドリンク提供の帰り時間を多く作る • おかわりを申告してもらいやすい • 乗客の変化へ対応がしやすい 1. 乗客の変化に気づき対応するという 受動的な行動 2. 乗客の申告を促す能動的な行動 2種類の行動メカニズムの存在の示唆  得られた知見 [日経情報ストラテジー2015年12月号より]
  58. 58. Service Field Simulator •Supporting service design using VR technology – Evaluating service environment and its process in advance by sensing and analyzing human behavior in virtual environment Risk reduction by evaluation of the new service in advance comparison between • current layout and new layout plan • current process and new process Acquiring more detail and reliable data • Various sensors are available because of limited sensing area • Easy to control the condition As is New plan With EEG With Eye-Tracker 59
  59. 59. Continued improvement SFS Ver. 1.0 • Low resolving power: 0.2 • Short of vertical FOV SFS Ver. 2.0 • 24 Full-HD 27-inch LCD: Resolving power is improved to 0.7 SFS Ver. 2.1 • 40 Full-HD 24-inch LCD: Vertical FOV is improved (Upper 35°, Lower 58.5°) 60
  60. 60. Case studies for verifying efficiency •Gaze point analysis using combination of eye- tracking device and SFS – Hypothesis •we can do the same investigation using an eye-tracker and the SFS as real in-store marketing in-store marketing experienced person(subjective opinion): "the motion of the gazed point in the virtual environment is similar to that in the real store especially from the entrance to in front of the shelf where target products are laid out" 61
  61. 61. Case studies for verifying efficiency •Investigation for a method for measuring human interest using EEG and the SFS 62
  62. 62. Example of Analysis and Future Work 63 To compare the shopping behavior in detail, we made heat-map visualization of the stay time for each 50 cm grid in the real and virtual store. The read area indicates subjects spent longer time than other area. Because position data of the real store situation is recorded by hand, we only have the discrete position and timestamp data. Therefore, we could not compare both of them strictly, but we found out we could get the similar results. Comparison of heat-map visualization of stay Virtual store in SFSReal store
  63. 63. Dollhouse VR: An Asymmetric Collaborative System for Architectural-scale Space Design 64 提供︓慶応⼤ 杉浦先⽣ Collaborative system for multi-stakeholders
  64. 64. 国立研究開発法人 Research cases on LFF and FFL in AIST 65
  65. 65. 国立研究開発法人 Thank You!! 66 SFS Dollhouse VR CCE Lite Wearable RGB-D sensing PDR Handheld AR