An innovative use for Kinect as trackingconsumerNew idea creates a different utility proposition Nikkei Business Online January 18, 2012 Satoe Kuwahara, Nobuyuki Yajima Satoe Kuwahara Sapporo Sparkle k.k.
An innovative use for Kinect as tracking consumer —- New idea creates a different utility proposition An innovative use for Kinect as tracking consumer New idea creates a different utility proposition Nobuyuki Yajima (Research associate of Nikkei Visionary Management Institute): G " reat grand design"is a project that looks at the grand design with Ms. Kuwahara who has been de- signing information systems for long time(main purpose of this project is discussed in " aking money m with great idea". So what is todays topic? ) Satoe Kuwahara(System coordinator in Sapporo Sparkle K.K.): Today, we will consider the idea that makes the best use of technology through an example that con- sumer IT products create added value. First of all I pick up an application that tracks the is not a big deal you can put it wherever you like), shoppers location in the store and analyzes the this recognition software tracks customers move- shopping traffic lines of individual consumer by ment and behavior and allows you to acquire data using Microsofts Kinect(reference article: Shopper you need. Trackers Kinect hack is like Google analytics for This application analyzes the data acquired by Ki- retail store shelves). nect and indicates which shelves in the store are attracting shoppers and which products they touch or take. Motion sensing input device for the Yajima: Xbox 360 video game console analyses So Kinect can be used for a purpose other than a the shoppers movement. game controller. Yajima: Kuwahara: Shopper Tracker is a new product developed by Yes, its true and Microsoft has already released Argentinean developer Agile Route built off of Mi- Kinect software development kit for developers. It crosofts Kinect. Kinect is famously known for be- will allows the software developers to create new ing a completely controller-less experience. uses for motion-based system beyond gaming such as industrial robots. Kuwahara: In order to analyze customers movement, Shopper Yajima: Tracker uses pattern recognition technology includ- User can control and interact with the game con- ing spatial recognition software, heat sensor and sole without the need to touch a game controller, propriety algorithm. The Kinect measures the dis- through a natural user interface using gesture and tance to a shopper to figure out the physical attrib- spoken command. Using these characteristics, de- ute of the shopper. The data reception and interpre- velopers try to create applications for various mar- tation occurs in almost real-time as the the system ket fields such as healthcare. receives 200 frames per second. This application using Kinects technology is Kuwahara: placed on the display shelf of the store and analyz- Yes, you right. Recently, the doctors at a hospital es customers movements to provide the traffic in Toronto, Canada have attached Kinect to their flow analysis and heat maps indicating which medical imaging computer that lets them to manip- shelves are attracting shoppers and which products ulate the 3D rendered CT image by waving their they touch or take. hands, to watch the video during an operation. Ac- tually we could find similar examples in Japan. Yajima: For example, Kinect is used as a virtual fitting Can we use this application right away? room that is an augmented reality tool that users allow to try on clothes simply by standing in front Kuwahara: of a mirror. The users are even able to swap their Yes, you can use it with default setting. After ap- outfits with a wave of your hand. propriate implementation(the location you place it 2 Sapporo Sparkle k.k.
An innovative use for Kinect as tracking consumer —- New idea creates a different utility proposition the virtual world, but its quit difficult in the real world. The idea is great and it works really well. Yajima: Retailers have tried to collect these data but they As you all know, Kinect has been used other than never really succeeded. the game console such as virtual reality, robotics and other great and new ideas. Shopper Tracker, Kuwahara: one of the latest Kinect application that allows For the retailers, t is very important to understand marketers and store owners track the way consum- demand forecasting. There are multiple factors that ers shop also applies a new idea to particular issue play crucial part to maximize the profit. The retail- of need. ers should pay attention to many things including What Im interested in this application is the idea of consumer preference, competitor pricing and mar- trying to analyze customer movement with Kinect. ket trend and so on. It is a totally different way of using Kinect than what was originally intended as a game controller. Yajima: In addition to this, it is used to gather data that is What is the point of using Kinect in this applica- essential to the operation of the business. I actually tion? thought that I was done for, because the distribu- tion is one of my specialty and I involved this kind Kuwahara: of project before. I knew that Kinect has been used I think the good point about this is that this Kinect- in variety of ways, but I couldnt think of using Ki- based application does not bother customers unnec- nect for analyzing customers movement and be- essarily because it tracks and analyzes customer havior. behavior remotely. It is a completely new way of using Kinect and In the past researches, surveillance cameras placed fully utilizes its characteristics features. I really am closely to the store shelves somehow interrupted impressed by the idea and efforts of a developer. I customers natural behavior. Naturally, the credibil- think we can use this application in many areas of ity of the search was not very high. Some retailers business. used consumer monitors to collect data, but it actu- ally did not cooperate well with this research. Yajima: How did the developer create this application? Yajima: I think that using Kinect application makes the Kuwahara: analysis of the research results faster. I think they had an idea to improve store operation, but they didnt find a way how to realize it. They Kuwahara: took some time and then finally they found the way Yes thats true. Kinect-based application only re- to their idea into a reality, and their answer was trieves the data of the shoppers in the store. On the using Kinect. other hand, usual surveillance camera retrieves the The developer of this application said that they will data of whole image, consequently it takes long realize "Google Analytics in the real world", and time to analyze the data. they titled the demo video file "Shopper Trackers The unique and advanced technology used in Ki- Kinect hack is like Google Analytics for retail store nect inspires the developers. Since the release of shelves". the Kinect software development kit, the develop- A tool like Google Analytics can track website us- ers have created thousands of uses for Kinect, and ers behavior and help you analyze where your traf- the potential for additional innovation is literally fic comes and how long your visitor stays on your limitless. Shopper Tracker that analyzes customer site and so on. However in the real world, it is al- movements to provide traffic flow analysis is just most impossible to track and analyze consumers one of these, however it is changing the whole pro- behavior in the store. Trying to change the whole cess of customer analysis. picture, they developed the Kinect application. And now the customer analysis is becoming a part Retailer collects the buying history of their custom- of the business process that maintain and manages ers in order to improve sales and store operation. store, and is of growing importance in the retailing However these data do not show the relationship industry. The substantial changes in the position between the sales operation and the customers be- and the process of the customer analysis affect the havior and are not sufficient to understand the cus- way of selling a business. tomers behavior. To collect these data is easy in 3 Sapporo Sparkle k.k.
An innovative use for Kinect as tracking consumer —- New idea creates a different utility proposition Yajima: you can control a stream of customer as you desire. Fresh and innovative idea helps to create a new Moreover, Kinect can track customers bone struc- application that maximizes the new technology. ture and recognize each customer as an individual. And the new application creates a new usage occa- Using these information, it is possible to track one sion. particular customers shopping frequency and be- havior etc. Kuwahara: And now, we look at a way of using the result of Yes, I think that is actually a power of design. De- the store research and analysis on the sales promo- signing the application with state-of-art technology tion activity. Using Kinect and the real-timeness of changes in the design of retailing. the application, it is possible to give messages in However, this structure, cycle of change has not accordance with the customers movements. been completed. Shopper Tracker demonstrates For concrete example, commercial message for the great potential for use of the new technology as product in customers hand on POP display or dis- analytical tool. From here, we go into the next count service by scanning bar-code coupon shown stage and should expand the idea of the technology on the display with customers mobile device. to create more valuable applications. We also can use the data accumulated by this re- Using Shopper Tracker, Kinect-based application search and analysis system. For example, showing will inspire you and help you to expand you imagi- a detailed information of a product that is frequent- nation. As many people have said Kinect can be ly taken from a shelf but never purchased on the used in very many different application with little display. In order to promote the product, the sys- modification and POS system feature a combina- tem automatically chooses what the information to tion of Kinect technology is heading to laptops provide to the customer. soon. There is another way of utilization of this research and analysis system. Using best available data, Yajima: combination of the POS system and the results of So we can develop various applications based on research and analysis, to determine appropriate dis- the need, demand and innovation. And they can be count price rate. used in various usage scenarios. Kuwahara: An exciting and usable product Thats correct. Kinect itself and the idea used in this application is very attractive. However, the Yajima: When I surf web pages, I often see the ad- structure of the application is quite simple. So we vertisement of sales promotion, and it is actually can use it in various ways. annoying. I think what you have suggested are This application is made of four parts, including a pretty similar to this advertisement. gesture recognition, retrieve and store of the data, analysis of the data, and display of the analysis re- Kuwahara: sults. The usage of the application will be deter- The important point is to make customers satisfied mined by the data source type selected and the op- or to give them help they need. In order to achieve eration process will be changed by of displaying this goal, we should design application from the the result of the analysis. viewpoint of customer. There is a wide variety of store research and analy- The usage scenario or use case of application will sis for a retail company. For example, if you want decide its effectiveness. However this is not only to research consumers behavior at the testing for the application but the any sales promotion ac- event, you should track the customers movements tivities that we propose. So the usage scenario is and how many people are tried of tasting. If you very important for retailing business and we have want to research the relationship between store studied effective and efficient ways to use the ap- shelf space and unit sale in a store, you should plication for long time. track the customers movements and shopping hab- its. Yajima: By placing a number of Kinect all over the store, I think this study can be used in other area of busi- you can check all area at all times. And it makes ness. you to analyze the information such as a level of crowdedness in the store, customers shopping be- Kuwahara: havior(its purchasing pattern and merchandize se- The operation is led by knowing about the move- lection), where shoppers stop in a store and what ment of worker, and is guided by connecting the they really do and so on. Using these information, 4 Sapporo Sparkle k.k.
An innovative use for Kinect as tracking consumer —- New idea creates a different utility proposition workers movements and the product. The relation- ship between work flow and work time can be ana- lyzed. The idea of store research and analysis is devel- oped by "Kinect + data utilization". However if you focus more on the data, the range of applica- tion will be expanded. Therefore we should focus on the currency of the data. There is a production system in which the industri- al robot automatically works by knowing about the movements of the worker. So we can use Kinect in the field of the industrial robot manipulators. The use of Kinect is now expanding and is getting better and better. Some consumer devices are very usable and may even inspire your imagination. The application that fully utilizes those usable and ad- vanced devices shows us that the idea realizes its full potential. There is a surprise that "we are able to do some- thing like this" and it inspires our imagination. The usage scenario of the use case will be expanded by the ideas and innovations of people. I think this is a great pleasure of developing a new device. 5 Sapporo Sparkle k.k.
Publisher Sapporo Sparkle k.k.Date of Issue January 31, 2012No. 2012013101