Your SlideShare is downloading. ×



Published on

Published in: Business, Technology

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-503 Multi-Touch Table With Rfid Technology For Hotels Rajat Verma#1, Preksha Gupta#2, Prof.C.Malathy#3 # Department of Computer Science and Engineering, SRM University, Kattankulathur Chennai, IndiaABSTRACT With the introduction of multi-touch, a available in different forms. Due to the improvementnew form of human computer interaction is of processing power of modern desktop computers,introduced. Due to recent innovations multi- it no longer requires expensive equipment. Moderntouch technology has become affordable. Unlike computer interaction consists of a monitor, keyboardinteraction on a desktop computer multi-touch and a mouse. Limited by the operating system, itallows multiple users to interact with the same allows us only to control a single pointer on thedevice at the same time. To demonstrate the vast screen. With multi-touch, multiple input pointers arepossibilities of multi-touch technologies an introduced to the system which all can be controlled‘interactive RFID-based Multi-touch Device’ can independently. Depending on the size of the displaybe constructed which can be utilised in a hotel. multi-touch allows multiple persons to interact withThe objective of the project is to enhance the the same display at the same time.customer’s dining experience and to provide anenjoyable and user friendly interface thereby Radio-frequency identification (RFID) isreducing human effort and time. The multi-touch the use of a wireless non-contact system that usestable constructed is a camera based multi-touch radio-frequency electromagnetic fields to transferdevice which is designed using front-side data from a tag attached to an object, for theillumination technique. Card associated with a purposes of automatic identification and tracking. ItRFID tag will be used, priced to a specific amount has been extensively applied to various fields suchwhich is issued to the customer at the reception as product surveillance, supply chain, automobiledesk. Each order table at the restaurant will have parts tracking and telemedicine. A RFID system isa multi-touch device along with the RFID reader. composed of three major components, includingCustomer can now interact with the multi-touch reader, tag and middleware. Unlike traditional bar-device by showing his RFID card and place an code system, RFID systems can carry dynamic asorder by selecting from the menu displayed on well as static data. Some tags require no battery andthe order table. This project proposes the are powered and read at short ranges via magneticmethodology of designing an interactive system fields (electromagnetic induction). Others use a localalong with applications to verify the effectiveness power source and emit radio waves (electromagneticof the same. radiation at radio frequencies).Tag contains electronically stored information which may be readKeywords - multi-touch, RFID, interactive, front- from up to several meters away. Unlike a bar code,side illumination the tag does not need to be within line of sight of the reader and may be embedded in the tracked object.1. INTRODUCTION The project will consist of a multi-touch Multi-touch (or multitouch) denotes a set of table containing a RFID reader. A card associatedinteraction techniques which allow computer users with a RFID tag will be used. Two computerto control graphical applications with several fingers. systems will be required, one to be used as a clientMulti-touch consists of a touch screen (screen, table, and other one as server. Network connectivity needswall, etc.) or touchpad, as well as software that to be established between the reception desk, multi-recognizes multiple simultaneous touch points, as touch table or order table and chef desk. This wholeopposed to the standard touch screen (e.g. computer setup can be utilised in a hotel/restaurant. Atouchpad, ATM), which recognizes only one touch customer can buy a card (with the RFID tag) pricedpoint. This effect is achieved through a variety of to a specific amount which he thinks is required ormeans, including but not limited to: heat, finger necessary for future transactions. Each table in apressure, high capture rate cameras, infrared light, restaurant will have a multi-touch device along withoptic capture, tuned electromagnetic induction, the RFID reader . Customer will show the card onultrasonic receivers,transducer microphones,laser this device and his details will be displayed alongrangefinders, and shadow capture. Multi-touch with the menu card on the table. He can then placedisplays are interactive graphics devices that the order according to the balance left in his card.combine camera and tactile technologies for direct This order will be sent to the chef‟s system. Aon-screen manipulation. Multi-touch technology is database will be needed for this purpose. Usingnot entirely new, since the 1970s it has been queries the chef will be able to identify the table 497 | P a g e
  • 2. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-503from which the request has been made and confirm in his account. As soon as he places the order, thethe order. ordered amount will be deducted from customer‟s account updating the database and timer will start2. SYSTEM OVERVIEW showing the estimated time required for the service In this section, we will overview the to be processed. The customer‟s request will be sentproposed rfid-based multi-touch interactive system to chef system. After receiving the order, estimatedfrom two different angles. Figure 1 shows the time required for processing the request (preparingstructure of the proposed rfid based interactive the order) will be sent to the customer table by thedisplay system. One is the hardware configuration chef.that contains two main components, RFID readerand multi-touch table. The rfid reader is user toidentify the user or customer which uses the rfid tagand the multi-touch table is an interface for user tooperate the proposed interactive display system. Theother part is from software viewpoint where theclient and server are connected in an ad-hoc networkand applications are run on both sides along with theblob tracker for multi-touch. Fig. 2 A flowchart of the proposed interactive RFID- based multi-touch device.Fig. 1 A graphical overview of RFID-based multi-touch interaction pipeline 3. HARDWARE AND SOFTWARE Figure 2 shows the flowchart of the rfid- CONFIGURATIONbased interactive multi-touch system. The This project aims at developing the rfidreceptionist at the reception desk will login into the based multi-touch interactive system whichsystem by entering the appropriate username and combines display technology with sensors which arepassword. The customer arriving at the Reception capable of tracking multiple points of input. TheDesk has two options: New Registration, for a new idea is that this would allow users to interact withcustomer or Recharge for an existing customer. the computer in a natural way .Using this displayAccordingly details like first name, last name, email more than one user can interact with the systemid, mobile no, amount and the rfid tag unique simultaneously.identification number for the customer will be added Hardware and software specifications for this projector updated in the existing database of the hotel. Now are as follows:-the customer is ready to place an order at the Hardware Requirements:-restaurant. The rfid reader is connected to the systemavailable at the customer side which is also in  Web Camera of minimum resolution 640 X 480conjunction with the multi-touch table. At the same  Acrylic sheet or glass( thickness ~6-10 mm)time connection needs to be established between the  Two computer systems with minimum of 1 GBcustomer table and the chef system using RAM.adhoc(computer-to–computer) network by manually  Light source for increasing light intensity whenconfiguring the IP addresses of both the systems. ambient light is not sufficient and to overcomeThe customer will show his rfid card issued to him shadow problem.before at the reception to the rfid reader at the  RFID-Reader module with RS232 table. As soon as she shows the card, user  RFID tag/cards.details will be displayed including the amount  USB to serial adapter.present in his account along with the menu card.  A display projector (if needed).Now the customer will place the order by selectingdifferent options available on the menu using themulti-touch table according to the amount available 498 | P a g e
  • 3. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-503Software Specifications:- user views the scene on the panel and decides to Touchlib, free open source multi-touch framework interact. To interact with the device, the user touches libraries like CONFIG.BAT, GATEWAY.BAT the device panel. On the hardware level the camera AND SERVER.BAT are required for configuring registers touch. Because the captured frame might this multi-touch device. not only include the contact points but also a (static) VISUAL BASIC-6.0 and Video Ocx Tools have background, it is required to perform image been used in application development for the processing on each captured frame. The captured multi-touch device. frame is converted to a gray scale image and the Hyper terminal to test connection with rfid reader. static background is removed by subtracting the Client and server side applications using VB6.0 current frame with a reference frame. As a result the for reception, table and chef desk. frame only shows white contours (blobs) which are Windows socket programming used for both client the points of contact. By tracking these blobs the and server side applications. system becomes capable of sensing touch. In order to track the blobs, the positions are transmitted to the3.1 Design considerations of the rfid-based multi- blob tracker which matches blobs from earlier touch device. frames with the current frame.After processing, There are various techniques available for events will be triggered which can be used in athe construction of the multi-touch device. But from multi-touch capable application. The applicationthe starting of this project, my prime area of modifies objects in the current scene according toconsideration was the camera based multi-touch the new blob positions. The result is returned to thetechniques. The camera based multi-touch user through the display. The performance of atechniques have been widely utilized in various camera based multi-touch device depends on themulti-touch devices. Microsoft Surface which used hardware and software. When a user touches areleased previously and created a lot of fuss was multi-touch device, it expects the device to respondbased on a camera based multi-touch technique directly. The responsiveness of the device depends(precisely saying rear-side illumination on the time it needs to process the users input andtechnique).Front-side illumination technique is present the users a result through the display. In thecamera based multi-touch technique but because of interaction pipeline two levels are important, theits impreciseness and low reliability, its potential has hardware and the software level. Using a cameranot been fully utilized. In my project I have used which is capable of 30 frames per second allowsfront-side illumination technique only but with smooth interaction with the system. However, thiscertain modifications so as to increase the reliability requires a system that can handle image processingand preciseness of the technique. Before designing a and blob tracking in 1/30 of a second. Amulti-touch device one has to consider various combination of smart algorithms implemented infactors such as the environment in which the device software and fast hardware helps to minimize thewould be used, the target audience, the cost which latency and increase the responsiveness of the device.would be involved in the construction of the device,etc. The following considerations were kept in mind Front-side illumination techniquewhile making the device: There may be a case when A technique for designing a touch interface,the surrounding light may not seem sufficient. To based on diffused illumination is known as Front-tackle that case an array small bulbs are put on the side Illumination (FI). This technique is based onboundary of the touch sensitive surface. The cost of light being diffused. However instead of usingconstruction of the device needed to be kept low; infrared light sources, it depends on the ambienthence I decided not to use the projector. Instead the light from the environment. With FI the diffuser iscamera inputs are directly supplied to the computer. attached to the front side of the display. The ambient light illuminates the diffuser, which from the cameras point of view, results in an evenly colored rectangle. By touching the surface, shadows will appear underneath the fingertips because the ambient light cannot reach it. These points are detected by the camera and a touch is registered. Figure 4. shows a basic setup for FI. Because the used touch library requires touched spots to be colored white, a simple invert filter is being applied. FI can be seen as theFig. 3 Multi-touch table outside and inside view. cheapest way of creating a camera based multi-touch capable device.3.1.1 Camera-based multi-touch techniques Camera based multi-touch devices share thesame concept of processing and filtering capturedimages on patterns. The pipeline begins when the 499 | P a g e
  • 4. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-503 (static) background a highpass filter is applied. This filter compares the contrast in the image and only allows `high values to pass through (depending on a pre-set value). As a result only the touched spots will be visible. Depending on the used diffuser and infrared LEDs the result might show weak blobs, in this case we can add a scaler filter to amplify the brightness of the blobs. Finally, the rectify filter is applied resulting in clear blobs ready for tracking. It is used to filter out noise and reduce the gray scale image to a black and white image only displaying the actually touched areas. An example of the Touchlib image processing results can be found in Figure 5.Fig. 4 Schematic view of the multi-touch panel usingfront side illumination3.1.2 Choosing an appropriate camera device When choosing a camera it isrecommended to find out which sensor is used andwhether the data sheets are available for this sensor.Whilst high-end consumer USB cameras are capableof transmitting images of VGA resolution (640X480pixels) at reasonable frame rates, they oftenintroduce latency. Because this reduces theresponsiveness of the multi-touch devices, it isrecommended to use a FireWire based camerainstead. Depending on the size of the display and theprojected image it is recommended to use at least acamera running a VGA resolution because of Fig. 5. Example of Touchlib filter chain usingprecision. The frame rate should be at least 30 front side illuminationframes per second to allow smooth interaction. 3.2.2 Blob detection and tracking3.2 Multi-touch detection and processing. In Touchlib the blob tracker handles theTo perform camera based multi-touch detection and blob detection and tracking. Blob detection is doneprocessing several frameworks are available. In this to detect touch in a camera image. In order to followpaper we describe the used multi-touch framework, the movement of touch, the blob tracker compareshow a multi-touch framework connects to a multi- the detected touch locations in a frame with thetouch application and the different types of gestures. positions of the previous frame. In each time step, Touchlib requests a frame from the video camera.3.2.1 TouchLib After a frame has been processed, the resulting Our multi-touch system uses Touchlib image is used to perform blob tracking. By using thewhich is a free open source cross platform multi- OpenCV function cvFindContours() we obtain a listtouch framework which provides video processing of contours found in the current frame. All foundand blob tracking for multi-touch devices based on contours (and their properties) are stored in aFTIR and DI. Video processing in Touchlib is done contours list. Each contour is checked on whether itthrough the Intels OpenCV graphics library. is a fiducial marker or a touch. On each contourTouchlib currently runs on MS Windows, Linux and Touchlib tries to fit a polygon which matches theMac OS X. When Touchlib is used for the first time outlines of the contour. If the fitted polygon is build(in an application) it stores a copy of the current out of four points (we are talking here of squareframe into the memory. This frame is used as a fiducial marker) it might be a possible fiducialreference frame and used to perform background marker. Next it will check whether the angle of allsubtraction on the next frames. Front-side the corners matches approximately 90 degrees. If theillumination requires more video processing filters result is true, the contour is considered a fiducial andbefore blob tracking can be applied. First a capture the position (center) will be calculated. It will alsofilter is selected depending on the used interface assign a unique tag identifier based on the pattern(USB or Firewire). If the system uses FI it is found within the square. If the polygon is morerequired to add an invert filter because Touchlib complex than four points, it is assumed to be a touch.expects white colored blobs. The next filter is the Touchlib will fit an ellipse on the contour. Thebackground subtraction filter. After removing the 500 | P a g e
  • 5. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-503properties of the ellipse are used to determine the implemented in conjunction with the Touchlibposition, orientation and the size of a blob. If the size software designed for the multi-touch table. Theof the blob fits the minimum and maximum three modules for the reception desk, multi-touchrequirements on height and width, the blob will be table ,chef table are integrated using the algorithmsadded to the blob list. specially designed for this purpose and various In order to track blobs it is required to have forms which demonstrate the use of such anat least two data sets that contain blobs in different application is shown in the project. Figure 6 showsstates. We first define the two data sets. the various modules developed for RFID integratedThe first data set contains the blob list from a multi-touch device.previous frame and is defined as follows: p1; p2; p3;…………….; pnwhere n is the number of active blobs in the previousframe. The second set contains the blobs list of thecurrent frame and is defined as follows: c1; c2; c3;……………..; cmwhere m is the number of active blobs in the currentframe.After each frame, the data from set p is replaced byset c. Set c will be filled with the new blob list.3.2.3 Generating events In the last part of the blob tracking, Figure 6(a) Reception desk module.Touchlib prepares the events to be triggered. FirstTouchlib walks through the list of currently activeblobs. If a blob is new in the current frame (it did notmatch with a blob in the previous frame) it willcontain an ID with the value -1. In that case a newID will be generated and assigned. After theassignment a “touch event” of the blob will bedispatched. If a blob in the current frame matches ablob of the previous frame, it will use the sameidentifier of the previous frame. Depending on theamount of movement an update event will bedispatched. If the delta of movement is higher thanthe set value of distanceThreshold it means the blob Figure 6(b) Chef desk module.„traveled‟ over a larger distance than one wouldassume to be correct. In this case instead of anupdate event a “touch event” is dispatched instead. Ifthe delta of movement from the current and theprevious frame is below the value ofminimumDisplacementThreshold, no update eventwill occur. After Touchlib processed the list ofcurrently active blobs, it compares the list with theblob list of the previous frame. If a blob was activein the previous frame but has not been found in thecurrent frame, the blob is marked for deletion and a“touch up event” will be dispatched.3.2.3 Programming Language Interfaces The client side or reception desk module Figure 6(c) Order table module.will consist of programs and algorithms which arewritten in accordance with communication 3.3 RFID Reader/RFID tagperformed over a ad-hoc network. The serial An RFID system consists of a tag readercommunication ports will be able to transfer and (also called the interrogator) and a tag. Allreceive the data via the RFID reader and the same communication between the tag and reader occursinformed can be sent via the network to the server completely through a wireless link that is sometimesside or the chef system using windows socket called an air interface. Through a sequence ofprogramming. The coding parts are written in Visual commands sent and received between both devicesBasic 6.0 which creates several forms or applications (called the inventory round), an RFID reader canto be used by the interactive system and are identify the electronic product code (EPC) of an 501 | P a g e
  • 6. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-503RFID tag. For passive tags, the basic idea is that the encoding and updates. Passive RFID Tags have nointerrogator initiates an interrogation round with a internal power supply. An electrical current inducedquery command. The query command essentially in the antenna by the incoming radio frequency“wakes up” the tag, which responds with the signal provides just enough power for the integratedappropriate information. Figure 7 shows a basic circuit (IC) in the RFID tag to power up and transmitblock diagram of the tag/reader system. Figure 8 a response. RFID tags communicate in various waysshows the RFID reader with RS 232 port used in the with the RFID reader. The aerial (antenna) has to beproject. designed to both collect power from the incoming signal and also to transmit the outbound signal. Lack of an onboard power supply means that the passive RFID tag can be quite small: commercially available products exist that can be embedded under the skin. In this project I will be using a passive RFID tag. These do not require batteries and have an unlimited life span. As we have already seen there are two important components of a RFID tag – A microchip and a coil (antenna). The antenna receives power Fig. 7 Block diagram of typical RFID tag/reader and RF signals from the RFID reader and sendssystem those signals to the chip. The chip receives those signals, computes them and sends back the data to RFID reader. We can figure out the precise working of a RFID tag through this diagram. Fig. 8 RFID reader with RS 232 kit. Note from Figure 7 that many RFID readersand measurement systems actually use a three-portRF component called a circulator that gives bothtransmit and receive front ends the ability to use thesame antenna. Note that with many RFID, standards,timing information between transmit and Fig. 6 Working of RFID tag.receive commands is defined by strict guidelines. Infact, a sort of “handshaking” is required between the To recognize the identity of an RFID tag,tag and reader to complete an interrogation RFID reader sends radio signals which is capturedround. This actually creates a unique test challenge by the coil (working as antenna) for the tag. The coilbecause the instrumentation must be capable of the receives these signals as alternating current andsame behaviour. On an interrogator, an embedded passes to the chip. The chip extracts both the powerprocessor is required to decode and generate and the information from this alternating current. Bycommands within a tight timing interval. As communicating with the non volatile memory of thediscussed in a later section, this design is quite chip that stores unique id as well as othersimilar to field-programmable gate array (FPGA)- information, it sends back the required signal to theenabled RFID measurement systems, which use antenna which is then transmitted to the RFID reader.similar embedded processing to fully emulate either There are three main roles a RFID reader plays othera tag or a reader. RFID tags come in many shapes than signalling RFID tag to transmit desiredand sizes each suited to a specific application, but all information back to the RFID reader. Firstly, aRFID tags can be generally grouped into two main RFID reader has the responsibility of keeping RFIDcategories, regardless if they are encased, a sticky tags powered up. Secondly, a RFID readerlabel or just a solid button like tag .RFID tags are demodulates incoming signals from the RFID tageither “passive” (no battery) or “active” (self- down. This process slows the incoming signalspowered by a battery). RFID tags also can be read- down enough so that the RFID reader is able toonly (stored data can be read but not changed), process the signals. Finally, after the incomingread/write (stored data can be altered or rewritten), signals are slowed down, RFID then has theor a combination, in which some data is permanently responsibility of decoding the incoming signals intostored while other memory is left accessible for later the words people can interpret 502 | P a g e
  • 7. Rajat Verma, Preksha Gupta, Prof.C.Malathy / International Journal of Engineering Research and Applications (IJERA) ISSN: 2248-9622 Vol. 3, Issue 2, March -April 2013, pp.497-5034. CONCLUSIONS AND FUTURE WORK [6] A novel interactive system that integrates [7] Experiences with interactive multi-touchRFID and multi-touch technologies is proposed in tables : Wim Fikkert, Michiel Hakvoort,this paper. Customers can now have a whole new Paul Van Der Vetexperience by using the touch interface for ordering [8] food at restaurants and paying via the RFID y-Article.asp?ArtNum=4 for RFID working.card issued to them. With the construction of our [9] camera based multi-touch table we have w-rfid-tag-works?page=5 Rfid tags.demonstrated that multi-touch technology has [10] Hyperterminalbecome affordable. The precision, reliability and of a multi-touch table depends on the al.shtmlused hardware. Because of the use of a camera based [11] Configurationmulti-touch technology, ambient light has a large in the tracking performance. Compared to perterminal.htmldesktop applications, users were able to manipulate [12] in a natural way of touch. Instead of Reader-Kit-Rs232-Interface.html RFIDcorrecting the entire camera image, it is also possible reader kit with RS 232 interfaceto apply position correction on the output of the blobtracker. Currently image processing is done on theCPU. In order to reduce the load on the CPU it ispossible to use graphics processing units (GPU) forimage processing. The current blob tracker used inTouchlib is not scalable. Depending on the usedhardware, Touchlib will fail to track fifty or moreblobs in real-time. A smarter algorithm would notonly take the distance into account but also theregion and direction of a blob. It would also beinteresting to use the GPU for blob tracking. Basedon our own experiences the touch surface should beimproved. The friction with the acrylic makes it hardto use the system for a longer period of time.ACKNOWLEDGMENTS This research project would not have beenpossible without the support of many people. Theauthor wishes to express his gratitude to hissupervisor, Prof.C.Malathy of Computer Science andEngineering Department who was abundantlyhelpful and offered invaluable assistance, supportand guidance. The author wishes to express his loveand gratitude to his beloved families for theirunderstanding & endless love, through the durationof his studies.REFERENCES [1] Han, J.: Low-cost multi-touch sensing through frustrated total internal reflection. In: UIST ‟05: Proceedings of the 18th annual ACM symposium on User interface software and technology, New York, ACM Press (2005) 115–118. [2] RFID :A Technical Overview and its application to the enterprise by Ron Weinstein. [3] Jefferson Y.Han Multi-touch Interaction wall In Siggraph ‟06:ACM Siggraph 2006 Emerging technologies. [4] Jazz Mutant .Lemur Input Device.2006 [5] Didier Brun .Mouse Gesture Recognition 2007 URL 503 | P a g e