The Initiative Experiments to Make Virtual and Real Space Seamless
                                      Jong-Hyoun Kim1
 ...
2.1. Direct manipulation interface

   The Tangible Media group at MIT is creating a repertoire of physical devices for Ta...
Figure 1. A die artifact and pip detection process


3.2. System flow

  This system has been implemented under ARToolkit,...
d)    Dice Client sends the packet to Dice Server.
  e)    Dice Server forwards the packet to the other party's Dice Clien...
Each TCG system has a fundamental set of rules that describes the players' objectives, the
categories of cards used in the...
The recognition system based on Image Hot Spot Extraction Algorithm can recognize any position
of planar image and can be ...
addition to origin system we just attached a Map Table to catch Card ID from detected the image. So,
as soon as the Recogn...
I am still working on following areas.
        Motion or gesture interface for networked new game interface
        Game b...
Upcoming SlideShare
Loading in...5
×

Ijipm jong hyounkim_ijipm1-070061ip

322

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
322
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Ijipm jong hyounkim_ijipm1-070061ip

  1. 1. The Initiative Experiments to Make Virtual and Real Space Seamless Jong-Hyoun Kim1 Department of Gameware Kaywon School of Art & Design 125 Naeson-dong, Euwang, Republic of Korea hyoun@kaywon.ac.kr Abstract Nearly all of current networked games still use old-fashioned interfaces - a keyboard, a mouse or a joystick at the most - which are out of touch with our daily lives. These interfaces don’t comprehend our metaphors, don’t reflect on our natural behavior aspects, disrupt immersion in a gaming, and become a serious impediment to perceiving virtual space as another extended real one. It is very challenging issue to utilize human-being’s behaviors or reactions as a way of interacting to virtual world. I attempted a few experiments on new game interfaces for a networked dice game and a TCG game. For a user to communicate with virtual game applications, the former uses dice artifacts with standardized markers which a camera may distinguish the spot on a die well, and the latter uses real TCG cards as it is instead of untouchable cards on a deck of virtual space. These new game interfaces guarantee the consistency between the game-player’s behaviors and the gaming environment. This consistency increases friendliness and immersion to a game as well as the ability of bodily sensation for a game. These new interfaces will open the new field of digital interfaces for networked games including mobile environment. Keywords: new game interface, marker/marker-less, camera- based 1. Introduction I am embarking on the exploration of developing interfaces that can regain missing intelligences and can revitalize dull creativities. To do this, interfaces must understand our metaphors, must solicit information on its own, must acquire experiences, must talk to a wide variety of people, must improve over time, and must be intelligent in context. To mirror that captures the essences and agendas represented by contemporary digital information and communications society, designing digital creativity is a critical asset to built tactical forces that blur the boundary between people and information in new computing and communication culture. Currently so many online games are serving. Nearly all of these networked games still depend on traditional input interfaces only like a keyboard or a mouse though. These interfaces can’t comprehend our metaphors, can’t reflect on our natural behavior aspects, can’t make us immerse into a game, and makes a barrier between virtual game space and our real behavior. As a trial to overcome these shortcomings I did two experiments one is marker-based the other is marker-less. The former uses dice artifacts attaching markers which a camera to distinguish the spot on a die, and the latter uses the real TCG cards as it is. The remainder of this paper is organized as follows. Section 2 compares my approaches with existing related work. Section 3 introduces my experiments on maker-based interface for the networked dice game. Section 4 represents another trial on the online TCG game using marker-less real images. Finally, I conclude with a note on the current status of my project and future work. 2. Related work A variety of research efforts have recently explored computationally augmented interfaces that emphasize human interaction using the human being’s behaviors and senses. 1 This research is supported by Ministry of Culture, Sports and Tourism(MCST) and Korea Creative Content Agency(KOCCA) in the Culture Technology(CT) Research & Development Program 2010
  2. 2. 2.1. Direct manipulation interface The Tangible Media group at MIT is creating a repertoire of physical devices for Tangible User Interface (TUI) [1]. TUI seeks to build upon the sophisticated skills that people have developed for sensing and manipulating our environment. The Bricks "Graspable User Interface" project involves putting one or more bricks onto some screen-based virtual object [2]. Bricks can then be used to physically rotate, translate, or scale and deform the attached virtual entities by manipulating the brick devices. Direct Manipulative Interfaces are computationally enhanced versions of traditional children's toys. The goal is to provide children with tools that enable them to explore complex concepts through direct manipulation of physical objects [3]. 2.2. Position and Orientation Sensor interface The ScrollPad is a mouse-mounted PDA allowing the visualization of large documents while scrolling it on a table [4]. The problem of all these techniques is that the user’s viewpoint to the screen changes permanently, losing the optimal visualization angle. 2.3. Camera-based interface MagicMouse allows the user to operate within both 2D and 3D environments by simply moving and rotating their fist. Position and rotation around the X, Y and Z-axes are supported, allowing full six degree of freedom input. This is achieved by having the user wear a glove, to which is attached a square marker. Translation and rotation of the hand is tracked by a camera attached to the computer, using the ARToolKit software library [5]. Wagner and Schmalstieg use a PDA camera to recover the position of specific markers positioned in the real environment [6]. Rohs uses visual codes for several interaction tasks with camera-equipped cell phones [7]. His IPARLA Project designed a new 3-DOF interface adapted to the characteristics of handheld computers. This interface tracks the movement of a target that the user holds behind the screen by analyzing the video stream of the handheld computer camera. The position of the target is directly inferred from the color codes that are printed on it using an efficient algorithm [8]. 2.4. Sensor-based interface Affective Computing Group at MIT is making an effort for sensing, recognizing, understanding, and synthesizing the human behavior patterns. This group discussed the use of biometric sensors with wearable computers. Such sensors allow for new interactions between the wearable and the wearer, which they based upon affect detection, prediction, and synthesis [9]. No one of above approaches has been applied to online game environment. I think my experiments are the first trial to apply camera-based interface including marker and marker-less methods into networked game applications. I am sure these experiments will give a refreshing jolt to the related researches and will open the new field of digital interfaces for networked games including mobile environment. 3. Marker-based new interface for dice game 3.1. Online dice games Dice games are games that use or incorporate a die as their sole or central component, usually as a random device. There are many online dice games; Backgammon, Dudo, Jacquet, Strat-O-Matic, et cetera. All of them requires mouse click to roll a die. Instead of mouse interface, I introduced new marker-based input interface using ARToolkit for online dice games, especially Backgammon. Backgammon is a board game for two players in which the playing pieces are moved according to the roll of dice. A player wins by removing all of his pieces from the board.
  3. 3. Figure 1. A die artifact and pip detection process 3.2. System flow This system has been implemented under ARToolkit, Microsoft Visual Studio 2008, and Windows XP. This system has been tested using the Logitech QuickCam camera. Figure 2. A dice game system flow The artifact and its pip detection are shown in figure 1 and here is the flow of new interface system as shown in figure 2. a) The player in a game rolls the die artifact on the real floor twice. b) As soon as the interface recognizes a spot on a die, it represents the digit on the screen. c) The interface sends the digits to Dice Client (e.g. Backgamman application) using Window Message Passing method.
  4. 4. d) Dice Client sends the packet to Dice Server. e) Dice Server forwards the packet to the other party's Dice Client. f) Dice Client displays the other party's pips. g) The player moves his pieces on the board. h) Now the opponent’s turn. i) Repeats from a) to h) until one of the players removes all of his pieces from the board. 3.3. Packet structure Here is the structure of major packets transferred to/from Dice Client to/from Dice Server. The contents of packets are very simple. toServerPacket consists of 3 elements and toClenetPacket has 4 elements. Table 1. Packet structure // Enumeration enum PIP {ONE=1, TWO, THREE, FOUR, FIVE, SIX} // Packet from client to server struct toServerPacket { int id; // Client Identification PIP face; // the spot on a die int bet; // bet }; // Packet from server to client struct toClientPacket { int id; // Client Identification bool result; // result, win or lose int numOfDrawer; // number of drawer int earn; // earn }; 4. Marker-less new interface for TCG game 4.1. Online TCG Game Figure 3. Two TCG real cards used in experiment
  5. 5. Each TCG system has a fundamental set of rules that describes the players' objectives, the categories of cards used in the game, and the basic rules by which the cards interact. Each card will have additional text explaining that specific card's effect on the game. During a game, players usually take turns playing cards and performing game-related actions. Restore - Make all in-play cards ready for the upcoming turn. Draw card - Necessary in order to circulate cards in players' hands. Play card - Use the cards in hand to interact with the game. Conflict - The primary method for victory in most games. Discard card - Discard to a maximum hand size, or need to refresh for next turn. In the online environment, instead of receiving physical cards, a player establishes a virtual collection that exists only as a set of data stored on a server. Such cards can be purchased or traded within this environment. For applying real TCG card into online TCG game I used two real TCG cards shown in Figure 3 for experimenting purpose. One is Albertosaurus and the other is Spectral Armor Majungadaurus card. 4.2. Real card recognition system Figure 4. Recognizing any position of the planar image
  6. 6. The recognition system based on Image Hot Spot Extraction Algorithm can recognize any position of planar image and can be calibrated to recognize the tilted image within 20 degree above and below both, so the user does not have any difficulties or does not need to train it to make the card be recognized by a camera. The system can be scaled by the distance of images from a camera. Figure 4 and Figure 5 shows you how much recognize planar and declined image respectively. As shown in Figure 4 and Figure 5, the recognition system may detect the image rotated by any angle, and may catch the tilted image within about 25 degree above and below. Figure 5. Recognizing the tilted image The recognition system has been implemented under OpenCV2.0, Microsoft Visual Studio 2008, and Windows XP. This system has been tested using the Point Gray’s Flea2 IEEE 1394b compatible camera. Figure 6 shows the picture after recognizing the specific cards. As shown in these pictures, after detecting the card the annotating information (name, value of rock-paper-scissors. figure of power, et cetera) of the card is displayed on the top-left corner of a screen. Figure 6. After recognizing the cards 4.3. System flow We use the whole structures of origin TCG games as is. In other words this means that we may apply The Card Recognition System to origin TCG game application independently as like plug-in. The system communicates to TCG client program through RPC message passing mechanism. In
  7. 7. addition to origin system we just attached a Map Table to catch Card ID from detected the image. So, as soon as the Recognition System detects an image the system send Image ID to TCG Client. TCG Client gets Card ID using the Image ID and sends it to TCG Server. The overall system flow is depicted in Figure 7. Figure 7. New TCG system flow 5. Conclusion and future work I am embarking on the exploration of developing interfaces that can work with missing intelligence and aesthetics. To do this, interfaces must understand our metaphors, must solicit information on its own, must acquire experiences, must talk to a wide variety of people, must improve over time, and must be intelligent in context. As part of this exploration, I attempted an experiment on camera-based interface for networked dice game application and experimented for seamless gaming environment between TCG virtual game space and our real behavioral space. The former experiment uses dice artifacts putting a marker which the camera can distinguish the spot on a die, and uses marker-less real card image for the latter. These new game interfaces guarantee the consistency between the game- player’s behaviors and the gaming environment. This consistency increases friendliness and immersion to a game as well as the ability of bodily sensation for a game. Nevertheless my trials are not complete deployment to a running system served currently but just experiment covering a toy system, these new interfaces, utilizing human-being's behaviors as input event, will open the new field of digital interfaces for forthcoming ubiquitous computing.
  8. 8. I am still working on following areas. Motion or gesture interface for networked new game interface Game building design mixing real space and virtual space Interaction mechanism to make real and virtual world seamless 6. References [1] B. Ullmer, H. Ishii, “The metaDESK: Models and Prototypes for Tangible User Interfaces”, Proceedings of UIST'97, 1997, 223-232. [2] G. Fitzmaurice, H. Ishii, and W. Buxtori, “Bricks: Laying the Foundations for Graspable User Interfaces”, Proceedings of CHI'95, 1995, 452-458. [3] M. Resnick, F. Martin, R. Berg, R. Borovoy, V. Colella, Kramer, and B. Silverman, “Digital Manipulatives”, Proceedings of CHI'98, 1998, 281-287. [4] FALLMAN, D., LUND, A., AND WIBERG, M., “Scrollpad:Tangible scrolling with mobile devices”, In Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS’04), 2004 [5] E. Woods, P. Mason, M. Billinghurst, “MagicMouse: an Inexpensive 6-Degree-of-Freedom Mouse”, In Proceedings of Graphite 2003, Feb 11th-13th, 2003, Melbourne. [6] Wagner, D., and Schmalstieg, D., “First steps towards handheld augmented reality”, In Proceedings of Seventh IEEE International Symposium on Wearable Computers, 2003. [7] ROHS, M., “Real-world interaction with camera-phones”, In 2nd International Symposium on Ubiquitous Computing Systems (UCS 2004), 2004. [8] M. Hachet, J. Pouderoux, and P. Guitton, “A camerabased interface for interaction with mobile handheld computers”, In Proceedings of ACM symposium on 3D interactive graphics and games (I3D’O5). 2005. [9] R.W. Picard and J. Healey, “Affective Wearables”, Proceedings of the First International Symposium on Wearable Computers, IEEE, October 13–14, 1997, 90–97.

×