040708 Experimental Ecomonics Metanomics Transcript
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

040708 Experimental Ecomonics Metanomics Transcript

  • 885 views
Uploaded on

Metanomics is a weekly Web-based show on the serious uses of virtual worlds. This transcript is from a past show. ...

Metanomics is a weekly Web-based show on the serious uses of virtual worlds. This transcript is from a past show.

For this and other videos, visit us at http://metanomics.net.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
885
On Slideshare
885
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
2
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. EXPERIMENTAL ECONOMICS IN VIRTUAL WORLDS: APRIL 7, 2008 ROBERT BLOOMFIELD: Welcome, everybody, to Metanomics business and policy in the Metaverse of Virtual Worlds. Today we have three academic researchers joining us today, one of my favorite types of sessions. Keep in mind that Metanomics is really intended to serve four distinct communities, as we describe it on our Facebook profile, and I encourage you to check it out there and join our Facebook group Metanomics. The four communities are first, Real World Enterprises seeking to understand how they can use Virtual Worlds to achieve their Real World strategic goals; virtual entrepreneurs who are identifying ways to serve both Real World and Virtual World clienteles; World developers and regulators who are looking at the forces and policies that are going to affect the future of the Virtual World industry. And then finally, academics who are trying to understand and educate these other communities or who are using Virtual Worlds as a laboratory for their research. Now it turns out that this last group is pretty small right now because Virtual Worlds research is, frankly, a pretty gutsy move for most academics. I have tenure, as does John Duffy, our guest today from the University of Pittsburg, so we can afford to take a flyer on this. If it doesn’t pan out, we’ve lost nothing but a little time. It’s not like it’s a major career risk. Our two other guests are taking a bigger chance. Thomas Chesney is a lecturer from Nottingham University Business School and Steve Atlas is just completing his Masters degree at Tufts University and is entering a doctoral program in business this fall. So these
  • 2. two are taking much more significant risks, and I, for one, applaud them for it. If you know of others who are taking these types of risks, please do point me to them. I’d love to be able to have a lot more shows like this one. Now we come to our long list of thank-you’s and announcements before our show. First, thank you to your sponsors SAP, Generali Group, Cisco Systems, Saxo Bank, Kelly Services, Sun Microsystems and my own institution, Cornell’s Johnson Graduate School of Management. Of course, a big thank you to SLCN which is using some new technology today. So if you are seeing us at one of our event partners or if you are seeing this on SLCN.TV, you probably are seeing my lips move, which is a new and very exciting thing that I’ve never experienced in Second Life, only in There.com. I’d like to thank JenzZa Misfit, of Rendezvous, for letting Metanomics hold this event here on Muse Isle. Muse Isle’s also going to be the location of a new regular event: Metanomics Rewind, during which we’ll have a special rebroadcast of Monday’s show every Tuesday, 3:00 P.M., Second Life time. So this is a chance to catch the show. Any shows that you miss, you can see them the next day. I know a lot of you in the U.S. find this a difficult time. So come along, see the show and watch it with other people who share your interests, and, hopefully, you can make some new friends. I’d also like to say hi to those of you who are watching from Outreach Amphitheater of the New Media Consortium Educational Community sims. New Media Consortium, NMC, is a community of hundreds of leading universities, colleges, museums and research centers. I think many of you probably know that NMC has been in the news lately because their CEO,
  • 3. Larry Johnson, testified to Congress last week on the state of Virtual Worlds, along with Linden Lab CEO Philip Rosedale and some others. Now, in my book, Larry wins the prize for the best opening line in congressional testimony. He says, “My name is Dr. Larry Johnson, and I have an avatar.” Well, my name is Rob Bloomfield, and I have an avatar, Beyers Sellers, and I am very glad to have you all here for this show. I also want to tell you very quickly about the other shows that are coming up. Next Monday we have Cory Ondrejka, a former chief technology officer of Linden Lab, along with Wagner James Au, former resident journalist, and they’re going to discuss the history and the future of Linden Lab, Second Life and the Metaverse. We’re going to take a broader Metaverse view the following week with Mitch Wagner, of Information Week, and Steve Prentiss, from Gartner Incorporated, who are both very well versed analysts not just of Virtual Worlds but the larger tech community in which they reside. Finally, April 28th we’re going to take a trip into a Virtual World called Sophia’s Garden based on Quaq and will be joined by Quaq’s CEO, Greg Nuyens. If you have ideas for guests or topics, please do let us know. You can email me, rjb9@cornell.edu. Send an IM in Second Life to Beyers Sellers, or you can get us through contact at metanomics.net, our website. You can also follow us on Twitter and send us messages, replies, on Twitter. Finally, I do want to encourage everyone to join the Metanomics Group in Second Life. You can do that right now. We use that for backchat, which is already cruising along right now I see. So I and my guests will be watching the backchat during the show so that we can
  • 4. respond to your questions. Okay, now on to the show. For many of you, the first question may be this: What is experimental economics? Well, the idea’s really pretty simple. You set up a small economy under controlled conditions, manipulate some key variables that economic theory says should matter, and then you compare what happens to the predictions of economic theory. Professor Vernon Smith won the Nobel Prize in 2002 for laying out the basic tenets of induced value, which he argued makes it possible to test economic theory in laboratory settings. What does induced value mean? Well, it means we’re trying to get the people in the economy that we are creating to actually have the incentives that economic theory assumes. So the first tenet here really is rewards. There have to be rewards of some value that participants can pursue within this small economic setting. The second one is non-satiation, which is a two dollar word, which, I guess, would make it a 700-Linden word or so--excuse me--550. But it said non-satiation basically says participants have to prefer more reward to less. They can’t ever be satisfied, no matter how much they earn, which we say is very much the case in economic theory. Saliency means that the rewards and how the games actions and outcomes map into those rewards must be clear and apparent to the people in the economy. And finally, dominance, that the rewards have to be large enough relative to anything else the participants care about, that what the participants in the experiment care about really is the rewards that we are identifying. So people have been doing experimental economics for many decades and in a lot of different fields. So two of our guests today, Steve Atlas and Thomas Chesney, are conducting experiments in what’s called behavioral game theory. They basically construct
  • 5. very simple economic interactions and use a blend of economics and psychology to predict what’s going to happen. Our third guest, John Duffy, uses experiments to study macroeconomics, the behavior of entire economies, with a fair bit of emphasis on bank runs and monetary policy. Or maybe I just emphasize that because my own interests lie in that direction. I typically use experiments to study financial economics, how financial markets behave and, in particular, how they respond to financial reporting regulations and trading regulations. Okay. Now, Thomas, I’d like to start with you. So first, welcome to Metanomics. THOMAS CHESNEY: Thank you very much. ROBERT BLOOMFIELD: We’re delighted to have you on the show. Now you recently put out a working paper in which you describe your experience and the results of a number of games that you conducted in Second Life. Let’s start with the simplest. You ran some experiments for the Dictator game and the Ultimatum game. And, for starters, can you just describe those games for our listeners? THOMAS CHESNEY: Certainly. The Ultimatum game and the Dictator game are used to examine the behavior surrounding fairness and altruism. So if we start with the Ultimatum game, we bring in participants, and we pair them up, and we say to one of the two in the pair, “We’re going to give you a stake. We’re going to give you some money, and you have to decide how you want to split it between yourself and your partner.” And, of course, you
  • 6. don’t know who your partner is. And once you’ve made that decision and communicate it to us, we go to the partner, and we say, “This is the split that has been suggested. Do you want to accept it?” And, if the person number two thinks that there’s a fair split, they will accept it. And, if they think it’s an unfair split, they will reject it. And, if they reject it, people get nothing. You go away empty-handed. So this is to examine fairness. And with the Dictator game version, you go to the first person and you say, “Here’s the stake. How do you want to split it?” And whatever they decide, you take it to person two, and you just give them that amount so they don’t have the option of rejecting it. And that examines a number of things. First of all, it looks at whether or not the behavior in the original Ultimatum game was strategic or not, whether or not it was genuinely altruistic or if people were just splitting it in a way they thought was fair just so that they could get their split. So we did this, and we tried to make our Second Life games as close to the real versions as possible. So we went to a number of papers, and we took the method that was used in those papers, and we tried to recreate it as closely as possible in Second Life to see whether or not behavior in Second Life was as it was in Real Life. Because there are a number of theories in the Information Systems--and I’m from the Information Systems discipline. There are a number of theories that say that, in this sort of environment, people will act differently. For instance, I could act hyper-selfishly because you don’t get to see me blush. I can cheat you, and it doesn’t matter because you don’t really know who I am. So we expect it--
  • 7. ROBERT BLOOMFIELD: So, Thomas, when you say “hyper-selfishly,” let’s talk about that Dictator game for a minute. THOMAS CHESNEY: Mm-hmm. Yeah. ROBERT BLOOMFIELD: So I come into your experiment. I’m the dictator. You give me--I’m sorry, how many Lindens was it? THOMAS CHESNEY: We choose 3,000 Lindens because that’s round about ten dollars, which is what the Real Life versions use. ROBERT BLOOMFIELD: So I’m given 3,000 Lindens. I get to decide how much of it I keep and how much I give to you. And there’s nothing keeping me from giving you a Linden, if I want to, so I end up with 2,999 Lindens? THOMAS CHESNEY: Yeah, mm-hmm. ROBERT BLOOMFIELD: So now I guess the first question is: What do people do in the traditional laboratory experiment? And why do you think that would be different in a Virtual World? THOMAS CHESNEY: Yeah. Okay. In the traditional version, people tend to give about 20 percent of their stake, so you wouldn’t keep the whole thing. You would hand over whatever 20 percent of 3,000 is. And I guess perhaps the reason they do that is because
  • 8. they are face to face with the experimenter, and they’re in the same room as their partner, even though they might not know which one of the group is their partner. But, in this environment, I guess you could argue that people might act more selfishly because they don’t get to see the experimenter’s face. They don’t have to look me in the eye and say, “No, I’m not giving anything,” or, “I’m only giving one Linden.” So that’s why we might expect behavior here to be different from the Real World. ROBERT BLOOMFIELD: Now you found it was really pretty similar. THOMAS CHESNEY: Yes, exactly. Statistically, there was no difference so people act the same here as they do in the lab, in the real lab. ROBERT BLOOMFIELD: So what’s your takeaway from that? I mean is your takeaway that it’s safe to conduct these types of experiments in Second Life rather than in the controls of a laboratory setting? THOMAS CHESNEY: Exactly. Yes. That’s one our main findings. We think that it’s a methodological contribution so if, for instance, somebody wanted to do some behavioral game experiments on the cheap, Second Life is an environment where you could do that. So for instance, I might want to do experiments with Spanish people, and there’s a big Spanish community within Second Life, and I can go there, and I can be confident, or I can refer to my results to try and imply confidence that those Spanish people will act as they would have if I had set up a real lab in Barcelona and run the experiment there.
  • 9. ROBERT BLOOMFIELD: And so that would definitely save you a lot of money in travel expenses, doing it in a Virtual World to-- THOMAS CHESNEY: Exactly. Yes, exactly. ROBERT BLOOMFIELD: --reach a global community. Now the other way to think about doing these things on the cheap is that you can actually pay people less. So you paid people roughly the same amount in--well, since Linden Lab maintains the Linden Exchange steady to the U.S. dollar, we’ll keep this in dollars. And so you mentioned that you chose about ten dollars to be split because that’s what is done typically in lab studies. Now our second guest, Steve Atlas, has actually used a much smaller amount of money. So first, Steve, it’s a little funny to say this, but, welcome to Metanomics. STEVE ATLAS: Thanks, Rob. It’s a pleasure to be here. ROBERT BLOOMFIELD: So I say that a little sheepishly since, in fact, for those who don’t know, Steve Atlas--Curric Vida--has been working with me on Metanomics as he has been also working on his Masters thesis at Tufts University. So that welcome is a little tongue-in-cheek. Now you also ran, well, a slightly different type of experiment, but it’s still behavioral game theory. Can you quickly describe the type of game that you had people play, and also maybe emphasize a little bit more of how you conducted your experiment? Because I know
  • 10. you saved a huge amount of money, which is important for a Masters student. STEVE ATLAS: Yes. I replicated the classic Berg, Dickhaut and McCabe 1995 study on trust and reciprocity on a graduate student budget, where I had offered subjects a much smaller than usual amount to split between them. The way this experiment works is that subject A gets offered a certain amount of money. In this case, it was a hundred Lindens or around 35 cents. Subject A was offered to send a certain amount of that money to the other subject, who they did not know, who was anonymous and [AUDIO GLITCH] the experiment much later. And that amount that they sent was triple, and then recipient B had the opportunity to give some amount of money back to the initial participant who had given them the money. Now neoclassic economic theory will predict that knowing that player B is going to send nothing back to player A because they have no incentive to take that loss, then player A isn’t going to choose to trust and to send any money to player B in the first place. However, as was shown in the original lab study, that isn’t the case. Player A chooses to trust sending, on average, five dollars and 16 cents of the ten-dollar endowment, and player B chooses to reciprocate, sending around four dollars and change back. ROBERT BLOOMFIELD: Okay. I’m a little confused by the numbers. I thought you were starting out with much smaller numbers. STEVE ATLAS: I’m sorry to interrupt there. So basically what I was talking about in the splitting of the ten dollars was the classic Berg, Dickhaut and McCabe study. What I did in this experiment was to provide only a hundred Lindens in contrast, and this allowed, through providing a smaller amount of money per data point, as well as having an automated data
  • 11. collection system, we were able to collect a massive amount more data. Specifically, it was nearly 1,500 observations. ROBERT BLOOMFIELD: Okay. Actually let’s just run through this experiment ourselves really quickly to make sure that everyone is seeing what we’re talking about. So let’s start with I get a hundred Lindens. So the experimenter gives me a hundred Lindens. I can do whatever I want with it. I could actually put it in my pocket, and I’m done, and I have a hundred Lindens. But, in fact, I might decide I’m going to give some to you, and, if I do that, they get increased, and then you decide how much you’re going to give to me. So let’s say I give you 50. Now what’s your decision? STEVE ATLAS: Then that 50 that the original person gave gets tripled, and the other person decides how much of that tripled, now 150 Lindens--how much of that to give back to the original player. ROBERT BLOOMFIELD: Okay. So what are you going to give me? STEVE ATLAS: I will give you 51 since I’m so generous and want to ensure that you’re better off than you had been. ROBERT BLOOMFIELD: Okay. So unlike the Dictator and Ultimatum games that Thomas was describing, here we can actually make the pie bigger. The more I give you, the more wealthy we both are in combination. But then you basically, at the tail end of your game, you have a Dictator game. You can just keep it all, if you want, but, hopefully, for my sake
  • 12. anyway, you’d feel guilty. So the economics would say I don’t give you anything because I don’t believe you’re going to give me anything in return. But, in fact, what we usually see is trust and reciprocity. And I know, boy, hundreds--literally, right--hundreds of experiments of this trust and reciprocity game have been conducted in laboratories. In fact, even in small cultures where anthropologists have had to hack through brush and jungle to do this. I admit I’m forgetting the name of the guy from Emory University study. THOMAS CHESNEY: Was it Heinrich? ROBERT BLOOMFIELD: Yes. Yes, that’s right. It was Heinrich. So he actually has had a team of anthropologists doing this study around the world, and results are reasonably consistent. Now one of the interesting things about your study, Steve, is that you offered so little money. Now your results, how would you compare them to what we see in the literature? STEVE ATLAS: In general, the results were very consistent. They appeared slightly lower, but not in a statistically significant way. So in general, I would say that there was no statistically different outcome when they’re asked the same question online. I’m sorry. What is that? ROBERT BLOOMFIELD: Oh, go ahead. STEVE ATLAS: One of the interesting things that we were able to do, however, was to take a look at the effect of age on how people make this trust and reciprocity decision. We did
  • 13. find an interesting significant relationship between age and how people--or at least reported age, and what they were going to say to this trust question. We found that 18- to 22-year-olds, which would be the age group that would be typically tested in a normal trust experiment that would be done on undergraduates, actually, in fact, trusted less than the older population. So this supports the oft cited concerns that there is some degree of age-related bias by sampling undergraduates. What’s particularly interesting is that this difference was in the opposite direction of what would be expected. Typically undergraduates are seen as being naïve and might be trusting more, whereas, the sophisticated response is to not be putting any money into this trusting account. However, in this particular case, they were actually the ones who were trusting less than the older population, which was an interesting and surprising finding. ROBERT BLOOMFIELD: Now I’m seeing some questions coming in through the backchat, and one of them is from Joia Sands. She just says quite simply, “I don’t see why they think that behaviors would be different in a virtual environment versus Real Life.” So now Thomas mentioned some of the issues with face-to-face interaction, though I know a number of people have tried to deal with that in laboratory settings. Steve, why did you expect a difference in your study? STEVE ATLAS: Well, one of the differences is demographic in nature. The lab experiments you will often have [AUDIO GLITCH] college students tested there, unless it is specially designed to bring in others. College students are often used because of the convenience and easy access because they’re very controlled, in a sense, that you’re going to have of the same draw from the population. So they’re going stay about to the constant over time. In
  • 14. a Virtual World, it’s just demographic [AUDIO GLITCH] ROBERT BLOOMFIELD: Now, John Duffy, welcome to Metanomics. JOHN DUFFY: Hi. ROBERT BLOOMFIELD: Hi. One reason it’s great to have you on is that you actually participated in Steve’s study. You created an avatar. You went in and just acted exactly as a subject, and you wrote about it in your paper called Trust in Second Life. You expressed some concerns about the study and about experimental work in Virtual Worlds in general. You want to just tell us a little about that? JOHN DUFFY: Yes. I’m interested in doing experimentation in Virtual Worlds. And so one way to find out about it, I thought, was to go to some laboratories, and I visited several, and Steve’s was one of the experiments that I participated in. I think it’s an interesting place in which to do research. I have doubts that it will lead to any significantly different findings, and that’s, I think, what both Thomas and Steve are showing us. But I do have some concerns about the control that’s offered in Second Life relative to physical laboratories. ROBERT BLOOMFIELD: What are the key concerns that you would have? Go ahead. JOHN DUFFY: Well, I think there’s several issues. One is the recruitment of subjects, who are already expressing a preference for a leisure activity. I think opportunity costs here would be very high. Chesney, et al., for instance, say that the internet message boards
  • 15. didn’t generate much interest. They had to basically teleport willing subjects to their experiments just prior to the start time, and that leads to issues of association, lots of people may be in the same session. They may behave more nicely to one another because they know each other, these sorts of things. But another issue is the reliability of data collected may be quite low. This is especially true of demographic data such as data on gender, race, income and age, where it’s known that pretending to be someone other than you are is quite the norm on Second Life. The third issue is common knowledge, where experimenters seek to minimize any non-strategic uncertainty by publicly reading instructions at the start of an experiment. This not only informs subjects of the rules of play, but also makes the game structure public knowledge, the closest thing possible to inducing common knowledge in the laboratory. Having subjects read instructions anonymously, as I did in Steve Atlas’s trust game, doesn’t really inspire confidence that the players I’m matched with all had the same instructions, nor does it inspire confidence that there is no other player in the room with me in which to engage in trust and reciprocity. Another issue is collusion. There’s very little control over communication protocols. IM-ing is possible between subjects. People can be logged in with multiple avatars. And finally, I would say there’s demand effects, in the sense that you’re telling your choices to the experimenter--well it's mostly to this program that’s running the experiment, and I think that that changes the nature of interactions as was shown in the Chesney, et al., study.
  • 16. ROBERT BLOOMFIELD: Yeah. I see you’re getting some people who are agreeing with you. Venus Jervil is actually saying the verification of the data in general in Second Life is a problem and something we’ve talked about before on this show with some of our marketing research folks. Also, I do want to mention that, let’s see, CeAire Decosta is asking, “Why are they calling the participants “players” and Second Life a game?” And I think this is probably a good time to point out that what the panel members are doing here, well, in particular, Thomas and Steve, is creating a game within Second Life. So rather than saying Second Life is itself a game, you can think of this a little more like creating some type of game within Second Life that people can play, and then they become players just like the giant snails racing on SLCN’s giant snail races. You can call them the players or the racers. So hopefully that will clear that up. Now I was going to ask Steve if he would respond to those comments, but I see he has taken a little trip, this trip that Second Life seems to be sending lots of us on the last few days, and, hopefully, he will be back momentarily. Until we get there, Thomas, what’s your reaction to John’s concerns about conducting experiments in Second Life? THOMAS CHESNEY: Well, certainly, they all have validity, without question. We did take steps to try and minimize these problems. For instance, at the start of all of our games, everybody was given a note card, exactly the same with instructions on it. And then there were questions at the end to make sure that they had understood those instructions, and the game didn’t proceed until we had correct answers from everybody.
  • 17. ROBERT BLOOMFIELD: If I can stop you on that one. John, correct me if I’m getting your point wrong, but I think John’s concern was not that people would misunderstand the instructions, but that, in theory, one person could be getting one set of instructions, another person could be getting a very different set of instruction. And so typically in a laboratory setting you would actually have someone stand up in front of the whole room and read something aloud, and everyone can see everyone hearing the same thing. Is there a way? JOHN DUFFY: That’s the issue. ROBERT BLOOMFIELD: Is there a way, really, to deal with this in a Virtual World or, for that matter, any online environment? THOMAS CHESNEY: Well, for instance, we could use voice like we’re doing now, and somebody could read the instructions out to everybody so that everybody knows they are getting the same experience. No, we didn’t want to do that because we didn’t want to introduce factors such as tone or voice that somebody could say might be suggestive, although that wouldn’t really matter for reading out the instructions. Or, alternatively, you could put the instructions up on a poster on the wall, and then everybody knows that they’ve read the same instructions. So perhaps there are ways around it, but, yes, that is an issue, and it was an issue with our games. We could have been giving out different instructions to different people, and they wouldn’t have known it. We did take steps to minimize collusion. We programmed some lab chairs and, as soon as you sat in them, it would alert us whether or not you had IM’d somebody. So we couldn’t
  • 18. prevent collusion, but we could detect it. And the point about lies, yeah, absolutely. The demographics that we collected--the people could have been lying to us. Now we should point out that that could happen in a real lab as well. Yes, you might have more control over it in a real lab, but still you could be lied to. So yeah, it’s not a perfect substitute for a real lab, but there are advantages and disadvantages. ROBERT BLOOMFIELD: Oh, go ahead. THOMAS CHESNEY: That’s me. I’m done. ROBERT BLOOMFIELD: Okay. Steve, are you back with us? STEVE ATLAS: Yes, I am back, and I did hear Professor Duffy’s-- ROBERT BLOOMFIELD: I’m not a hundred percent sure of this, but I believe that you were here long enough to hear John describe his concerns. And then right when I was going to ask you of your responses, you impolitely left. So would you like to react to any of them? STEVE ATLAS: Yes, actually I would. What an inconvenient time for a crash to take place. I did hear what Professor Duffy had to say, and I’ve been very interested to hear his critique of my work so far. Ultimately, there are special considerations and concerns that come into play when conducting experiments online. One of the things in particular that got pointed out were concerns about collusion and about subjects knowing who they were partnered with. But one of the ways that we had control for this in our experiment was by having a delay mechanism in play, in the chair, so that subjects really had no way of knowing who they
  • 19. were going to be partnered with. The way that we did it is that we pulled several player A’s in a row and then pulled player B’s later on so that players didn’t know who they were actually going to be partnered with, and there was anonymity preserved there. Whether or not subjects believed they knew who they were partnered with falsely is a different concern, a different issue. And the reality is that there are a lot of other concerns that are at play when analyzing this type of research, but it’s also important to note that these methods are still in their nascent stages. So I think the solution is to try to identify exactly what the problems with initial studies are and try to improve the infrastructure so that we can collect better data as time goes on. I think that there are certain unique benefits of conducting research in Virtual Worlds, that aren’t necessarily lost even if there are some data quality and control considerations. At the very least, we know that Virtual Worlds provide the ability to run pilot studies in a much more cost-effective way before spending a significant amount, more resources to do it in a real lab at a higher cost. Secondly, it would allow access to conducting experiments to individuals who might not otherwise be able to get the significant amount of funding necessary to [AUDIO GLITCH] physical lab. ROBERT BLOOMFIELD: Now about how much in total did this thesis research cost you? STEVE ATLAS: In total, including the false start of creating lab infrastructure that was supposed to work in World of Warcraft that ultimately didn’t pan out, even with a couple of false starts, this was all conducted in under a thousand dollars. And this is about 1,500 data points that were collected under that budget.
  • 20. ROBERT BLOOMFIELD: Yes. And so if we were to think about the numbers more consistent--well, it seems like the primary savings that you had was probably with the size of the stakes. So you had 750 groups? Is that it or pairs of subjects? STEVE ATLAS: Yes, that’s correct. ROBERT BLOOMFIELD: So at ten dollars a pair even, all of a sudden you’re talking $7,500 so that’s quite a significant savings. It does seem like it would be nice if we could figure out how to deal with John’s concerns while keeping the money-saving aspects here. One of the big concerns that has come up that I guess we haven’t really talked about too much is, I guess, what you might call experiment farming. So if someone really wanted to try to exploit the fact that you’re giving away free Lindens, they could either be creating alts or just telling their friends to do it and don’t pay attention to what you’re doing. Just click around and get some money and go away. Games deal with that problem all the time. Do you have any specific thoughts, Steve, on how you can avoid that sort of experiment farming? STEVE ATLAS: What’s really interesting is that, in these Virtual Worlds, participating in an experiment can actually be a relatively high paying job relative to what other options are out there. Why exactly participants are willing to accept such low stakes for their time clearly has something to do with the fact that they’re getting some enjoyment out of use of experimenting something new and different. But the reality is that, if you’re either offering
  • 21. even a small amount of money for very little time or a large amount of money for a lot of time, that’s such an attractive offer that, even if you insert controls as we did that prevented more than--basically that player to participate under the same avatar more than once, what users were doing since they’re very savvy is that they were logging out and then logging back in under the guise of a different avatar and participating again. So that can result in another source of bias and problems with the data that we wound up collecting. Ultimately, it’s worse with the more you pay and the less you’re asking in return. So I think there have got to be some ways of getting around that kind of thing, and I’m not sure if they’ve been discovered to their full extent yet. JOHN DUFFY: Can I jump in and say something? ROBERT BLOOMFIELD: Oh, yes. John, please. JOHN DUFFY: I think as a possible solution is to get away from these two-person games that have been studied in the laboratory where there are issues of collusion and communication and move towards larger-scale games that really take advantage of the scale of Virtual Worlds like Second Life. And here what I have in mind are experiments involving asset pricing or voting where we really can’t do justice to the environments we’re hoping to study theoretically or in the laboratory, without some input from the decisions of many hundreds, maybe thousands, of participants. And in those settings, you would be much less concerned about issues of collusion communication because the payoffs from those activities would be much reduced.
  • 22. ROBERT BLOOMFIELD: Yeah. I think that you get at a point that I think is even larger than just making collusion harder, which is that given that both Thomas and Steve are conducting experiments to replicate what was done in a small laboratory naturally these are experiments that are well suited to the laboratory. And so it does seem to me that, while it’s nice to save a little money, that’s probably not the most important thing that we can be doing when we come to Virtual Worlds to conduct research. And especially, John, when I think of the types of research that you do typically. You’ve been studying macroeconomics in a very small laboratory setting, and, boy! maybe Virtual Worlds give you the opportunity to study macroeconomics with very large groups of people that are really going to bring into play some of the forces that you just can’t get at in these small groups. So let me ask John: Since you have so many different research streams, is ultimately your interest to take macroecon into Virtual Worlds? JOHN DUFFY: Well, I do think that what Steve and Thomas are doing is very useful. It will pave the way for doing larger-scale experimentation if there is a consensus that there is very little difference in the behavior of Second Life participants in two-player games and those of participants in Real World laboratory studies. So I do feel that it’s a necessary step and that it’s good that they’re finding this. But I do think that the capabilities of Second Life have not even begun to be explored. Or other Virtual Worlds as well. And, yes, I am interested in studying asset pricing. Voting is an issue where you can’t study issues of turnout in electorate sizes that approximate those of real elections in the laboratory. But let me give you another instance where I think Second Life could be very useful, and that is in experiments where subjects are asked to choose various effort levels. And the way
  • 23. we typically do this in the laboratory is, you click a button according to effort level one through seven or something, and there’s a cost associated with each. But essentially it’s effortless effort. ROBERT BLOOMFIELD: So instead of actually seeing how hard it is, how much effort it takes to do something, we say, “Are you willing to pay a dollar?” JOHN DUFFY: That’s right. Yeah. You could actually design experiments where effort involved some task that had to be performed, and more effort meant just that the task was more onerous or took more time. You could really think about real time too. In the laboratory, we’re always concerned about how long we could keep subjects. That’s an issue on Second Life as well, but, if you structure the experiment correctly, you could induce a time or an onerous burden associated with higher effort levels, and that’s something where you could really take advantage of the environment better. ROBERT BLOOMFIELD: When you mention effort to me, and here we are in Virtual Worlds, it makes me think of what is often called the grind in game worlds, like World of Warcraft where, I think, everyone who plays these games knows that there’s a tremendous amount of effort that you have to go through in order to level up. Now, Steve, you mentioned that you had a false start. You were going to run these experiments in World of Warcraft, which actually seems like you would have lots of opportunities to conduct studies along the lines of what we’re describing. Why didn’t that work out?
  • 24. STEVE ATLAS: Well, one of the reasons why I had initially gone into that was because basically we were going to exploit the black market premium, and the hope was that we could get even more--essentially we could buy a ton of World of Warcraft gold for a relatively low cost and then compensate participants in what felt like an awful lot of money. However, in doing so, in exchanging that money through black market channels, since there is no official currency exchange mechanism between currency in World of Warcraft and U.S. dollars, in order to do that we placed all of our participants at risk of having their accounts terminated. Consequently, that did not go over well with the IRB at Tufts and had to change venues, in order to do it in a space that would not place participants at risk. ROBERT BLOOMFIELD: Actually I’m glad you brought up the IRB. So for those who don’t conduct research that is either taking place at universities in the U.S. or funded by federal bodies you may not know, the IRB is the Internal Review Board, an organization we have to make sure that research is being conducted in line with standards of safety and disclosure. And so actually let me ask you, Steve, did you have any unusual IRB issues in getting approval for this research at Tufts? STEVE ATLAS: Other than having to switch venues, that was most significant one. I think their concern about that was actually quite valid. I don’t think that that means that it would have been totally impossible to use that space. We just would have to do it in cahoots with the game developers [AUDIO GLITCH] ROBERT BLOOMFIELD: I see Allister Beerbaum is asking whether you approached Blizzard, the maker of World of Warcraft, directly, to see if they would help you out. Did you
  • 25. try that? STEVE ATLAS: Yes. I wrote a half dozen emails to them, didn’t get any responses. ROBERT BLOOMFIELD: Thomas, I’m less familiar with how you need to conduct research and get approvals in the UK. Can you just tell us a little bit about any unusual challenges you faced doing that with a Virtual Worlds study? THOMAS CHESNEY: Certainly. We didn’t have any problems at all. The game theory research project didn’t have to go through an ethics committee at all. We did another project, and there was a psychologist on the team there, and he had to put the project through their ethics committee, but, no, we don’t have an equivalent to your review board. ROBERT BLOOMFIELD: Okay. Well, actually, I read in your paper, Thomas, you did mention there was just a little throwaway line regarding Linden Lab’s guidelines for researchers. THOMAS CHESNEY: Yes, that’s right. ROBERT BLOOMFIELD: So can you talk a little about those? THOMAS CHESNEY: Certainly. In actual fact, Linden don’t require this anymore, but it used to be, when we started the project, Linden required all researchers to submit proposals to them and get approval from them before we could do anything. So yes, you’re right, we
  • 26. did do that, but you don’t need to now. If you were starting the project tomorrow, you wouldn’t have to do that. ROBERT BLOOMFIELD: Okay. And another question. And let’s see, this was from Joia Sands: Were participants participating blind in this study? Or did they know what the study was about, that it was an economics study? And I guess, why don’t we start with you, Tom? THOMAS CHESNEY: With me? Sorry. Did you say? ROBERT BLOOMFIELD: Yes. THOMAS CHESNEY: We told them that it was a university project, and it was looking at behavior. So we invited people to come along for an experiment, but we didn’t mention economics, and we didn’t tell them what would be involved other than it was a decision-making task and that they could make some money, and we’d give them the min and the max amount that they could make. ROBERT BLOOMFIELD: And, Steve, [AUDIO GLITCH] you? STEVE ATLAS: [NO RESPONSE] ROBERT BLOOMFIELD: I think we may have lost Steve for a little bit there. Let’s see. I see that--actually we got a late start, and we are actually getting to the end of our time. An hour
  • 27. goes quickly in these sessions. Let’s see. I guess I would like to encourage right now if there are any more questions from the audience, and there have been quite a few. I’m just scrolling back to see how many of these I may have missed. I’ve probably missed a lot. So let’s see. And also, just from our guests, I’m wondering what you might like to mention at the end of our show that you haven’t had a chance to. John, shall we start with you and any observations that you’d like to make in closing? JOHN DUFFY: Just to reiterate what I mentioned before. I think that emphasizing that Second Life is a cheap place to do experiments is not the way to go. I think what needs to be said or discovered is to use Second Life to do a very different kind of experimentation than is done in the laboratory. And obviously, you need to build a bridge between laboratory and Virtual World experience [AUDIO GLITCH] at least to what other researchers are doing. But I think the real payoff will come when someone figures out how to make the best use of Virtual World environments for experimentation. I suggested a couple of ways, and I’m thinking about it. I’m very interested in issues of monetary policy, for instance, and how you would even go about doing this, I don’t know yet, but that’s something where you would use the entire scope of the environment. You would have to play around with the laws by which money is injected into the system, an epic kind of experimentation you certainly can’t do in a laboratory, not with much empirical validity. And you can’t do it in the Real World. So that’s what I’m looking forward to. ROBERT BLOOMFIELD: Steve? STEVE ATLAS: I would say that there’s one other scope of research that is very strongly
  • 28. related to conducting economic experiments, and that would be working with the game developers themselves possible to exogenously change certain macroeconomic variables in a controlled way. And, in doing that and then measuring results of that, we might be able to come up with evidence to either support or refute existing theories about how entire economies operate. And, in doing so, we could do it in a way that is much less costly to actual people, by inconveniencing the residents of Virtual Worlds with a small amount, it would be preferable in terms of welfare impacts than to say try other Third World countries and changing the economic conditions there. So that’s one other area that is related to what we’re talking about here where the types of tools that economists use could be particularly valuable to do on Virtual Worlds themselves rather than in the Real World. ROBERT BLOOMFIELD: Okay. And, Thomas, anything you’d like to say? THOMAS CHESNEY: Certainly. I think it’s fantastic that this discussion is going on and that we are considering these issues. And the future for this does actually look quite exciting. I mean, for instance, where we’re going with our experiments is, we are trying to combine two things that have been mutually exclusives before Virtual Worlds came along, which is the realism of a field experiment with the control of a lab experiment. So we are trying to use the environment as Professor Duffy suggests, and, yeah, I think things are exciting. ROBERT BLOOMFIELD: I have been studying this space for, I guess, over a year now, and I think it’s very exciting. I think it’s clearly more challenging than I thought it was going to be. But let me just echo some of the things that you guys have said in closing.
  • 29. Starting with John’s comment that the way to think about it is not that this is going to be easier or cheaper, and it’s sort of standard advice to doctoral students as they start presenting their research to faculty and as they respond to peer reviews, that whenever they’re asked why they made a choice, they shouldn’t be saying, “Oh, well, that was easier,” because, frankly, no one cares how hard doctoral students have to work. In fact, the answer needs to be, “Because it was the better way of doing it.” And so the question that we need to be addressing is really: What is it that Virtual Worlds are the best thing for examining? And I think that it really does come down to when I think of the advantages of Virtual Worlds, what we have are lots and lots of people who are looking for things to do. So Steve is right that we don’t need to be paying Real World wages to people who are in a place like Second Life, looking for entertaining content. And as much as experimental economists will deny it, participating in economic experiments is a lot of fun. There’s a reason that they’re called games. It’s not just a technical theoretical term. They really do give people challenging decisions to make. And so--yes, and I agree, I see Venus is saying, “but you have provide a reward.” Absolutely. But experimental economists are also having to provide a lot of money just to get butts in the seats in their labs. And the butts are already here in Second Life looking for a good seat to sit down in and do something interesting. So the other thing that we’ve got in Virtual Worlds is very complicated institutions. And certainly in business, in macroeconomics, in financial economics, really these experiments are about how psychology and economic forces meet complicated institutions.
  • 30. Very quickly, let me describe a study that I am talking about doing with a doctoral student here at Cornell. The idea is that we’re going to create what we’re calling synthetic firms, little firms that people are working together to run. They’re going to be Cornell students who will work on those here at Cornell, actually running the firms. And, like so many businesses in Second Life, they will actually be issuing stock on a stock exchange in Second Life. Unlike the virtual businesses, they will be audited, and everything will be above board. Everything will be trustworthy. We hope to get thousands of people in Second Life to be trading the shares of these synthetic firms, and our hope is to manipulate the financial reporting mechanisms that people can [AUDIO GLITCH]. So the people running the firms will be using different methods of financial reporting to explain to the investors how they’re performing, what they’re doing, what the risks are, and we’ll be able to see, in a much better way than we can in a laboratory, exactly how it is that financial reporting standards affect firms’ ability to communicate with the stock market and raise capital. So sorry I went on at such length in that, but I just wanted to get across the idea that I think this is one way to use the complex rich environment that Second Life allows, in order to look at issues that you just couldn’t do with a small number of people in a small lab for a short amount of time. And, in answer to CeAire Decosta’s question, “Didn’t Linden Lab already shut that type of thing down?” our plan is to make sure that this is--and these I think are challenges everyone is going to have to worry about, as they do economic studies in Virtual Worlds--they’re going to have to make sure it’s not gambling. They’re going to have to make sure that it is not inappropriately unregulated financial services. So we may be going to the SEC, for example,
  • 31. looking for a letter of no action, as the Iowa electronic market did in doing very similar research. So there are ways to do this. I think it’s going to be a very exciting and very challenging decade for experimental economists as they figure out how to use Virtual Worlds to do the type of research that I think really represents the next generation of experimental economics. So I think I will answer Pandora Bailey’s question of whether share trading is gambling, which it is not. I will answer that after the session. But, for now, let me just thank everyone for coming to Metanomics. I’d like to thank our guests Thomas Chesney-- THOMAS CHESNEY: Thank you. Thank you very much. ROBERT BLOOMFIELD: --Steve Atlas and John Duffy. I hope we will all have a chance to talk and figure out how to move this field forward. So bye bye, everyone, and see you all next Monday. JOHN DUFFY: Thanks. STEVE ATLAS: Thanks a lot. Document: cor1015.doc
  • 32. Transcribed by: http://www.hiredhand.com Second Life Avatar: Transcriptionist Writer