November 11th 2009 until February 1st 2010, 14G#Tcot: top conversation on twitter#mm: music monday
D. Zhao and M. B. Rosson. How and why people twitter: the role thatmicro-blogging plays in informal communication at work. In Proceedings of theACM 2009 international conference on Supporting group work. ACM, 2009.C. Wilson, B. Boe, A. Sala, K. P. Puttaswamy, and B. Y. Zhao. User interactionsin social networks and their implications. In Proc. of the 4th ACM Europeanconference on Computer systems. ACM, 2009.J. Weng, E.-P. Lim, J. Jiang, and Q. He. Twitterrank: finding topic-sensitiveinfluential twitterers. In Proc. of the third ACM international conference on Websearch and data mining. ACM, 2010.
Asur, S., and Huberman, B. A. (… )Predicting the Future with Social Mediahttp://www.hpl.hp.com/research/scl/papers/socialmedia/socialmedia.pdfBoyd D. Golder S., and Lotan G. (2010 ) Tweet, Tweet, Retweet: Conversational aspects of retweeting on Twitter. HICSS-43. IEEE: Kauai, HI, January 6.Honeycutt C. and Herring S.C. (2009) Beyond Microblogging: conversation and collaboration via twitter. Proceedings of the forty-second Hawai’I International conference system sciences (HICSS-42) Los Alamitos, CA: IEEE pressFocal Point (game theory) http://en.wikipedia.org/wiki/Focal_point_%28game_theory%29Steels L., and Kaplan F. (1999) Collective learning and semiotic dynamics. In D. Floreano and J.-D. Nicoud and F. Mondada, editors, Advances in Artificial Life: 5th European conference (ECAL 99), Lecture Notes in Artificial Intelligence, 1674, pp. 679-688, Berlin. J. Ke, J.W. Minett, A. Ching-Pong, W.S.-Y,Wang, Selforganization and selection in the emergence of vocabulary, Complexity 7, 41-54 (2002).
The network: simply the network of your followers/followings. Those are the people whose updates you might be reading and who might be receiving your updates. This is the reach of your Twitter stream.The FOAF-network: the network of your followers/follwing’s networks. Those are the people you could potentially reach via retweeting messages. This is the extended reach of your Twitter stream.Asur, S., and Huberman, B. A. (… )Predicting the Future with Social Mediahttp://www.hpl.hp.com/research/scl/papers/socialmedia/socialmedia.pdfFriends and Followers are not the real network@mention and RT indicate a closer network@-conversation: within a one-hour period indicated that about 31% of tweets with @ received a response. (Honeycutt and Herring, 2009).
Physicists in Germany claim to have developed a new computer model that can describe how human languages evolve over time. Dietrich Stauffer and Christian Schulze of Cologne University have taken techniques used by biologists to describe evolution and applied them to the rise and fall of languages. In particular they find that the size distribution of languages - a measure of the relative popularity of different languages - can be described by a nearly "log-normal" curve (arXiv.org/abs/cond-mat/0411162).All languages change over time, with some languages disappearing because they are not spoken by enough people. Stauffer and Schulze describe a particular language by a string of 8 or 16 bits, where each bit can equal 0 or 1, and start their simulations with one person speaking language zero (all bits equal to zero). Two languages are different from each other if they differ by at least one bit. The model works as follows: After a given time, this person produces one offspring who speaks a language that might differ from that spoken by their parent by one bit: the possibility of such a mutation occurring is governed by a probability p. The model also allows for the possibility of a person dying during any iteration: this is governed by a factor called the "carrying capacity" in biology. Lastly, it is also possible that the parent decides to start speaking a different language: this is determined by several factors including the carrying capacity and the fraction of the population who already speak that language. The Cologne physicists found that, for a sample of 10 million people, high mutation rates are needed to ensure that no single language dominates. This finding agrees with data on real languages, as does the prediction that the size distribution of languages is close to a "log-normal" distribution (see figure). "Our model is more realistic than other similar models we know of since it allows for numerous languages, instead of only two," say Stauffer and Schulze. "In these models, only one language survived because it was assumed to be superior to the other. We, on the other hand, have regarded all languages as being equally fit." However, it remains to be seen how the work will be received in the linguistics community. "Linguistics is a relatively new topic for physics and complex systems theory and any tentative way to understand and quantify it is useful and welcome," says Marco Patriarca of the Helsinki University of Technology. "However, while the model Stauffer and Christian Schulze is interesting and worth investigating, it also seems preliminary."
Transcript of "Proposal defense"
Dissertation proposal defense<br />Xiaoju Zheng<br />June 9, 2010<br />Life Cycle of #hashtags: “words” in the #Twittertopia<br />1<br />
Road Map of the Presentation<br />Background of research questions<br />Research Questions<br />Overview of the data<br />Follow-up Experiments<br />Diffusion models<br />Challenges<br />2<br />
The Laws of Imitation <br />“why, given one hundred different innovations conceived of at the same time – innovations in the form of words, in mythological ideas, in industrial processes, etc. – ten will spread abroad, while ninety will be forgotten.”<br /> --- Gabriel Tarde (1903) “The Laws of Imitation”<br />Merit of its own? Or something else?<br />3<br />
The Laws of Imitation<br />“……..we see that the incessant struggle between minor linguistic inventions which always ends in the imitation of one of them, and in the abortion of the others, finally comes to transform a language in such a way as to adapt it, more or less rapidly and completely, according to the spirit of the community, to external realities and to the social purposes of language. ..”<br /> --- Gabriel Tarde (1903) “The Laws of Imitation”<br />4<br />
The Laws of Imitation<br />Merit is not the only catalyst of the spread of an idea.<br />In situations where “the poorest innovations, from the point of view of logic, are selected because of their place, or even date of birth.”, Tarde attributes these irrational occurrences to “extra-logical influences”<br />5<br />
Social Network Analysis<br />Current research in social network analysis asserts that these “extra-logical” influences can be explained by examining the dynamics of the network through which influence is transmitted between individuals. <br />In other words, if we view individuals as nodes in a social network, where a directed edge indicates that one node influences another, then some graph configurations make it more likely that an innovation will be widely adopted than others.<br />6<br />
Basic Research Questions<br />How is a word created?<br />What makes a newly-created word better than others?<br />How is a newly-created word picked up by users at large?<br />How does a word gain popularity among the population?<br />In a word, the life cycle of a word.<br />7<br />
Research Question --- Data<br />Word creation in EnglishIn spoken English, it can take decades – even centuries – for new words to emerge, become part of common parlance, and then fade into disuse.<br />Word creation on Twitter,a word in the form of #hashtags can live the entire lifecycle in very short period of time, e.g. a couple of days<br />A news story breaks, and competing hashtags vie for dominance. Then a few influential people adopt the same one. Suddenly the conversation coalesces around it, the term trends, the spammers start using it, and then the conversation peters out as we move on to the next topic. (only one possibility)<br />Is that the pattern? And how closely does it map onto the ways that words and phrases emerge in spoken language?<br />#hashtag – word on twitter<br />8<br />
Twitter<br />Twitter.com:Twitter is a social networking and micro-blogging service that enables its users to send and read messages known as tweets. Tweets are text-based posts of up to 140 characters displayed on the author's profile page and delivered to the author's subscribers who are known as followers. <br />9<br />
Twitter: some conventions <br />10<br />@mentions - following word is the name of a twitter user and as such this tweet refers to that user, e.g. ”@dave thanks for the help” or ”Talking with @paul about twitter”. (can be used to spot smaller network)<br />Retweets -”RT” means ”I am retweeting (copying) something from elsewhere”, e.g. ”RT@john I just saw Madonna” means that I am retweeting theoriginal message from John (can be used to spot smaller network)<br />#hashtags –give contextual relevance to a tweet or identified as a keyword, e.g. ”Like this demo #acita09” or ”Why does #ms-word keep crashing”<br />
Top Influential People on Twitter the Edinburgh Twitter Corpus (around 2 billion tokens) <br />11<br />Six Singers<br />
Top 10 trending topic from the Edinburgh Twitter Corpus (around 2 billion tokens) <br />12<br />
Research Question<br />Word creation and its propsperitywhat count as criteria for a newly-coined “word” to be accepted as a good #hashtag and how a good #hashtag gain popularity among groups of people. <br />Logical: linguistic groundings of a good #hashtagLinguistic analysis of the #hashtags and behavioral studies <br />Extra-logical: social groundings of a popular #hashtage.g. network structure and dynamics<br />14<br />
I. Linguistic Analysis of #hashtagsmore at https://docs.google.com/Doc?docid=0AWbvIzcQLhQXZGdoY256cDJfMTExZjZjbjNxbTM&hl=en<br />Linking words into a sentence: e.g. whatsyourbackground, tweetwhatyoueatLetsMakeATrendingTopic,goodluckjustin<br />Part of word + existing word: e.g. animtip,appstore<br />Compounding: noun + noun e.g. sundayhug, pubquiz, waikikilunch<br />Compounding: verb + noun e.g. hashtagme, pickon,killcapscop<br />Compounding: adv + verb e.g. currentlycrushing<br />Compounding: adj + noun e.g. digitalbritan, GoodTimes, morningsickness<br />Splinter: e.g. socialmem (SocialCamp Memphis), <br />Acronym & Initials: e.g. smlb (St Michael Le Belfrey church), emr (electronic medical records), #eu (European Union), #cah (Crimes against humanity)<br />Neologism (splinter involved): e.g. twacker(twitter users who lose user account), tweetie (Twitter client for Mac and iPhone), twitvorce(to divorce yourself from a Twitter member by unfollowing them), twittertopia, twendsetter<br />MISC: omgfact, #tcot (top conversation on twitter)<br />16<br />
Preliminary Analysis<br />Public timeline: 20 tweets per minute<br />20 days of non-stop crawling<br />Total tweets = 567,091<br />Total words = 8,495,323<br />Average words per tweet = 14.98<br />NPS Chat Corpus: 45010 tokens/6,066 types<br />Webtext corpus in NLTK: 396,736 tokens/21,537 types<br />17<br />
Twitter presents a different genre of texts<br />Self expression: "I" is the top-ranking word that tweets begin with.<br />Stats update: "Watching", "trying", "listening", "reading" and "eating" are all in the Top 100 first words, revealing just how often people use Twitter to report on whatever they are experiencing at the time.<br />News broadcast: The abbreviation "RT" (retweet) is extremely common<br />21<br />
Twitter presents a different genre of text<br />popular web addresses (e.g. URL shortening service) among the top 500: "tinyurl.com", "twitpic.com", "ff.im", "twurl.nl". These all appear because they offer services useful to twitterers. <br />Tech vocabulary: among top 500: "Google”, “Faceobok”:, “internet”, “website”, “blog”, “Mac”, and “app”. <br />popular web addresses (e.g. URL shortening service) among the top 500: "tinyurl.com", "twitpic.com", "ff.im", "twurl.nl". These all appear because they offer services useful to twitterers. <br />Tech vocabulary: among top 500: "Google”, “Faceobok”:, “internet”, “website”, “blog”, “Mac”, and “app”. <br />22<br />
Research Question<br />23<br />linguistic groundings of a good #hashtagLinguistic analysis of the #hashtags and behavioral studies <br />social groundings of a popular #hashtage.g. network structure and dynamics<br />Would linguistically equally good #hashtags have different degrees of popularity? Is it because of the different network structure?<br />Behavioral studies to get quantitative measurement about linguistic goodness of #hashtags.<br />
Linguistic Grounding<br />24<br />Question 1: Does the tag length distribution of adopted #hashtag demonstrate a different distribution from words? Does it conform to a power law distribution or a lognormal distribution? Do #hashtags of different length receive different goodness judgement (e.g. are extremely short tags better than extremely short words?)<br />
Linguistic Grounding<br />25<br />Question 2: What are the linguistics processes of creating a #hashtag? What count as a good #hashtag (morphologically, phonotactically, and semantically)?<br />A more qualitative analysis of the #hashtags needs to be done to design a metrics of analysis: e.g. compounding, splinter (of what kind)<br />
Linguistic Grounding – behavioral experiements<br />26<br />Word vs. Nonword: Subjects will be presented with #hashtags collected from twitter.com, and asked to label them as either word or nonword.<br />Come up with specific criterion for word vs. nonword<br />Morpheme identification: based on the results obtained from the Word vs. Nonword experiment, #hashtags will be presented for subjects to divide them into morphemes and identify meaningful subparts.<br />
Linguistic Grounding – behavioral experiements<br />27<br />Semantic transparency:word association game: for hashtags like “twitvorce”, subjects will be asked to provide free word associations. For instance, subjects are likely to provide “twitter” and “divorce” for the “twitvorce”.<br />
28<br />Goodness rating: general: for both #hashtags, that are “nonwords”, subjects will provide subjective goodness ratings, e.g. on a scale from 1 to 7. phonotactic: subjects rate the pronouncability, e.g. for acronyms and initials.For instance, some acronyms are just strings of consonants without vowels, some are strings of vowels, and others are mixture of consonants and vowels. Would more pronouncable #hashtags be perceived as better #hashtags?<br />
Linguistic Grounding – statistical parser<br />29<br />Phonotactic likelihood: <br />Develop a statistical parser (e.g. finite state machine) for #hashtags and words, and compare the phonotactic probability. Also compare the statistical parser with e.g. Vitevich (2004) model. <br />
Social Grounding<br />30<br />Based on the realistic data from twitter, diffusion models can be tested.<br />Diffusion models:Linear Threshold ModelCascade Model<br />
The Threshold model<br />Threshold Model.It says that people adopt a new behavior because a sufficiently large proportion of their friends have adopted that behavior. E.g. Early adopters have a very low threshold, say 5% or 10%, while late adopters would have a much higher threshold. Every person, however, has their own individual threshold.<br />The key variable here is the initial distribution of thresholds across a social network, which describes in totality the final extent of the behavior.<br />But this model says nothing about how people initially adopt behavior. That is, it says nothing about innovators or the things that are being invented, only about the spread of innovation through a social network.<br />31<br />
The Threshold model<br />32<br />In the threshold model every person u has a threshold :and each of their neighbors v is weighted according to: W u,v.If then the person u adopts the behavior. <br /> The set of thresholds, weights, and initial adopters determines the extent of the behavior in the social network. <br />
The Cascade Model<br />Cascade Modelevery person has a chance of adopting a new behavior whenever one of their neighbors adopts it.<br />The probability that a person adopts the new behavior is the conversion rate for the notification.<br />This probability is both a function of the sender and the recipient, so more influential people are more likely to convince others to adopt a behavior. <br />33<br />
The Cascade Model<br />34<br />In the cascade model, for every person u and neighbor v there is a random variable X u,v<br /> which describes the likelihood of u adopting the behavior if v has adopted it. <br />
Diffusion Model<br />35<br />Threshold model: neighborhood densityadopt if enough friends do so.<br /> Cascade Model: function of the sender and receiverpeople have a chance of doing something if one of their friends is doing it.<br />
Several Challenges at this step<br />36<br />Design a metrics for #hashtag classification ( e.g. p. 16): position of #hashtag, functions, word structure.<br />Different #hashtag may have different adoption patterns and diffusion patterns.<br />Quantitative measurement of “success” of a #hashtag: by frequency of mentioning, logevity (within a short or long time frame)<br />Design a way to find competing, equally good #hashtags<br />Representative sample<br />
Twitter Network: spot the right network<br />Despite having large networks, a smaller circle is maintained: for users with a high number of followers, they actually only still communicate with a smaller subset of users. Where’s the value? Within the hidden network: find out the true influence model of who people really trust above all other users by looking at actual “@” behavior and follow behavior. <br />37<br />