The Rise of Social Bots

1. XX The Rise of Social Bots EMILIO FERRARA, Indiana University ONUR VAROL, Indiana University CLAYTON DAVIS, Indiana University FILIPPO MENCZER, Indiana University ALESSANDRO FLAMMINI, Indiana University The Turing test asked whether one could recognize the behavior of a human from that of a computer algorithm. Today this question has suddenly become very relevant in the context of social media, where text constraints limit the expressive power of humans, and real incentives abound to develop human-mimicking software agents called social bots. These elusive entities wildly populate social media ecosystems, often go- ing unnoticed among the population of real people. Bots can be benign or harmful, aiming at persuading, smearing, or deceiving. Here we discuss the characteristics of modern, sophisticated social bots, and how their presence can endanger online ecosystems and our society. We then discuss current efforts aimed at detection of social bots in Twitter. Characteristics related to content, network, sentiment, and temporal patterns of activity are imitated by bots but at the same time can help discriminate synthetic behaviors from human ones, yielding signatures of engineered social tampering. The rise of the machines Bots (short for software robots) have been around since the early days of computers: one compelling example is that of chatbots, algorithms designed to hold a conversation with a human, as envisioned by Alan Turing in the 1950s [Turing 1950]. The dream of designing a computer algorithm that passes the Turing test has driven ar- tiﬁcial intelligence research for decades, as witnessed by initiatives like the Loebner Prize, awarding progress in natural language processing.1 Many things have changed since the early days of AI, when bots like Joseph Weizenbaum’s ELIZA [Weizenbaum 1966] —that employed a Rogerian approach to fool the interlocutor— were developed as proofs-of-concept or for delight. Today, social media ecosystems populated by hundreds of millions of individuals present real incentives —including economic and political ones— to design algorithms that exhibit human-like behavior. Such ecosystems also raise the bar of the challenge, as they introduce new dimensions to emulate in addition to content, including the social network, temporal activity, diffusion patterns and sentiment expression. A social bot is a computer algorithm that automatically produces content and interacts with humans on social media, trying to emulate and possibly alter their behavior. Social bots have been circulating on social media platforms for a few years [Lee et al. 2011; Boshmaf et al. 2011]. 1www.loebner.net/Prizef/loebner-prize.html This work is supported by the National Science Foundation (grant CCF-1101743), by DARPA (grant W911NF-12-1-0037), and by the James McDonnell Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Corresponding author: E. Ferrara (ferrarae@indiana.edu) Author’s addresses: E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. Center for Complex Networks and System Research. School of Informatics and Computing. Indiana University Bloomington. 919 E. 10th Street, Bloomington (IN) 47408 - USA arXiv:1407.5225v1[cs.SI]19Jul2014

2. XX:2 E. Ferrara et al. Engineered social tampering What are the intentions of social bots? Some of them are benign and, in principle, in- nocuous or even helpful: this category includes, for example, social bots that automatically aggregate content from various sources (like simple news feeds), and automatic responders to inquiries, increasingly adopted by brands and companies for customer care. Although this type of bots are designed to provide a useful service, they can sometimes be harmful, for example when they contribute to the spread of unverified information or rumors. A notable example is that of the Boston marathon bombing [Cassa et al. 2013], when false accusations spread widely on Twitter mostly due to bots automatically retweeting posts without verifying the facts or checking the credibility of the source. With every new technology comes abuse, and social media are no exception. A second category of social bots includes malicious entities designed specifically with the purpose to harm. These bots mislead, exploit, and manipulate social media discourse with rumors, spam, malware, misinformation, political astroturf, slander, or even just noise. This results in several levels of societal harm. For example, bots used in political astroturf artificially inflate support for a candidate; their activity can endanger democracy by influencing the outcome of elections. In fact, this might already have happened: during the 2010 U.S. midterm elections, a small set of social bots started a smearing campaign against one U.S. Senate candidate for Delaware, injecting thousands of tweets pointing to a pernicious website reporting unverified information, aimed at slander- ing this candidate and support his opponent [Ratkiewicz et al. 2011a]. A similar case was reported for the Massachusetts special election of 2010 [Metaxas and Mustafaraj 2012]. In the latter case, the target candidate lost the elections, even if it is impossible to determine whether and to what extent this could be attributed to these bots. So- phisticated bots can generate personas that are more credible as fake followers, and thus harder for simple filtering algorithms to detect. They make for valuable entities on the follower market: recent allegations regarding the acquisition of fake followers have involved Mitt Romney,2 Newt Gingrich,3 Barack Obama,4 and other exponents of the international political scene. More examples of the devastating effects of social bots are reported every day by journalists, analysts and researchers from all over the world. A recent orchestrated bot campaign successfully created the appearance of a sustained discussion about a tech company. Automatic trading algorithms, now trained to use Twitter signals to predict the stock market [Bollen et al. 2011], picked up this conversation and started trading heavily the company’s stocks. This resulted in a 200-fold increase in market value, bringing the company’s worth to 5 billion dollars.5 The analysts realized too late the orchestration behind this operation. This event, however, may not be unprece- dented: on May 6, 2010 a flash crash occurred in the U.S. stock market, when the Dow Jones plunged over 1,000 points (about 9%) within minutes —the biggest one-day point decline in history. Researchers speculated wildly about the causes of the disas- ter, and trading interference due to social bots could not be ruled out [Hwang et al. 2012]. A similar effect on the stock market was recorded when, on April 23, 2013, the Syrian Electronic Army hacked into the Associate Press Twitter account and posted a 2Fake Twitter accounts may be driving up Mitt Romney’s follower number — www.theguardian.com/world/ 2012/aug/09/fake-twitter-accounts-mitt-romney 3Update: Only 92% of Newt Gingrich’s Twitter Followers Are Fake — gawker.com/5826960/ update-only-92-of-newt-gingrichs-twitter-followers-are-fake 4Pols have a #fakefollower problem — www.politico.com/story/2014/06/twitter-politicians-107672.html 5The Curious Case of Cynk, an Abandoned Tech Company Now Worth $5 Billion — mashable.com/2014/07/ 10/cynk

3. The Rise of Social Bots XX:3 false rumor about a terror attack on the White House, in which President Obama was allegedly injured.6 The bot effect These anecdotes illustrate few harmful effects of social bots on our increasingly in- terconnected society. In addition to potentially endangering democracy, causing panic during emergencies, and affecting the stock market, social bots can harm our society in even subtler ways. A recent study demonstrated the vulnerability of social media users to a social botnet designed to expose private information, like phone numbers and addresses [Boshmaf et al. 2011]. This kind of vulnerability can be exploited by cybercrime and cause the erosion of trust in social media [Hwang et al. 2012]. Bots can also hinder the advancement of public policy, by creating the impression of a grassroots movement of contrarians [Ratkiewicz et al. 2011b], or polarize the political discussion driving attention only on specific, biased sources [Conover et al. 2011]. They can alter the perception of social media influence, artificially enlarging the audience of some people [Edwards et al. 2014], or they can ruin the reputation of a company, for com- mercial or political purposes [Messias et al. 2013]. A recent study demonstrated that emotions are contagious on social media [Kramer et al. 2014]: elusive bots acting unnoticed in a population of unaware humans could easily manipulate them and alter their perception of reality, with unpredictable results. Indirect effects of social bot activity include the alteration of social media analytics, adopted for various purposes such as TV ratings,7 expert finding [Wu et al. 2013], and scientific impact measurement.8 For all these reasons, we deem it crucial to design advanced methods to automatically detect social bots (or to discriminate between humans and bots). The research in this direction has just started: some groups are trying to reverse-engineer social bots to understand their functioning [Freitas et al. 2014], while others are creating bots themselves [Hwang et al. 2012; Briscoe et al. 2014] to study the susceptibility of people to their influence [Boshmaf et al. 2011; Wagner et al. 2012]. The strategies currently employed by social media services appear inadequate to contrast this phenomenon. Act like a human, think like a bot One of the greatest challenges of bot detection in social media is understanding what modern social bots can do. Early bots mainly performed one type of activity: posting content automatically. These bots were as naive as easy to spot by trivial detection strategies. In 2011, James Caverlee’s team at Texas A&M University implemented a honeypot trap that managed to detect thousands of social bots [Lee et al. 2011]. The idea was simple and effective: the team created a few Twitter accounts whose role was solely to create nonsensical tweets with gibberish content, in which no human would ever be interested. However, these accounts attracted many followers. Further inspection confirmed that the suspicious followers were indeed social bots trying to grow their social circles. In recent years Twitter bots have become increasingly sophisticated, making their detection more difficult. Our replication of Caverlee’s honeypot strategy —and more sophisticated variants— yielded only a handful of bots in 2013. The boundary between human-like and bot-like behavior is now fuzzier. For example, social bots can search 6Syrian hackers claim AP hack that tipped stock market by $136 billion. Is it terrorism? — www.washingtonpost.com/blogs/worldviews/wp/2013/04/23/ syrian-hackers-claim-ap-hack-that-tipped-stock-market-by-136-billion-is-it-terrorism/ 7Nielsen’s New Twitter TV Ratings Are a Total Scam. Here’s Why. — defamer.gawker.com/ nielsens-new-twitter-tv-ratings-are-a-total-scam-here-1442214842 8altmetrics: a manifesto — altmetrics.org/manifesto/

4. XX:4 E. Ferrara et al. Table I. Classes of features extracted by Bot or Not? Class Number of features Description Network 112 Network features capture various dimensions of information diffusion patterns. We build networks based on retweets, mentions, and hashtag co-occurrence, and extracts their statistical features. Exam- ples include degree distribution, clustering coefficient, and centrality measures. User 56 User features are based on Twitter meta-data related to an account, including language, geographic locations, and account creation time. Friends 208 Friend features include descriptive statistics relative to an account’s social contacts (followees), such as the median, moments, and entropy of the distributions of their number of followers, followees, posts, and so on. Timing 24 Timing features capture temporal patterns of content generation (tweets) and consumption (retweets); examples include the signal similarity to a Poisson process [Ghosh et al. 2011], the average time between two consecutive posts, and such. Content 411 Content features, are based on linguistic cues computed through natural language processing, especially part-of-speech tagging; examples include the frequency of verbs, nouns, and adverbs in the phrases produced by the account. Sentiment 339 Sentiment features are built using general-purpose and Twitter- specific sentiment analysis algorithms, including happiness, arousal- dominance-valence, and emotion scores [Golder and Macy 2011; Bollen et al. 2011]. the Web for information and media to fill their profiles, and post collected material at predetermined times, emulating the human temporal signature of content production and consumption —including circadian patterns of daily activity and temporal spikes of information generation [Golder and Macy 2011]. They can even engage in more complex types of interactions, such as entertaining conversations with other people, commenting on their posts, and answering their questions [Hwang et al. 2012]. Some bots specifically aim to achieve greater influence by gathering new followers and expanding their social circles; they can search the social network for popular and influential people and follow them or capture their attention by sending them inquiries, in the hope to be noticed [Aiello et al. 2012]. To acquire visibility, they can infiltrate popular discussions, generating topically-appropriate —and even potentially interesting— content, by identifying relevant keywords and searching online for information fitting that conversation [Freitas et al. 2014]. After the appropriate content is identified, the bots can automatically produce responses through natural language algorithms, possibly including references to media or links pointing to external resources. Other bots aim at tampering with the identities of legitimate people: some are identity thieves, adopting slight variants of real usernames, and stealing personal information such as pictures and links. Even more advanced mechanisms can be employed; some social bots are able to “clone” the behavior of legitimate people, by interacting with their friends and posting topically similar content with similar temporal patterns. Bot or Not? At the beginning of 2014, we embarked on the design of a social bot detection frame- work for Twitter, called Bot or Not?. The main idea was to identify several classes of features that help recognize and separate bot-like from human-like behavior. Previ- ous familiarity with Twitter bots [Ratkiewicz et al. 2011b] allowed us to isolate six classes of features, summarized in Table I. Overall, our system generates more than one thousand features used to learn human and bot prototypes. To classify an account as either social bot or human, the model must be trained with instances of both classes. Finding and labeling many examples of bots is challenging.

5. The Rise of Social Bots XX:5 Fig. 1. Classification performance of Bot or Not? for four different classifiers. The classification accuracy is computed by 10-fold cross validation and measured by the area under the receiver operating characteristic curve (AUROC). The best score, obtained by Random Forest, is 95%. As a proof of concept, we used the list of social bots identified by Caverlee’s team. We used the Twitter Search API to collect up to 200 of their most recent tweets and up to 100 of the most recent tweets mentioning them. This procedure yielded a dataset of 15 thousand manually verified social bot accounts and over 2.6 million tweets. Caver- lee’s list also contains legitimate (human) accounts. The same procedure resulted in a dataset of counterexamples with 16 thousand people and over 3 million tweets. We used this dataset to train the social bot detection model and benchmark its performance. Bot or Not? achieves very promising detection accuracy (see Fig. 1). Some feature classes, like the user meta-data, appear more revealing and they can be easily ex- plained (see Fig. 2). Note that such performance evaluation is based on Caverlee’s dataset from 2011; we are already aware of more recent social bots that cannot be re- liably detected. Bots are continuously changing and evolving. Further work is needed to identify newer annotated instances of social bots at scale. Borderline cases also exist, such as accounts (sometimes referred as cyborgs) that are a mix of humans and social bots (e.g., when humans lend accounts to bots), or hacked accounts [Zangerle and Specht 2014]: detecting these anomalies is currently impossible. To make the detection system broadly accessible, we developed a Web-based applica- tion that interfaces with the Twitter API and retrieves the most recent activity of any account, to make a determination of whether that account exhibits bot-like or human- like behavior. The Web interface, depicted in Fig. 3, allows one to inspect any active Twitter account. Data about that account and its contacts are collected and processed in real time. The classifier trained on all feature classes provides a likelihood score that the account is a social bot. For instance, in Fig. 3, we see that one of the authors is deemed less likely to be a bot compared to an algorithm developed by Sune Lehmann’s

6. XX:6 E. Ferrara et al. Fig. 2. Subset of user features that best discriminate social bots from humans. Bots retweet more than humans and have longer user names, while they produce fewer tweets, replies and mentions, and they are retweeted less than humans. Bot accounts also tend to be more recent. team.9 The system also presents disaggregate scores according to models trained on each feature class independently. Often, an account may be classified as a social bot according to some feature classes, but not according to others. This is due to the large heterogeneity of features exhibited by people —some may have bot-like features, for example their meta-data or friend information. In addition to the classification results, Bot or Not? provides a variety of visualizations that capture some insights about the features exploited by the system. Examples are displayed in Fig. 4. We invite the reader to explore these interactive visualizations directly at truthy.indiana.edu/botornot. Master of puppets If social bots are the puppets, additional efforts will have to be directed at finding the “masters.” Governments10 and other entities with sufficient resources11 have been alleged to use social bots to their advantage. Even assuming the availability of effective detection technologies, it will be crucial to reverse-engineer the observed social bot strategies: who they target, how they generate content, when they take action, and 9You are here because of a robot — sunelehmann.com/2013/12/04/youre-here-because-of-a-robot/ 10Russian Twitter political protests ‘swamped by spam’ — www.bbc.com/news/technology-16108876 11Fake Twitter accounts used to promote tar sands pipeline — http://www.theguardian.com/environment/ 2011/aug/05/fake-twitter-tar-sands-pipeline

7. The Rise of Social Bots XX:7 Fig. 3. Web interface of Bot or Not? (truthy.indiana.edu/botornot). The panels show the likelihood that the inspected accounts are social bots along with individual scores according to six features classes. Left: The Twitter account of one of the authors is identiﬁed as likely operated by a human. Right: A popular social bot is correctly assigned a high bot score. what topics they talk about. The ability to extrapolate such information will allow us to systematically infer who are the entities behind them. Tools like Bot or Not? help shed light on the intricate world of social bots. Yet many research questions remain open. For example, nobody knows exactly how many social bots populate social media, or what share of content can be attributed to bots —estimates vary wildly and we might have observed only the tip of the iceberg. Bot behaviors are already quite sophisticated: they can build realistic social networks and produce credible content with human-like temporal patterns. As we build better detection systems, we expect an arms race similar to that observed for spam in the past. Static training instances are an intrinsic limitation of supervised learning in such a scenario; machine learning techniques such as active learning might help respond to newer threats. The race will be over only when the increased cost of deception will no longer be justiﬁed due to the effectiveness of early detection. The future of social media ecosystems might already point in the direction of envi- ronments where machine-machine interaction is the norm, and humans will navigate a world populated mostly by bots. We believe there is a need for bots and humans to be able to recognize each other, to avoid bizarre, or even dangerous, situations based on false assumptions of human interlocutors.12 12That Time 2 Bots Were Talking, and Bank of America Butted In — www.theatlantic.com/technology/ archive/2014/07/that-time-2-bots-were-talking-and-bank-of-america-butted-in/374023/

8. XX:8 E. Ferrara et al. A E C B D Fig. 4. Visualizations provided by Bot or Not?. (A) Part-of-speech tag proportions. (B) Language distribution of contacts. (C) Network of co-occurring hashtags. (D) Emotion, happiness and arousal-dominance-valence sentiment scores. (E) Temporal patterns of content consumption and production. ACKNOWLEDGMENTS The authors are grateful to Mohsen JafariAsbagh, Prashant Shiralkar, Qiaozhu Mei, and Zhe Zhao.

9. The Rise of Social Bots XX:9 REFERENCES Luca Maria Aiello, Martina Deplano, Rossano Schifanella, and Giancarlo Ruffo. 2012. People are Strange when you’re a Stranger: Impact and Influence of Bots on Social Networks. In ICWSM: 6th AAAI Inter- national Conference on Weblogs and Social Media. AAAI, 10–17. Johan Bollen, Huina Mao, and Xiaojun Zeng. 2011. Twitter mood predicts the stock market. Journal of Computational Science 2, 1 (2011), 1–8. Yazan Boshmaf, Ildar Muslukhov, Konstantin Beznosov, and Matei Ripeanu. 2011. The socialbot network: when bots socialize for fame and money. In ACSAC: 27th Annual Computer Security Applications Con- ference. ACM, 93–102. Erica J Briscoe, D Scott Appling, and Heather Hayes. 2014. Cues to Deception in Social Media Communica- tions. In HICSS: 47th Hawaii International Conference on System Sciences. IEEE, 1435–1443. Christopher A Cassa, Rumi Chunara, Kenneth Mandl, and John S Brownstein. 2013. Twitter as a sentinel in emergency situations: lessons from the Boston marathon explosions. PLoS currents 5 (2013). Michael Conover, Jacob Ratkiewicz, Matthew Francisco, Bruno Gonçalves, Filippo Menczer, and Alessandro Flammini. 2011. Political polarization on Twitter. In ICWSM: 5th International AAAI Conference on Weblogs and Social Media. Chad Edwards, Autumn Edwards, Patric R Spence, and Ashleigh K Shelton. 2014. Is that a bot running the social media feed? Testing the differences in perceptions of communication quality for a human agent and a bot agent on Twitter. Computers in Human Behavior 33 (2014), 372–376. Carlos A Freitas, Fabr´ıcio Benevenuto, Saptarshi Ghosh, and Adriano Veloso. 2014. Reverse Engineering Socialbot Infiltration Strategies in Twitter. arXiv preprint arXiv:1405.4927 (2014). Rumi Ghosh, Tawan Surachawala, and Kristina Lerman. 2011. Entropy-based Classification of “Retweeting” Activity on Twitter. In SNA-KDD: Proceedings of the KDD workshop on Social Network Analysis. Scott A Golder and Michael W Macy. 2011. Diurnal and seasonal mood vary with work, sleep, and daylength across diverse cultures. Science 333, 6051 (2011), 1878–1881. Tim Hwang, Ian Pearce, and Max Nanis. 2012. Socialbots: Voices from the fronts. Interactions 19, 2 (2012), 38–45. Adam DI Kramer, Jamie E Guillory, and Jeffrey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences (2014), 201320040. Kyumin Lee, Brian David Eoff, and James Caverlee. 2011. Seven Months with the Devils: A Long-Term Study of Content Polluters on Twitter. In ICWSM: 5th International AAAI Conference on Weblogs and Social Media. Johnnatan Messias, Lucas Schmidt, Ricardo Oliveira, and Fabr´ıcio Benevenuto. 2013. You followed my bot! Transforming robots into influential users in Twitter. First Monday 18, 7 (2013). Panagiotis T Metaxas and Eni Mustafaraj. 2012. Social media and the elections. Science 338, 6106 (2012), 472–473. Jacob Ratkiewicz, Michael Conover, Mark Meiss, Bruno Gonçalves, Alessandro Flammini, and Filippo Menczer. 2011a. Detecting and tracking political abuse in social media. In ICWSM: 5th International AAAI Conference on Weblogs and Social Media. 297–304. Jacob Ratkiewicz, Michael Conover, Mark Meiss, Bruno Gonçalves, Snehal Patil, Alessandro Flammini, and Filippo Menczer. 2011b. Truthy: mapping the spread of astroturf in microblog streams. In WWW: 20th International Conference on World Wide Web. 249–252. Alan M Turing. 1950. Computing machinery and intelligence. Mind (1950), 433–460. Claudia Wagner, Silvia Mitter, Christian Körner, and Markus Strohmaier. 2012. When social bots attack: Modeling susceptibility of users in online social networks. In WWW: 21th International Conference on World Wide Web. 41–48. Joseph Weizenbaum. 1966. ELIZA – a computer program for the study of natural language communication between man and machine. Commun. ACM 9, 1 (1966), 36–45. Xian Wu, Ziming Feng, Wei Fan, Jing Gao, and Yong Yu. 2013. Detecting Marionette Microblog Users for Im- proved Information Credibility. In Machine Learning and Knowledge Discovery in Databases. Springer, 483–498. Eva Zangerle and Günther Specht. 2014. Sorry, I was hacked A Classification of Compromised Twitter Accounts. In SAC: the 29th Symposium On Applied Computing.

The Rise of Social Bots

You are reading a preview.

Check these out next

The Rise of Social Bots

More Related Content

Slideshows for you (20)

Viewers also liked (9)

Similar to The Rise of Social Bots (20)

More from Goodbuzz Inc. (20)

Recently uploaded (20)