Full Paper: Twitter - A study of the community - Anand Paka - HomeDocument Transcript
Twitter, Inc<br />A lead user driven product and community<br />Anand Paka<br />Class Paper for <br />15.356 How to Develop Breakthrough Products and Services<br />27 February 2010<br />“The service has taught us what it wants to be”<br />- Biz Stone, Co-founder, Twitter, Inc<br />About<br />This paper was originally written as a term paper for Prof. Eric Von Hippel’s class “How to Develop Breakthrough Products and Services”, class number 15.356 taught at MIT Sloan in H1 Spring 2010. The class covered various topics such as lead user innovation, user communities, hacking, innovation sharing/trading, mass customization through toolkits etc. I chose Twitter to illustrate various concepts of the class deployed in practice. The class restricted the paper to be about 8 pages long, hence my topics were selective and constrained to the most relevant. Since submitting the paper, I have added two sections: About (this one) and Latest Developments (at the end).<br />Introduction<br />With the widespread adoption of the internet and the growth in knowledge workers, innovation has shifted from the corporation to the users. In 15.356 we discussed how lead users often push the boundaries of products, resulting in fundamental innovations. Companies either support (E.g. Lego) or discourage (E.g. Sony Aibo) such innovations. <br />Over time, the relationship between the company and lead users has become increasingly symbiotic and the boundaries between them have blurred. Organizational designs have expanded to include the customers. In this paper, I will study in detail one example of this high degree of integration – the micro blogging service, Twitter, Inc. I will explore how this entire service and community (almost an industry) has evolved due to lead users and a supportive stance taken by the corporation. There are lessons here to be learned by companies, especially those that operate largely through the internet and in areas such as social media – once you have a core product that fulfils a user need, tightly integrate the user community into the evolution of that product. By listening and responding to their needs, the end product takes its own path to innovation.<br />What is Twitter?<br />Twitter is a micro blogging service that allows registered users to send short (140-word long) messages or status updates. Other users can choose to receive these updates by selectively opting to “follow” the sender. By default, all messages sent are public, although users have the option to keep them private. Twitter can be accessed through phones and internet, and using various devices (computers, various smart-phones etc.). The messages can be sent real-time to your phone or accessed on demand. In addition to Twitter’s website, several other sites and desktop applications have been developed to allow access to the content. Twitter is also being increasingly integrated with other networking sites such as Facebook and LinkedIn.<br />Creation of “Twttr” – User driven brainstorming<br />First there was Odeo, a podcasting company, whose founders were already intelligent users of growing media such as podcasts, RSS, text messaging, etc. In 2006, the company held a day-long brainstorming session to generate ideas to reinvent the company. Employees split into teams for the exercise. An engineer on one of the teams, Jack Dorsey, proposed a service that would use SMS to broadcast a user’s update to small groups. Jack was an early innovator in automated dispatch services (used for taxis, couriers, emergency services etc.). As a lead user in the space, Jack’s early use case was urban – he wanted a way to tell people that a club that he was at was happening. This led to the idea for the basic dispatch service, originally named ‘TWTTR” to comply with 5-digit American SMS short codes. The initial version was developed inside Odeo and made available for internal employees who tested and used it. The service was officially launched after 6 months and Jack became the CEO.<br />Adoption of Twitter<br />Twitter quickly garnered users after launch, mostly individuals who were passionate about technology. They began using Twitter for various business and fun activities. While this growth was healthy, the turning point in the adoption of Twitter came with the South by Southwest festival (SXSW) – a conference for music, films and emerging media. The Twitter team placed two 60-inch plasma TVs at the venue. Updates to Twitter or “Tweets” from users at the conference were broadcast live on these screens. Since the service was freely available and only needed a phone for sending messages from, conference users quickly started using the service to track each other and share feedback. The buzz created from this and the positive feedback from users led to Twitter winning the Web award at the conference. The service’s message volume tripled overnight. To a large extent, this success was possible because the conference attendees were largely lead users of media and phones. The conference represented an unfilled need that was addressed by the service. The growing user base quickly started evolving the service and testing its boundaries with different problems.<br />Influence of Lead Users: Feature definition, Culture, Business Applications & Symbiotic Companies<br />In this section, I will explain how lead users shaped various aspects of the service. From early on, the management team at Twitter was attentive to how the service was being used and folded in changes as appropriate. These changes are well documented in the official blogs on Twitter’s websites which cover the last 4 years of the service’s existence.<br />Feature definition<br />Several features in Twitter have their origins in lead users. <br />
One of the most widely used features on Twitter is the ability to reference other users by using the “@” symbol, followed by the target user name. This feature came from users and was not part of the initial offering (see Exhibit 1). As the user base grew, conversations increased in volume. The 140-character limit meant that users had to find short ways of referring to each other. They did so by using the “@username” format. Upon seeing users using “@” to reference each other, the Twitter team built the official feature around it.
During the first few months, Twitter did not have any official API (Application Programming Interface). Nonetheless, some lead users reverse engineered the HTTP commands that the service was using and began developing applications using this. For example, there was a website that showed a nifty map depicting where “tweeters” were located. Upon seeing this activity, the Twitter team released their official API. Users then began hacking this API E.g. iChat updates sent directly to a Twitter update. Through all this, the team at Twitter encouraged these hackings and supported them! This was a great way to learn and grow. Eventually, the team also created a Google Group forum to invite ideas and feedback on the developer API. See Exhibit 2. Today, the Twitter API is a highly evolved system with lots of user input.
The popular hashtags – keywords in status updates that are prefixed with “#” – were also a user driven innovation. Users began using the ‘#’ symbol to identify common subjects in their discussions and updates, which allowed them to manage topics better. These hashtags are now routinely used to facilitate search and to group together tweets that pertain to the same topics.
Culture<br />In addition to shaping features, lead users have created the culture around the service as well. Within a couple of months, they created a fan site; http://twitter.pbworks.com/. This site served as a repository for FAQs, etiquette, help, applications etc. It was also a user moderated forum to share ideas, media articles, blogs, tools etc. <br />Another great user-managed site is http://wthashtag.com. As users began using hashtags (explained earlier), there was a need to maintain a repository of what different tags meant, how they were to be used etc. The above site evolved as a forum for users to provide this information (similar to Wikipedia). Hashtags are a crucial element of Twitter culture and are used widely by people to interact on the platform. <br />Perhaps most importantly, the users began exchanging ideas on what the service itself was and how it should be used. This was crucial because Twitter was a new paradigm. By developing a mindshare around what the service is, the users evangelized Twitter to the world and simultaneously built the culture and the market for it.<br />Business Applications<br />Lead users from various businesses began to use Twitter to explore solutions to their problems. This led to a rich and varied experimentation, often resulting in improvements to their businesses and to growing Twitter’s use scenarios (see Exhibit 3). Here are some examples:<br />
Early experimentation by celebrities has led to an increased adoption of Twitter as a marketing tool. Several celebrities use Twitter extensively to stay close to their fans. Some of the users with the largest followings are Hollywood people (E.g. Ashton Kutcher). To prevent fake user accounts, Twitter actually added a “Verified User” feature.
News sites such as CNN and NY Times are extensive users of Twitter. There was early validation of the effectiveness of Twitter to catch breaking events faster than traditional channels (E.g. Hudson river landing of United Airlines flight).
Retailers had early success in engaging users through Twitter. For e.g. Dell Outlet used it as a means to quickly dispatch its inventory. JetBlue used the service to respond to user complaints. Eventually, Twitter hosted case studies of successful business usages of Twitter on its website to help other users.
Symbiotic Companies<br />Several companies were created around Twitter: URL Shorteners such as Bit.ly, Picture repositories such as TwitPic, applications such as Twitterific, PC & iPhone applications such as Tweetdeck etc. The sheer number of these companies has in fact led to development of sites that are devoted to aggregating applications – OneForty.Com was created for this purpose. This has led to a rich ecosystem and a burgeoning user base for Twitter.<br />One of the key examples of how this helps Twitter is Realtime Search. Originally, Technorati, a company that does internet search for blogs, approached Twitter for a comprehensive, high-volume API to get all tweets. Twitter built such a real-time feed over the open XMPP protocol. Since it was resource intensive, Twitter limited it to a few companies. One of these companies ended up being Summize, a startup founded as an academic research project. Summize offered a real-time conversation search, quickly becoming the best of breed API in its class. Twitter was in touch with the company and, seeing the importance of filtering real time information feeds, ended up acquiring Summize. Since then, Twitter has incorporated the search technology into its own platform offering the best search functionality for tweets. This goes to show how a request from lead users results in fundamental offerings in the product. <br />The XMPP API has also evolved into a revenue stream for Twitter. Recently, Twitter announced that it would charge tiered rates for usage of this API depending on the client company’s size. Given the importance of revenue streams for a startup like Twitter, this example goes to show how valuable a user-driven feature can be for the company.<br />Open Source adoption<br />Twitter embraced the open source paradigm from early on by opting to use an open source distributed queue server as their back-end (see Exhibit 4). Given the high reliability and availability demands for the service, it is highly beneficial to use software that has undergone inspection and improvement from thousands of other software programmers. When the Twitter development team authors software from the ground up, it does so with a view of contributing it back to the open source community. Exhibit 4 has two examples of Twitter server functionality that was released in this manner.<br />Using open source is a clear signal of Twitter’s openness to its user community and encourages users to share code and information between each other.<br />Current Directions<br />Twitter continues to evolve based on directions from its users. One of the biggest trends today is in geolocation – using user location in tweets. Lead users of the Twitter API began doing innovative use with location even though the information in the API was minimal. They used account level information which was not necessarily accurate. Twitter responded to this by adding latitude and longitude level information to tweets. This would enable a rich set of applications built around location. Among other companies, two startups have already begun to develop a strong user base and business models around this functionality: FourSquare and GoWalla.<br />Over the last year Twitter has also started to explore means for generating sustainable revenue. There were vociferous concerns from the customer base over the implications of adding advertising into the ecosystem as a means of generating revenue. The Twitter team was very transparent with its customers as it went about thinking of revenue streams. It is extremely important for the company to ensure that whatever solution it comes up with is acceptable to its users. This will shape Twitter’s future in the coming months.<br />Conclusions<br />The study of Twitters evolution shows how lead user driven innovation can result in a rich array of features, products and communities. What is necessary is for the company to recognize this and to foster it. As users begin hacking and extending the product, companies should understand the viability of these requirements and fold them into the official product. They can encourage this creativity through public recognition of user achievements and by offering technical support to them. The combination of active lead users and a supportive management team can result in far reaching impact.<br />Latest Developments<br />Twitter’s ongoing developments continue illustrating the user-based and user-centric innovation and are too many to list. Nonetheless, here are some key latest ones:<br />
Twitter recently announced Promoted Tweets as a revenue model. A promoted tweet is a sponsored tweet that is shown at the top of a user’s search results. Eventually, these tweets will be inserted in the user’s time line view. Twitter is taking small, measured steps as it experiments with this concept, to ensure that its user community can adapt to this model. Furthermore, Twitter is defining a new monetization scheme based on Resonance. Resonance is a measure of the level of user engagement with a promoted tweet through retweeting, replying, marking it as a favorite etc. For a promoted tweet to retain its top-level status, it must have high resonance, in addition to being paid-for by the sponsor. The elegance in the model is that if users do not like a tweet, it will automatically drop off. This is a great demonstration of building a revenue model that goes hand-in-hand with users’ preferences. Finally, Twitter has announced a 50-50 revenue split with app developers who deploy promoted tweets – another signal of the company’s commitment to its developer community.
Twitter introduced the @anywhere functionality to easily allow client websites to tap into Twitter functionality on their own pages, thus creating more engagement with their visitors. This is a great effort to keep the Twitter community integrated across the web.
Twitter acquired the iPhone App-maker for Tweetie. There were several such apps already created around Twitter. This is another illustration of the symbiotic company concept.
Appendices<br />Exhibit 1 – Twitter Official Blog, “@” replies<br />Are You Twittering @ Me?<br />Tuesday, May 29, 2007<br />At some point, Twitter-ers came up with their own method of directing updates to one another using an @ symbol. We started supporting this behavior by doing things like creating the in reply to link, confining the reply messages to friends in the loop, and linking the @username to a profile. Another layer of support to this behavior is a feature we launched tonight which collects updates directed at you—just in case you missed 'em.The Replies page is where you can go to see all the updates that folks have intentionally Twittered in your general direction. In other words, if somebody Twitters "
@biz liking the new Replies tab!"
it will get saved at Twitter.com/replies. For more info, check out these support documents Crystal wrote up: What is the Replies tab?, and What does @username do?<br />Posted by @Biz at 6:16 PM<br />Exhibit 2 – Twitter Official Blog, Twitter API<br />Introducing the Twitter API<br />Wednesday, September 20, 2006<br />Some smart folks out there on the that series of tubes we call the Internet have been putting together interesting projects like this map and this page without any help from us so we thought it was high time to release an API<br />From iChat to Twitter via API<br />Tuesday, September 26, 2006<br />The folks over at TikiRobot have put the Twitter API to good use. First of all, it's a group blog so they've got all three of their Twitters up there which is neat. More importantly, they've hacked our api so that an iChat status automatically publishes as a Twitter status. Thus, they have gotten right down to the roots of what Twitter is all about. Nice!<br />Posted by @Biz at 11:33 AM<br />Now for the really geeky update -- we have released an Application Program Interfacefor developers. This means we've exposed some of the inner workings of Twitter so engineers who want to creatively extend our functionality can do so. We were happilly surprised at how quickly some really neat projects emerged.<br />Mo.istCellyTwittermapiChat<br />That's about it for now. We're busy on some big improvements and feature updates but those will have to wait until next time.<br />Join the Twitter Development Group?<br />Thursday, March 15, 2007<br />There's lots of activity around the Twitter API so we set up a Google Group for discussing various issues. Please do subscribe to Twitter Development Talk if you're working a project involving Twitter and want to be in touch with folks who are doing the same. We'll be in there too!<br />Posted by @Biz at 12:09 PM<br />Exhibit 3 – Business Applications<br />Wife in The Fast Lane<br />Thursday, January 11, 2007<br />Karen Quinn has written a new novel called Wife in the Fast Lane and she's also created a fun contest for folks to share a scene from their fast lane lives with a video, essay or a quick one-liner. Of course, Karen has a Twitter profile so you can keep up with the contest.<br />Posted by @Biz at 11:38 AM<br />Hillary Clinton and Twitter<br />Wednesday, January 24, 2007<br />The folks over at Campaign08blog.com want canditates to get Twitter accounts.<br />We could watch as their campaign went from state to state and find out what fund raiser they are attending and what speaking engagement they might be attending.<br />That would be neat!<br />Posted by @Biz at 4:04 PM<br /> Twitter Goes Hollywood<br />Thursday, April 12, 2007<br />My friend Greg directs tv shows and films and he's got a new series premiering on FOX this Sunday night called 'Drive.' The show was co-created by the same guy who made Firefly (Tim Minear) and stars the same lead actor, Nathan Fillion. The series is about a secret, illegal underground race and there was a recent New York Times article about the groundbreaking technical achievements of 'Drive'.Anywhoo, Greg is going to do Twitter-style director's commentary live during the premiere. As far as we know, this is a first! Follow along if you like, just text FOLLOW FOXDRIVE to 40404 or visit the profile page and follow from there. The show is on this Sunday night at 8/7c. Here is the official web site. Should be good times!<br />Posted by @Biz at 10:18 AM<br />Another Twitter User in The News<br />Using Twitter to track his diabetes attracted attention from other diabetics as well as a health reporter from a local NBC affiliate. Check out Scott's blog post about the experience and watch the clip.<br />Posted by @Biz at 9:08 AM<br />PC World - LA Fire Department all 'aTwitter'<br />Friday, August 03, 2007<br />How the LAFD uses Twitter: "
The fire department uses Twitter to post information about fires or other emergencies that it is responding to. These messages are then sent to users signed up to receive the information on their mobile devices."
<br />Posted by @Biz at 7:59 PM<br />BestBuy, Good Stuff<br />Wednesday, July 22, 2009<br />We're beginning to discover the value that customers and consumers are getting from businesses who use Twitter in smart and interesting ways. Businesses of all shapes and sizes are discovering the value too—whether it's raising the bottom line or engaging more actively with consumers.A small business like a bakery will send out a tweet that the cookies just came out of the oven and a few dozen local followers will rush over and buy warm cookies. The customers like it and the small businesses owners love it. Big companies are using Twitter in interesting ways too.BestBuy has created a program they call HYPERLINK "
Twelpforce. The idea is that employees from across the organization can interact quickly and easily with customers who have questions about products. This is a different approach but it's smart—they're building trust and fostering happy, informed customers. That's good stuff.We've been studying how customers and businesses interact and derive value from Twitter. From this research, findings, use cases, and best practices have emerged. We're putting together a document based on our studies and we'll find a spot on our web site to share it with everyone when it's ready.<br />Posted by @Biz at 8:07 AM<br />Exhibit 4 – Open Source <br />Twitter's 'Starling' Released as Open Source<br />Friday, January 18, 2008<br />Twitter has a warm spot for innovative simplicity and an open approach to technology development. We make use of open source software when it makes sense and we think it brings good karma to contribute back to the open source community whenever possible.However, our open approach is more than just good karma. Twitter, Inc. is committed to building a reliable social messaging utility which people trust enough to use every day. Gaining trust means showing our work. When a technology is shared, conversations and understanding form around it.Our open approach is very much driven by Twitter engineers like Blaine Cook. Blaine coded the distributed queue server Twitter uses to route vast numbers of messages in the background so front-end response time can remain quick.Starling is a light-weight persistent queue server that speaks the MemCache protocol. It was built to drive Twitter's backend, and is in production across Twitter's cluster. There's a little more about Starling on the Twitter Developer Blog. And here is theStarling page at RubyForge.<br />Posted by @Biz at 8:58 AM<br />Building on Open Source<br />Tuesday, January 13, 2009<br />When we plan new engineering projects at Twitter, we measure our requirements against the capabilities of open source offerings, and prefer to use open source whenever it makes sense. By this approach, much of Twitter is now built on open source software.In some cases, our requirements—in particular, the scalability requirements of our service—lead us to develop projects from the ground up. We develop these projects with an eye toward open source, and are pleased to contribute our projects back to the open source community when there is a clear benefit. Below are two such projects, Kestrel and Cache-Money. Every tweet touches one or both of these key components of the Twitter architecture.Kestrel's Wonderful PlumageKestrel is a message queue server we use to asynchronously connect many of the services and functions underlying the Twitter product. For example, when users update, any tweets destined for SMS delivery are queued in a Kestrel; our SMS service then reads tweets from this queue and communicates with the SMS carriers for delivery to phones. This implementation isolates the behavior of SMS delivery from the behavior of the rest of our system, making SMS delivery easier to operate, maintain, and scale independently.Users of the Starling message queue server will find Kestrel familiar, as Kestrel is a port of Starling from Ruby to Scala. In addition to being generally more efficient, Kestrel adds several new features, such as a facility for handling significantly bursty queues.Robey is the lead developer of Kestrel. You can read his lively journal entry on Kestrel's latest features. Kestrel is available on HYPERLINK "
github.As Good as Cache-MoneyCache-Money is an elegant write-through caching plugin for Ruby on Rails. In write-through caching, new or updated data is first written to an efficient cache (such asmemcached) and then stored in a database; subsequent requests for this data are then likely to read the data from the faster cache, rather than from the slower database. In addition to the efficiency gains associated with caching, this technique also addresses the risk of short-term replication lag between master and slave databases since data written during the lag time will likely be present in the cache. Cache-Money plugs directly into Rails's ActiveRecord to transparently provide this functionality.Nick is the lead developer of Cache-Money. Check out his blog for an excellentintroduction. Cache-Money is available on HYPERLINK "
github.<br />Posted by @ HYPERLINK "
gregpass at 11:12 PM<br />