Draft of 5 minute talk about Features Engineering for Wikimedia Foundation Monthly Metrics.
Note this is a PDF because Keynote upload is currently broken on SlideShare. I am including the speaker notes because of this. You can view the builds on Youtube: http://www.youtube.com/watch?v=ltqNe-ZyANE&feature=youtu.be
Also the licensing is reserved because a couple of the images I used are not CC-BY-SA, if I have time I'll work on that.)
work on builds write script handle timing edit scripts
Two caveats. I’m keeping this at 5 minutes so that works out to only 30 seconds per Evil. And second, they’re not all evil, or really evil at all… if I use the “Google Standard” for Evil (if Google has done it, it can’t be evil.) Ignite style: 20 slides, 15 seconds/slide
I normally being my talks with an introduction to who I am. Before I worked at WordPress and here, I used to work at Plaxo (top google suggest for Plaxo was “Plaxo evil” when I worked there) and Tagged (Most annoying website in the world according to Time). I’ve learned a thing or two about Evil during that time. :-) Source: http://techcrunch.com/2006/03/22/plaxo-now-with-less-evil/ http://www.time.com/time/business/article/0,8599,1903810,0 0.html
Luckily Erik and Sue, in their infinite wisdom, decided to keep me from acting directly by appoint me to be a Pointy-Haired Boss. Which gets to my first Evil, the evilest of evil if you will—middle management. Source: http://en.wikipedia.org/wiki/File:Pointy-Haired_Boss.jpg
But I found out while I can’t act directly, I can act through my minions. They’re divided into four groups: Visual Editor, Editor Engagement, Experimentation, and FR-tech. Note that the pictures above only cover the engineers, there are other minions in this group, most of whom report to Howie Fung, who worked at eBay. eBay:that’s not quite Evil… more like Chaotic Neutral. Source: http://wikimediafoundation.org/wiki/Staff_and_contractors]
All of you are familiar with the “oh shit” graph. This is the reason why we do anything.
TODO Source: http://www.toothpastefordinner.com/index.php?date=091410
TODO Source: https://commons.wikimedia.org/wiki/File:Toggleswitch.PNG + screenshots of my Safari browser window
TODO Source: screenshots of my browser window http://ns.wikimedia.org/wiki/File:Visual_Editor-logo.svg
The second features team is Editor Engagement. “Engagement” is also an industry term meaning “time spent on a website.” Whether it’s Echo or Flow, the Editor Engagement team is building the pieces that allow the community to send the notifications or the messages that close the habit loops already on the Wikis, thus installing the infrastructure allowing to create a positive feedback loop of editor engagement on the site. Source: http://en.wikipedia.org/wiki/File:WMF-Flow-Concept-open.png http://upload.wikimedia.org/wikipedia/commons/5/50/Wikimani a_-_2012_-_Athena_Project.pdf
(The Gameification habit loop is designed to optimize user time instead of user count.) Source: http://www.socialsignal.com/system/files/images/mouth-to-ear-whisper.hallmark.jpg
Whether it’s EventLogging, Post Edit Feedback, Account Creation, Onboarding, or Guided Tours, what the group creates should now be understandable from words you’ve already learned so far. Source: Eventlogging: http://meta.wikimedia.org/wiki/Schema:GettingStarted ACUX: http://en.wikipedia.org/w/index.php?title=Special:Us erLogin&returnto=Main+Page&type=signup PEF: http://meta.wikimedia.org/wiki/File:PostEdit_s creenshot,_sandbox.png Onboarding: http://www.mediawiki.org/wiki/File:Screen_Shot_2013-01-02_of_ version_of_GettingStarted_special_page.png Guided Tours: https://www .mediawiki.org/wiki/File:Screenshot_2013-01-28_of_GuidedTours_test,_step_two.png
Not automatic, instead it responds to USER action.
This is what How Features should do things: create bold systems that empower the community to be responsive (not reactionary).
put in VE, Editor Engagmenet, Experimentation, Fundraising. It’s about how they’re used, not about being really “evil.” None of these are really evil…well except for middle management—that guy IS evil!
10 Evil(ish) Things and how they relate to Features Engineering at the WMF
10 Evil(ish) Things and how they relate to Features Engineering @ the Wikimedia FoundationWe’re going to try to provide FRAMEWORK for understanding FEATURES ENGINEERING herefrom the through 10 evil things. A CAVEATS: I’m keeping this at 5 minutes so that works outto ONLY 30 SECONDS PER EVIL.TIME: 15 seconds (BURN SLIDE)…And second, they’re not all evil, or necessarily inherently evil by the “Google Corrollary ofEvil” (if Google has done it, it can’t be evil.)
Who am I?I normally being my talks with an introduction to who I am. A long time ago *in a galaxy far,far away), I used to work at Plaxo and TaggedI might have learned a thing or two about Evilduring that time. :-)Right before I left, Tagged was the World’s Most Annoying Website according to TimeMagazine. After I left Plaxo, TechCrunch is writing articles how they’re less Evil.TIME: 15sec (+0)…Long time ago: Before I worked at WordPress and heretop google suggest for Plaxo was “Plaxo evil” when I worked at PlaxoSource:http://techcrunch.com/2006/03/22/plaxo-now-with-less-evil/http://www.time.com/time/business/article/0,8599,1903810,00.html
http://en.wikipedia.org/wiki/Pointy-haired_Boss #1 PHBLuckily Erik and Sue, in their inﬁnite wisdom, decided to keep me from directly doing evilhere by making me a Pointy-Haired Boss. Which gets to my ﬁrst Evil—MIDDLEMANAGEMENT, which is what I am.TIME: 10sec(+5)…ﬁrst Evil: the evilest of evil if you willSource:http://en.wikipedia.org/wiki/File:Pointy-Haired_Boss.jpg
MY (NOT) EVIL MINIONS Visual Editor/Parsoid Editor Engagement +2 +2 Experimentation Fundraising Tech Ori Livneh S Page Matt FlaschenLike anyone evil, I can’t act directly, I can act through minions. Features Engineering isdivided into FOUR MAJOR TEAMS: Visual Editor, Editor Engagement, Experimentation, andFR-tech. I’ll be explaining what they do in WHITE SLIDES, so look out for them.TIME: 10sec (+10)…Note that the pictures above only cover the engineers, there are other minions in this group,most of whom report to Howie Fung, who worked at eBay. eBay:that’s not quite Evil… morelike Chaotic Neutral.Source: http://wikimediafoundation.org/wiki/Staff_and_contractors]
Text http://en.wikipedia.org/wiki/Wikipedia:Editor_engagement “but being stagnant without any data sucks. Even if staff doesn’t want to be stagnant, community wants to be stagnant.” —former WMF board memberFirst the reason that motivates all of us in Features: THE “OH SHIT GRAPH that all of you arefamiliar with. I won’t bother explaining it.TIME: 10sec (+15)
#2 Heat DeathInstead I want to give a different perspective on editor retention. There is a theory inCOSMOLOGY: because the arrow of time follows increasing ENTROPY, eventually theuniverse will reach a state where it EXHAUSTS THE USABLE ENERGY and the only thing left isheat. This is called the HEAT DEATH OF THE UNIVERSE. The “Oh shit” graph means if editorretention problem isn’t dealt with, the inertia of ACTIVE EDITOR ENERGY will be exhaustedand we have the Heat Death of Wikipedia.TIME: 20sec (+10)Source: http://www.toothpastefordinner.com/index.php?date=091410
#3 AffordanceWhat is Features going to do about it? Affordances are QUALITIES THAT ENABLEDISCOVERING ACTIONS WITHOUT THINKING. Take a lightswitch. You can ﬁgure out what itdoes without thinking. That’s an affordance. There are affordances on THE WEB also.Whether this box with magnifying glass is on Wikipedia or in Safari you know what it does.Web AFFORDANCES ARE NOT STATIC, or may not quite establish themselves before they’rediscarded: take WISHLISTS on Wiki and compare them to BOOKMARKING actions onPINTEREST. In fact, they can get quite advanced from the “salad days” of Wikipedia as in theWORDPRESS FAILED LOGIN example (which uses a real world affordance of SHAKING ONE’SHEAD.TIME: 20sec (+5)Source:https://commons.wikimedia.org/wiki/File:Toggleswitch.PNG + screenshots of my Safaribrowser windowhttp://en.wikipedia.org/wiki/Affordance
& PARSOIDWhen you consider the INTERFACE our EDITORS use interact with the Wikis, this is what yousee. This DOESN’T RESEMBLE MODERN AFFORDANCES such as WordPress or Google Docs.The Visual Editor and the Parsoid teams in Features are trying to update this affordancebecause it is the ALPHA OF ALL EDITOR INTERACTION.TIME: 15sec (+5)Source:screenshots of my browser windowhttp://ns.wikimedia.org/wiki/File:Visual_Editor-logo.svg
#4 The Habit Loop Routine Cue Reward • Alcoholism • Social Gaming • Exercise • Page PatrollingTo further understand WHAT we’re trying to do, let’s talk about Habits. In the Power ofHabit, Charles Duhigg talks about the Habit Loops. They consists of A CUE-ROUTINE-REWARD CYCLE, which is true whether it’s a bad habit like alchoholism, a good one likeexercise, when gaming on social networks or page patrolling on Wikipedia. Note though is isnot a real loop, because THE REWARDS DO NOT CYCLE BACK TO CUES.TIME: 15sec (+5)Reference: The Power of Habit
#5 Viral Growth signup send e-mails read e-mail click link new userA real loop is the exponential viral growth curve. Take the ﬁrst example on the web, Hotmail,as an example. Hotmail placed a the viral link. This created a FUNNEL where a user signedup, sent some emails and SOMEONE ELSE read the e-mail, maybe clicked on the link andpossibly signed up. If it’s a funnel, HOW DO YOU GET VIRAL GROWTH? A user sends e-mailto many users creating a MULTIPLICATIVE POSITIVE FEEDBACK to the loop.TIME: 15sec (+5)Source: http://farm2.static.ﬂickr.com/1290/4690702547_d9d5000e48_m.jpg
#6 Gameiﬁcation Routine: Click on stuff Cue: See Wall Reward: Brag on Post/Tweet social network = Reward: Brag on Cue: See Wall social network Post/Tweet different user Routine: Click on stuffWhen this principle is applied to HABITS it is known as GAMEIFICATION. In social games, auser sees some sort of social cue which reminds them to check their crops on Farmville. Theydo their routine by clicking on some stuff (I hope I’m not oversimplifying here), and get theirreward which is to brag about how awesome their crops are on a social network. Where doesthe loop come in? with a DIFFERENT USER where the rewards of one user become the cue toa habit loop of another thus you have A REAL FEEDBACK LOOP, but multiplying TIME notnumber of users (as in viral growth).TIME: 15sec (+5)…The Gameiﬁcation habit loop is designed to optimize user time instead of user count.
EDITOR ENGAGEMENT Routine Cue RewardTime spent is known as “Engagement” in the web world. And this explains the purpose ofthe Editor Engagement team in Features. Friday, July 13, 12Whether it’s Echo or Flow, the Editor Engagement team is building the pieces that empowerthe community to send the NOTIFICATIONS OR THE MESSAGES that close the habit loopsalready on the Wikis, thus INSTALLING THE LEVERS allowing to create a POSITIVE feedbackloop of editor engagement on the site. But this is purpose-driven, not random clicks unlikestandard gameiﬁcation.TIME: 15sec (+5)…The second features team is Editor Engagement.“Engagement” is also an industry term meaning “time spent on a website.”Source:http://en.wikipedia.org/wiki/File:WMF-Flow-Concept-open.pnghttp://upload.wikimedia.org/wikipedia/commons/5/50/Wikimania_-_2012_-_Athena_Project.pdf
#7 Viral MarketingBUT WHEN DO YOU PULL THE LEVER? Let’s go back to Viral Marketing. Have you everythought what the difference is between word-of-mouth marketing and viral marketing?TIME: 10sec (+10)Source: http://www.socialsignal.com/system/ﬁles/images/mouth-to-ear-whisper.hallmark.jpg
https://commons.wikimedia.org/wiki/File:Soft_ruler.jpg #8 A-B Testing (Measurement)Whether in the viral loop or in gameiﬁcation habits, unlike in meatspace world of word-of-mouth marketing, EVERY STEP IN THE FUNNEL CAN BE MEASURED AND OPTIMIZED. (Thisis known as A-B Testing in industry argot).TIME: 10sec (+15)Source: https://commons.wikimedia.org/wiki/File:Soft_ruler.jpg
EXPERIMENTATION Post Edit Feedback (Reward) Event Logging (Measurement) Onboarding (Cue) ACUX (viral funnel) Guided Tours (Affordance)This is what the Experimentation team is doing. In EVENTLOGGING’s case this measurementis LITERAL. But in terms of the experiments: whether Post Edit Feedback, Account Creation,Onboarding, or Guided Tours, what the Experimentation Team creates SHOULD NOW BEUNDERSTANDABLE FROM WORDS WE’VE ALREADY TALKED ABOUT.TIME: 15sec (+15)Source:Eventlogging: http://meta.wikimedia.org/wiki/Schema:GettingStartedACUX: http://en.wikipedia.org/w/index.php?title=Special:UserLogin&returnto=Main+Page&type=signupPEF: http://meta.wikimedia.org/wiki/File:PostEdit_screenshot,_sandbox.pngOnboarding: http://www.mediawiki.org/wiki/File:Screen_Shot_2013-01-02_of_version_of_GettingStarted_special_page.pngGuided Tours: https://www.mediawiki.org/wiki/File:Screenshot_2013-01-28_of_GuidedTours_test,_step_two.png
FUNDRAISING ENGINEERINGWhich brings us to FEATURES’S LAST TEAM unlike in the COMMERICAL WORLD, Wikimediahas an added restriction. As with many things at the Foundation, taking Fundraising as amodel. Yes, this graph of our statistics is cool—having a $2 MILLION DOLLAR DAY and all.But the meaning is NOT the NUMBERSS themselves, but in the fact that it means that theFundraiser is SHORTER, MORE EFFICIENT, LOW IMPACT. That’s a QUALITATIVEDIFFERENCE not a quantititive oneTIME: 15sec (+15)Source: http://www.ﬂickr.com/photos/tychay/8225719074/
#9 Quantity vs. Quality “Imagine a world in which every single human being can freely share in the sum of all knowledge. Thats our commitment.” • Virality (People) “every single human being” • Gameiﬁcation (Time) “freely share” • Commercial (Money) “sum of all knowledge”That’s a big difference. Commerical companies use Virality and Gameiﬁcation to optimizequantities like people and time for its own sake because it leads to the BIGGEST QUANTITYOF ALL: MONEY. But when you look at our vision statement, it’s not the quantitiesthemselves, they’re just proxies to describe the qualities of “every single human being”instead of people, to freely share INSTEAD OF SPEND TIME NEEDLESSLY; to create the sumof all knowledge, not generate huge piles of cash.TIME: 15sec (+15)
UNBREAKABLE! • Victor’s story • not timid, bold • not policy, system • community not obstacle, empowered • not automated, responsiveThis reminds me of Victor’s Story AT WIKIMANIA. He talks about his ﬁrst action of Wikipediabeing to VANDALIZE the site multiple times and have it REVERTED. But note the story, whathe did was not timid but bold, it was handled not with policy but a SYSTEM where thecommunity was not an obstacle but empowered. And this was not automated butRESPONSIVE. That Responsiveness is a model to how Features wants to tackle the EditorEngagement problem: CREATE BOLD SYSTEMS THAT EMPOWER THE COMMUNITY TO BERESPONSIVE (NOT REACTIONARY).TIME: 20sec (+0)
•Who? (1) PHB •Why? (2) Heat Death •What? (3) Affordance, (4) Habit Loop (5) Viral Growth •Where? (6) Gameiﬁcation, (7) Viral Marketing •When? (8) A-B Testing (9) Quantity/Quality •How? (10) ResponsivenessI hope this explains the four teams in features (VisualEditor/Parsoid, Editor Engagement,Experimentation, Fundraising) and how they want to tackle things, and how the four featuresteams.Who am I? I’m a PHB.Why do we do this? To prevent the Heat Death of Wikipedia.What are Features doing about it? Creating affordances and assisting in habit formation tocreate positive feedback loops of engagement.Where do we install these changes? Using principles of Gameiﬁcation and Viral Marketing toinstall leversWhen do we pull these levers? With measurement and A-B testing tempered by the knowledgethat the quantities are just proxies for the qualities we’re trying to achieve.How? By designing responsively: to create bold systems that EMPOWER the community, notignoring it or reacting to it.So those are the 10 evil things and you see it’s about how they’re used, not about being really“evil.” None of these 10 things are INSTRINSICALLY EVIL…well except for middlemanagement—that guy IS evil! :-)Thank you!TIME: BURN SLIDE (+10)