Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Michael Edson, profile picture
Uploaded byMichael Edson
PPTX, PDF13,007 views

Think Big, Start Small, Move Fast: Digital Strategy in a Changing World

The document critiques traditional strategic planning, arguing it is overrated, overly glamorized, and often fails to inspire or succeed in real-world applications. It emphasizes the importance of agile processes, encouraging organizations to focus on action over extensive planning, while also advocating for a shift towards a more impactful and proactive approach. The author concludes that meaningful change can be achieved through small, immediate actions rather than lengthy, rigid strategies.

Government & Nonprofit◦
Downloaded 245 times
Think big, start small, move fast Digital Strategy in a Changing World
NASA HQ Photo CC-BY-NC https://flic.kr/p/adDe8D For Museum Making Experience Randers, Denmark August 24, 2015 Michael Peter Edson @mpedson
NASA HQ Photo CC-BY-NC https://flic.kr/p/adDe8D I love strategy, you need strategy, but strategy is overrated. Most strategies fail. Strategy is not as important as what you choose to work on, or how you choose to work. Let me explain…
NASA HQ Photo CC-BY-NC https://flic.kr/p/adDe8D “Virtually all major for-profit corporations, threatened with rapid technological change and mounting international competition, develop strategic plans.” Michael M. Kaiser Strategic Planning in the Arts , 1995
NASA HQ Photo CC-BY-NC https://flic.kr/p/adDe8D "An organization chart is not a company, nor a new strategy an automatic answer to corporate grief. We all know this; but like as not, when trouble lurks, we call for a new strategy and probably reorganize […] Eventually the old culture will prevail.” Tom Peters & Robert H. Waterman Jr. In Search of Excellence, 1982
NASA HQ Photo CC-BY-NC https://flic.kr/p/adDe8D “In real life, strategy is actually very straightforward. You pick a general direction and implement like hell.” Jack Welch, CEO of General Electric Winning, 2005
The basic precepts of strategy are compelling 1. The creation of a unique and valuable position involving a set of activities 2. Requires trade-offs and choices 3. Creates a “fit” among an org’s activities “Employees need guidance about how to deepen a strategic position rather than broaden or compromise it” via Michael E. Porter What is strategy? Harvard Business Review, 1996
Strategies often say beautiful & important things… But take a look at your favorite organization’s strategy and ask yourself… Values, mission, general direction…
Image: Public Domain: https://en.wikipedia.org/wiki/Horace_Trumbauer#/media/File:Philadephi Image: Joe Ravi, CC-BY-SA 3.0 https://commons.wikimedia.org/wiki/File:Field_Museum_of_Natural_Hi …Does the strategy really matter? If there were two similar organizations, one that had this strategy and one that didn’t, could you tell which was which based only on their actions, decisions, and behaviors? Does the strategy translate into action? Does the strategy make a real difference? ?
NASA HQ Photo CC-BY-NC https://flic.kr/p/adDe8D Strategy is overrated 1. Strategy is over-glamorized 2. Strategy is too inward-looking 3. Strategy is too slow 4. Strategy is too static 5. Strategy overlooks crucial activities 6. Strategy is incomplete 7. Strategy has the wrong audience 8. Strategy is dishonest 9. Strategy fails to inspire 10. Strategy almost never succeeds
1. Strategy is over-glamorized Strategy, as a science, has tended to reflect a profit- oriented, academic, white, male, consultant-centric worldview biased towards large, sexy organizations Derived from Mike Licht, CC-BY https://flic.kr/p/rQKNa U
2. Strategy is too inward-looking Not about the outside world, people, relationships. Your success entirely depends on things that happen outside your organization Derived from Mike Licht, CC-BY https://flic.kr/p/rQKNa U
3. Strategy is too slow Strategies usually take too long to develop (usually, we’ll talk about that) Time More/ better/ a lot Less/ worse/ not much • The relevance of your problem • The relevance of your ideas • How much you care • How much everyone else cares • Your ability to motivate people, get things done, help Most strategy projects start here And end here
4. Strategy is too static CC-BY-SA Lars Lundqvist https://flic.kr/p/rc4kc7 Three to five-year strategic plans are still common. This span of planning and execution may have been appropriate in the 20th century, but is too static and inflexible for today’s rate of change. Start of Now What you put in your plan is probably based on what you learned over the last 3 years of process and execution. Your plan is probably 3 years out- of-date the moment you start it Three years from now Your 3 year plan is probably six years out-of-date now, and whole technologies and social behaviors will have emerged and come to maturity since your plan was approved
Image: http://commons.wikimedia.org/wiki/File:Washing_Line,_Iceland.jpg Washing Line in Iceland CC-BY-SA 5. Strategy overlooks crucial activities Strategic thinking is often biased towards economic/profit models that don’t fully explain important human endeavors See Tim O’Reilly, The Closhesline Paradox https://plus.google.com/+TimOReilly/posts/bhKxn9NbJz1
6. Strategy is incomplete Only partially explains how organizations succeed and fail Tim McFarlane CC-BY-NC-ND (modified) https://flic.kr/p/4dxUme “Changing your company’s strategy almost always proves tougher than you thought it would. But it’s inevitably easier than changing your culture.” The Lords of Strategy Walter Kiechel III
7. Strategy has the wrong audience Way too often, only lip service to users, customers, visitors, partners, collaborators, staff https://commons.wikimedia.org/wiki/File:Direktorium_LDE.jpg https://flic.kr/p/naUzay Thomas Hawke Oakland 2010, CC-BY-NC Too much of this Too little of this
8. Strategy is dishonest Too much cheerleading and taking credit for existing success. Rarely dives deep, exposes contradictions, presents dilemmas, or clarifies choices. “People don’t want to hear the truth because they don’t want their illusions destroyed.” —Friedrich Nietzsche
9. Strategy fails to inspire “At the end of the day, this strategy has to mean something to the people who are actually going to do the work.” Paraphrase Leo Mullen, CEO, Navigation Arts
10. Strategy almost never succeeds (see next slides)
“I had [this exchange] with every single consulting firm I interviewed…” Walter Kiechel III Corporate Strategists Under Fire Fortune magazine, December 27, 1982
"How many of your clients have strategies?" December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"If they're our client, they have a strategy by now.” December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"How many can effectively implement those strategies?” December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"Uhhh," long pause, much visible reluctance. "This is not for attribution, right?” December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"Right. Is it fifty percent?” December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"Oh no, not fifty percent.” December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"Thirty percent?” December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
"Um, no." December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III
“…After a little more back-and-forth, the final estimate would emerge: December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III fewer than 10 percent of their clients, in the consultants' judgment, were fully successful at putting their corporate strategies to work.”
“…After a little more back-and-forth, the final estimate would emerge: December 1982 Fortune “Corporate Strategists Under Fire.” Walter Keichel III fewer than 10 percent of their clients, in the consultants' judgment, were fully successful at putting their corporate strategies to work.” At the highest levels of the art of strategy only 10% of companies are capable of making it work (and I think that’s generous)
"We could do what [the board of directors] wanted and write a traditional business plan. That would keep our board happy, but it would not motivate or inspire our employees, it would not help attract the new talent the company so desperately needed, and it wouldn't address the strategic dynamics of [our] brand-new industry." Eric Schmidt (Google chairman) and Jonathan Rosenberg How Google Works
http://techcrunch.com/2014/04/12/culture-eats-strategy-for-breakfast/ “I used to think corporate culture didn’t matter…And then my first company failed. It failed…because we had no meaningful purpose to create team unity to fight through the tough times.”
“A large but unknowable proportion of businesses fail pursuing nearly perfect strategies.” http://boxesandarrows.com/we-tried-to-warn-you-part-2/
Why would we invest so much time and energy in such a flawed paradigm? There must be a better way.
http://archive.wired.com/science/discoveries/magazine/16-07/pb_theory/ The age-old process of “hypothesize, model, test—is becoming obsolete” in many ways
The end of theory I read somewhere that it was once the practice to design racing yachts through expensive modeling and prototyping that usually didn’t work…Until someone figured out that they could actually build and sail a couple of boats in the real world for the same amount of money they were spending on models.* CC-BY, Oracle EMEA PR https://flic.kr/p/bLM5kZ * And if the boats weren’t fast they could be sold as leisure craft
The end of theory Agile processes (doing quickly to see results with real users) beat long-term projections by experts
“The Agile Manifesto” Manifesto for Agile Software Development (2001) We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the right, we value the items on the left more. http://www.agilemanifesto.org/
https://www.kickstarter.com/backing-and-hacking/drew-conway-on-large-scale-non-expert-text-coding Even for difficult “expert analysis tasks”, many amateurs making a lot of small decisions can outperform a few experts making fewer, bigger decisions
A/B testing beats expert theories Testing with real users beats expert theories & HIPPOs* ideas about what users will like & do. * Highest Paid Person’s Opinion http://www.sparkpage.com/1-billion-worth-of-ab-testing/ Defined: https://vwo.com/ab-testing/
https://meta.wikimedia.org/wiki/Fundraising/2013-14_Report A/B testing of Wikipedia fundraising banners drove 92% increase in donations between 2012-2013
https://medium.com/the-business-of-living/why-quantity-should-be-your-priority-3bc2b16fe3f5
http://handsonpottery.ca/new-6-8-week-classes-means/ Quantity and practice beat the pure pursuit of excellence
“The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality. His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: 50 pounds of pots rated an “A”, 40 pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot — albeit a perfect one — to get an “A”. http://handsonpottery.ca/new-6-8-week-classes-means/ Quality via quantity
“Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work—and learning from their mistakes — the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.” http://handsonpottery.ca/new-6-8-week-classes-means/ Quality via quantity
Doing is SO MUCH EASIER NOW https://youtu.be/nnWZP5hUVNo
“We forget how far it’s come” https://youtu.be/nnWZP5hUVNo We forget how far it's come • Smartphones are a reality for 1 billion people in Africa in 2016 • access to the world's information • two-way video conferencing • Skype, high definition video cameras, • still cameras, libraries of books and music • We've changed their world far more than anybody else's' world, it's these rising billions I'm talking about, and we forget about that. “In 2000, the cost of starting and Internet company was an average of $5 million for the bandwidth, the servers, the software, for all of that. That number has gone from $5 million to $5,000 for the equivalent, in 2014. A thousand-fold reduction. And yet we forget about this stuff… we just take it all for granted.” Peter Diamandis Chairman, X-Prize Foundation
Lars Lundqvist, Swedish National Heritage Board, 2012 On aggregating 4.2 million objects from 40 organizations and making it available through their open API, SOCH http://www.ksamsok.se/in-english/ http://usingdata.tumblr.com/post/32333090538/swedish-open-data-badasses-exchange-between-me Mike: Jacob - - tell us one specific badass-y thing our Swedish Humanities brothers and sisters have done that you admire. Jacob: Lars, tell him how many objects you have delivered to Europeana - it’s like 100 times more than DK [Denmark] have. Jacob: And Lars, tell Michael how many views you’ve had on Flickr… Jacob: They have aggregated 4.2 million objects from 40 orgs, content that is available through their open API - - BOOM! Lars: Well, we try to *do* things. And surprisingly often you become a badass by just trying to do things
https://www.ted.com/talks/jaime_lerner_sings_of_the_city?language=en “Lack of resources is no longer an excuse not to act” Jamie Lerner, as quoted in Tactical Urbanism
Derek Muller “How to Make a YouTube Channel” Muller makes science videos "addressing counter-intuitive concepts in science, usually beginning by discussing ideas with members of the public.” 2,873,219 subscribers • 207,753,005 views https://www.youtube.com/user/1veritasium
Derek Muller “How to Make a YouTube Channel” “My advice sounds so simple: start making videos. So why don't we? Why does everyone find it so difficult? Well, the answer is, you want to make something that's good, that's popular, that everyone likes. You're worried that, what if you make it and nobody does like it, or it's bad. And so rather than face that reality, you just don't make anything.” https://youtu.be/QLIKgT-OSLQ “My advice sounds so simple: start making videos.”
“Real artists ship.” —Attributed to Steve Jobs https://commons.wikimedia.org/wiki/File:Steve_Jobs_Headshot_2010 -CROP.jpgMatthew Yohe, CC-BT-SA
Permissionless Innovation https://www.librarything.com/work/14343122 Today, you can do a lot of profoundly impactful, disruptive, and powerful things without asking anybody for permission “All you need is a laptop”
Permissionless Innovation https://www.librarything.com/work/14343122
Permissionless Innovation More info/slides on Permissionless Innovation, see: http://www.slideshare.net/edson m/dark-matter-the-dark-matter- of-the-internet-is-open-social- peertopeer-and-read-write/181
Think big, start small, move fast
Think big, start small, move fast I first started using this expression 5 years ago, after I heard it at the Clinton Global Initiatives summit It has proven to be an effective and powerful way to understand how to get change started Why? What does it really mean?
1. Think big Thinking big is important because there are so many more possibilities than there were a few years ago—and there are enormous problems to solve
3.1 billion people are online And another 5 billion are likely to join them in the next decade. http://www.internetlivestats.com/internet-users/ “…global connectivity, immense computational power, and access to all the world's knowledge amassed over many centuries, in everyone’s hands. The world has never, ever, been in that situation before…” http://edge.org/response-detail/10646 Keith Devlin Executive Director, H-STAR Institute, Stanford University
There are immense and compelling challenges to overcome • Climate change • Biodiversity • Globalization • Cultural tolerance • Cultural enfranchisement • Literacy (+ scientific) • Education • Human rights • Monetary system reforms http://globalchallenges.org/publications/globalrisks/about-the-project/
Big (global, impactful) work is easier than it has ever been …Even for small teams with limited resources Example: Hank and John Green https://www.youtube.com/user/vlogbrothers/about • 2,640,395 YouTube subscribers • 559,687,790 views Image via http://www.hypable.com/hank-johngreen-launch-subbable-video-platform/
“Within anyone’s reach” “If you are trying to do something big, it’s not enough to just grow, you need to scale… In the Internet Century, this sort of global growth is within anyone’s reach….It no longer takes a phalanx of people and a widespread network of offices to create a company.” Eric Schmidt & Jonathan Rosenberg How Google Works
“Bold goals attract bold people” http://www.roomtoread.org/ Room-to-Read has distributed 10 million books and built 17,000 libraries and 1,900 schools
“In 20 years…I’d like to have 100,000 libraries, reaching 50 million kids. Our 50-year goal is to reverse the notion that any child can be told ‘you were born in the wrong place at the wrong time and so you will not get educated.’ That idea belongs on the scrapheap of human history.” —John Wood, Founder, Room to Read http://www.nytimes.com/2011/11/06/opinion/sunday/kristof-his-libraries-12000-so-far-change-lives.html?_r=2&partner=rssnyt&emc=rss&
“Big bets…attract the best people” http://www.slideshare.net/ericschmidt/how-google-works-final-1#52 Eric Schmidt and Jonathan Rosenberg How Google Works
10x improvement is easier than 10% http://www.wired.com/2013/02/moonshots-matter-heres-how-to-make-them-happen/
2. Starting small Big change seems daunting at first, but small actions, aligned with a big goal, can add up quickly
http://readwrite.com/2014/01/15/world-economic-forum-davos-world-of-developers Will the future be made by big, monolithic institutions?
http://makezine.com/2008/10/10/100000-garages/ …Or in 100,000 garages?
CC-BY Ricardo Ricote, www.urbanity.es https://flic.kr/p/8u5MvX “’Over the past few years Microsoft has spent billions on research and new product development,' observes a professor at Harvard. 'What do they have to show for it? Nothing. Zero.' The innovators that create wealth come instead from interlopers, start-ups, smaller enterprises more capable of seeing and seizing opportunities thrown up by change." Walter Keichel III The Lords of Strategy
“Start doing immediately” “With time and thought, anyone can generate dozens of ideas…that are relevant to a specific situation. My advice: don’t try. A long list can be overwhelming. A sense of being overwhelmed stops action instead of encouraging it. A better strategy is to identify three or four ideas that will be easy to implement, and start doing so immediately.” John Kotter A Sense of Urgency, 2008
Compound Interest Small, early results that generate real knowledge about customers and outcomes generate a form of compound interest Image: http://www.batr.org/negotium/071112.html A little invested here… …Beats a lot invested here
The Lean Startup http://theleanstartup.com/principles “A core component of Lean Startup methodology is the build-measure-learn feedback loop. The first step is figuring out the problem that needs to be solved and then developing a minimum viable product (MVP) to begin the process of learning as quickly as possible.”
“Technology animates the lean start-up process. Free open-source programming tools and easily distributed Web-based software drive down the cost of developing new products and services. The early companies embracing the principles live largely on the Web, which makes it possible to measure and track customer behavior constantly and to invite suggestions and criticism.” http://dealbook.nytimes.com//2010/04/26/the-rise-of-the-fleet-footed-start-up/
Minimum Viable Product Figure out the smallest element of functionality that delivers value to your users, then build and test that. Refine and iterate. Image (modified) http://blog.fastmonkeys.com/2014/06/18/minimum-viable- product-your-ultimate-guide-to-mvp-great-examples/
Minimum Viable Product What I’m talking about is a kind of Minimum Viable Product mentality for strategy—and as an alternative to developing traditional strategies Image (modified) http://blog.fastmonkeys.com/2014/06/18/minimum-viable- product-your-ultimate-guide-to-mvp-great-examples/
Short-term wins "Real transformation takes time, and a renewal effort risks losing momentum if there are no short- term goals to meet and celebrate…Without short- term wins, too many people give up or actively join the ranks of those people who have been resisting change. […] Commitments to produce short-term wins help keep the urgency level up and force detailed analytical thinking that can clarify or revise visions.” John Kotter Leading Change: Why Transformation Efforts Fail 1995
Short-term wins "Real transformation takes time, and a renewal effort risks losing momentum if there are no short- term goals to meet and celebrate…Without short- term wins, too many people give up or actively join the ranks of those people who have been resisting change. […] Commitments to produce short-term wins help keep the urgency level up and force detailed analytical thinking that can clarify or revise visions.” John Kotter Leading Change: Why Transformation Efforts Fail 1995
Short-term wins "Real transformation takes time, and a renewal effort risks losing momentum if there are no short- term goals to meet and celebrate…Without short- term wins, too many people give up or actively join the ranks of those people who have been resisting change. […] Commitments to produce short-term wins help keep the urgency level up and force detailed analytical thinking that can clarify or revise visions.” John Kotter Leading Change: Why Transformation Efforts Fail 1995
Starting small ≠ incrementalism Don’t confuse starting small with avoiding big problems/opportunities Kathy Sierra, Creating Passionate Users: http://headrush.typepad.com/creating_passionate_users/2007/04/my_favorite_gra.html
Starting small ≠ incrementalism Don’t confuse starting small with avoiding big problems/opportunities Kathy Sierra, Creating Passionate Users: http://headrush.typepad.com/creating_passionate_users/2007/04/my_favorite_gra.html Way too many organizations are stuck here—and don’t even know it
What can you do today? Take a stand. Starting small and doing something useful for real people forces you to make a public commitment about what you believe in. This accelerates learning. The world is changing too quickly for any other kind of work. Old models can’t keep up.
My rule of 1 year Think about gathering together a year from now. What two or three things must you accomplish in the next year—things that are so basic and important that if you don’t finish them you should resign in shame. Start working on those things today.
3. Moving fast “Velocity matters,” but time is the most difficult dimension to understand
Wiki-Wiki! "HNL Wiki Wiki Bus" by Andrew Laing - Wiki-Wiki!!!. Licensed under CC BY-SA 2.0 via Commons - https://commons.wikimedia.org/wiki/File:HNL_Wiki_Wiki_Bus.jpg#/media/File:HNL_Wiki_Wiki_Bus.jpg “Wiki-wiki” means “super fast” in Hawaiian. The “wiki” in Wikipedia’s name is inspired by the Wiki-Wiki shuttle at Honolulu airport. Wikipedia succeeds and scales, in part, because wikis encourage speed
“We’re changing…maybe not fast enough.” – Deputy director of US museum, on the results of their widely acclaimed, award-winning museum transformation project “Our next strategy will take three years to develop. It’s too slow.” – Director of Strategy at a major European national library "I don’t know how to convince upper management that any sort of change is necessary, or that any particular change is worth making.” – A program officer hired to institute change at one of America’s largest foundations
"The problem is, most companies today are run to minimize risk, not maximize freedom and speed." Eric Schmidt and Jeffrey Rosenberg How Google Works
“Velocity matters.” Velocity does, indeed, matter, and Google deploys it to great effect. Conventional software is typically built, tested and shipped in two- or three-year product cycles. Inside Google, Mr. Schmidt says, there are no two-year plans. Its product road maps look ahead only four or five months at most. And, Mr. Schmidt says, the only plans “anybody believes in go through the end of this quarter.” Google maintains that pace courtesy of the cloud. http://www.nytimes.com/2007/12/16/technology/16goog.html?pagewanted=2&_r=0
http://www.nytimes.com/2009/12/21/opinion/21lu.html “In many ways, the key to innovation is speed of execution…To maintain a vibrant, innovative program, NASA needs to step up the rate of rocket launchings.” Astronaut Edward Lu argues that in order to improve and innovate, NASA must dramatically increase the number of launches
http://www.slideshare.net/ericschmidt/how-google-works-final-1#16 “Slow doesn’t work in the Internet Century” How Google Works, Eric Schmidt & Jonathan Rosenberg
About slowness… • Some things take time • But behind many “slow” projects are a lot of small, fast things • Most organizations waste enormous amounts of time in the initial “upstream” phases of their projects (when they could be doing and learning) Not everything needs to be fast, but most work is way too slow, complacent.
Think big, start small, move fast 8 Examples 1. Tactical Urbanism 2. Khan Academy 3. NY Times Archive 4. Smithsonian Web Strategy 5. NPR Tiny Desk 6. Vlogbrothers 7. Wikipedia 8. TED
1. Tactical Urbanism Photo by Nina Munteanu, http://ToulouseLeTrek.com , via Tactical Urbanism
“An alternative approach to master planning is beginning to emerge” http://www.citylab.com/design/2012/05/rise-temporary-city/1865/
https://islandpress.org/book/tactical-urbanism https://youtu.be/sMFrJxFxp1Q • Civic hacking at the neighborhood/city scale • A healthy balance of of planning and doing • Short term interventions aligned with long term goals • Open/iterative processes • “The creative potential unleashed by social interaction”
Photo via http://www.citylab.com/tech/2012/02/guerilla-wayfinding-raleigh/1139/ Fast/cheap (and illegal) “Guerilla Wayfinding” project in Raleigh, NC, encouraged walking and quickly lead to new policies, funding, and permanent change
Commandeering public parking spaces to make “parklets” shows the hidden potential of forgotten urban spaces
http://www.thenatureofcities.com/TNOC/wp- content/uploads/2015/07/Crosswalk-861x560.png Guerilla crosswalks (painted by citizens when city hall is unresponsive) A form of prototyping and user testing