The Metropolis Model: ANew Logic for SystemDevelopment Rick Kazman Hong-Mei Chen University of Hawaii University of Hawaii and SEI/CMU firstname.lastname@example.org email@example.com
Trends Two trends are transforming our world: the rise of the socio- technical ecosystem the business trend towards service orientation
The Rise of the NetworkYochai Benkler’s The Wealth of Networks: we are in the midst of a “radical transformation” of how we create our information environment this transformation is restructuring society; models of production and consumption Benkler calls this commons-based peer production
Service-Dominant Logic Service industries account for 55% of economic activity in the United States Businesses have moved from a goods- dominant view, to a service-dominant view In this new view customers are seen as co- creators of value
A Sea Change These changes are much more than just a shift from goods to services. They are a reframing of the purpose of the enterprise and its role in value creation. They are creating new phenomena, e.g. super-linear growth in projects, emergent behaviors in systems, …
Implications for Software Engineering Existing system development models are all broken in a crowdsourced world. These models assume: projects have dedicated finite resources management can “manage” these resources requirements can be known software is developed, tested, and released in planned increments system growth is sub-linear
A New Model We suggest that a new model of the software lifecycle is needed: the Metropolis Model. This model helps us think about system creation that is commons-based and peer produced: E.g. YouTube, Twitter, Facebook, Wikipedia, Orkut, Craigslist, Reddit, LinkedIn, WordPress, Pinterest, Flickr…
Metropolis Model Characteristics1. Mashability2. Conflicting, Unknowable Requirements3. Continuous Evolution4. Focus on Operations5. Open Teams6. Sufficient Correctness7. Unstable Resources8. Emergent Behaviors
1. Mashability Old: we make systems difficult to tear apart, for historical, intellectual property and security reasons. New: “mashability” is a core capability of commons-based peer produced systems mashups are accepted practice web-services are proliferating (e.g., Google Maps APIs). open source projects make it easy to interoperate; they are “nonrival”
2. Conflicting, Unknowable Requirements Old: iterative lifecycles accept requirements change, but in any given iteration they try to collect and analyze those requirements. New: requirements in a peer-produced system emerge from its individuals, operating independently requirements are never knowable in any global sense they will inevitably conflict, just as the requirements of a city’s inhabitants often conflict
3. Continuous Evolution Old: systems evolved in planned releases New: systems are constantly changing resources are non-centralized and so a peer-produced system is never stable one can not conceive of its functionality in terms of “releases” any more than a city has a release parts are being created, modified, and torn down at all times
4. Focus on Operations Old: focus on development/maintenance New: focus on system operations as a core competency Google, eBay, Amazon .... must be “always on” implies high availability, scalability, and seamless evolution
5. Open Teams Old: closed team of developers who work from a consistent set of requirements New: volunteer projects and decentralized production processes with no managers teams are diverse with differing, sometimes irreconcilable, views even for-profit companies need to lead and motivate knowledge workers
6. Sufficient Correctness Old: Completeness, consistency, and correctness as goals New: Perpetual beta collaborative tagging does not depend on widespread agreement among the taggers Wikipedia never claims to be complete or even fully correct
7. Unstable Resources Old: Resources and their deployments are planned New: Applications that are peer-produced are subject to the whims of the peers large numbers tend to ameliorate the actions of any individual despite the lack of guarantees, unstable resource pools have resulted in significant computational achievements
8. Emergent Behaviors Old: Goal was deterministic behavior New: Emergent behavior is normal and desirable SecondLife, eBay, MySpace, and Twitter have seen complex human and system behaviors emerge that were beyond the vision and intent of their creators
A New Logic for System Development Logic: the formal principles of a branch of knowledge; a particular mode of reasoning viewed as valid or faulty -- Merriam-Webster Old models (and their principles) are inadequate What are the principles upon which we can build a new model?
Metropolis Model Structure Open Source Open Content Kernel Periphery Periphery: Developers Prosumers Masses: Customers Masses: Users Masses
1. Egalitarian Management Management of projects is not top-down Work is not assigned; people undertake the work they choose Status and rights are earned
2. Bifurcated Requirements Requirements are bifurcated into: 1) kernel service requirements deliver little or no end-user value focus on quality attributes and tradeoffs slow to change 2) periphery requirements contributed by the peer network deliver the majority of the function and end-user value rapidly changing
3. Bifurcated Architecture Architecture is bifurcated into: 1) kernel infrastructure designed by a small, coherent team highly modular provides the foundation for the achievement of quality attributes 2) peripheral services enabled by and constrained by the kernel otherwise unspecified
4. Fragmented Implementation The majority of implementation (the periphery) is “crowdsourced” to the world using their own tools, to their own standards, at their own pace Implementers of the kernel are close-knit and highly motivated
5. Distributed Testing/V&V The kernel must be highly reliable this is tractable The reliability of the periphery is indeterminate “sufficient correctness” is the norm
6. Distributed Delivery/Maintenance The kernel must be stable: seldom changing, and when it does change it must be backwards compatible At the periphery, perpetual beta is the norm: constant stream of independent, uncoordinated “releases” no notion of a stable system state
7. Ubiquitous Operations Traditional systems—even highly reliable ones—could be “taken down” occasionally for maintenance and upgrades. Metropolis systems are “always on”, even when upgrading upgrades are not ubiquitous Systems must scale with the number of users.
Implications of the Metropolis Model For some projects there are no implications There will always be projects that are: high security highly proprietary safety critical too burdened by legacy
Implications - 2 However, there is an increasingly broad and important class of projects to which the Metropolis model does apply. So, the question is: what should crowdsourced projects do differently from the way projects are built and managed today?
Implications – 3. Main implication: separation of kernel and periphery Consequences on: Project management Requirements Kernel Architecture Testing/V&V Periphery Delivery/Maintenance Masses Operations
Implications: Project Management A Metropolis project implies a virtual organization. To manage such a project the periphery must share in its success. The project must: be (largely) self-governing and self-adaptive have a clear task breakdown structure, but a minimum of hierarchy and bureaucracy have technology for communication and coordination
Implications: Requirements Requirements are primarily asserted by the participants, not elicited from their users. Requirements emerge through their emails, Wikis, and discussion forums. So such forums must be made available, and the participants should be encourage to participate
Implications: Architecture The kernel architecture must be built with a small, experienced, motivated team who focus on modularity enable the parallel activities of the periphery. There should be a lead architect (or a small number of leads) manage project coordination have the final say in matters affecting the kernel
Implications: Implementation The project can guide, but not command, implementation. This implies the need for a focus on communication, negotiation, and leadership. good tutorials and examples an attention to usability (simplicity, learnability) of the kernel
Implications: Testing/V&V Kernel must be tested heavily and validated it is the fabric that holds together the system This can, however, be made tractable keep the kernel small have frequent builds
Implications: Delivery/Maintenance Delivery mechanisms must be created that accept: incompleteness, multiple versions, and incompatibilities of the installed base as the norm.
Implications: Operations Operations must focus on high reliability of the kernel accepting the fact that the periphery will often fail Monitoring and control mechanisms need to be created so that bugs in the periphery can not undermine the kernel.
Final Thoughts Lifecycle models arise in reaction to ambient market conditions. The Metropolis Model is formally capturing a phenomenon that is already occurring: the dramatic rise of commons-based peer production. If an organization wants to take advantage of this it needs to understand it, and needs to understand how to foster it.