Agile Playbook for Software Teams - Enterprise Drupal Edition
AGILE PLAYBOOK FOR SOFTWARE TEAMS Enterprise Drupal Edition
THE SOFTWARE CRAFTSMANSoftware Craftsman n. A man who practices the software craft.1. “software craft”A craftsman believes that software is a craft. This is important because not everyone believes this. A craftsman takes pride in his work anstrives to do the best job he can. He believes that writing good software requires skill and careful attention. That software is not somethingthat can be manufactured nor can it be delivered faster by merely adding more bodies.2. “practices”A craftsman practices his craft, always striving to become more skillful, to produce better software.There are traditionally 3 stages of craftsmanship: 1. Apprentice 2. Journeyman 3. MasterNo matter which stage one may be in, as long as he practices software as a craft, he is a craftsman.3. “A man”Technically the term “craftsman” is gender speciﬁc. Women are just as capable of software craftsmanship. Indeed, I’d like to see more softwarecraftswomen out there. In an effort not to alienate anyone we should use the term “software craftsperson” more liberally.
APPLICATION DEVELOPMENT: MANIFESTO We are continually uncovering better ways of developingsoftware by doing it and helping others do it. Through this work we have come to value: Individuals and interactions over processes and tools Working software over comprehensive documentation Customer collaboration over contract negotiation Responding to change over following a plan That is, while there is value in the items on the RIGHT, we value the items on the LEFT more.
WE FOLLOW THESE 12 PRINCIPLES Working software is the primary measure of progress. Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. Welcome changing requirements, even late Agile processes promote sustainable development. The in development. Agile processes harness change sponsors, developers, and users should be able to for the customers competitive advantage. maintain a constant pace indeﬁnitely. Deliver working software frequently, from a couple Continuous attention to technical excellence and good of weeks to a couple of months, with a preference design enhances agility. to the shorter timescale. Business people and developers must Simplicity--the art of maximizing the amount of work work together daily throughout the project. not done--is essential. Build projects around motivated individuals. Give The best architectures, requirements, and them the environment and support they designs emerge from self-organizing teams. need, and trust them to get the job done. The most efﬁcient and effective method At regular intervals, the team reﬂects on how to of conveying information to and within a become more effective, then tunes and adjusts its development team is face-to-face conversation. behavior accordingly.
OUR OPEN SOURCE MENU FOR SUCCESSOur Development Group:• Is part of a global community with a common vision• Is a pool of talented and motivated development professionals• Is a mix of open source and commercial• Follows an organized, light weight software development approach• Has a clear leadership structure• Excels at communication• Is a business model for success• Is Agile
SCRUM: A LIGHT WEIGHT DEVELOPMENT PROCESSBased on Agile Programmingmethodology and Getting Real, our • Supported with web-enabled toolsScrum process is: • Automated testing integrated with the software development cycle • High intensity design, test, • Short product delivery iterations to implementation cycle provide constant business value
OUR DELIVERY MODEL: A “CONTINUOUS” AGILE PROCESSWe will compresses the standard Analyze, Design, Implement, Test cycles into a Continuous Agile Process of Design, Develop, Implement and where Testing is always being done. Waterfall Iterative Agile (Continuous) … Test … Reqs. Design Code Requirements Requirements Design … Test … Code Reqs Design Code Design Requirements Test … Test … Design Reqs. Design Code Code Code Requirements … Test … Test Reqs. Design Code Design Test … Test … Code Reqs. Design Code Test
OUR DELIVERY MODEL: SEQUENTIAL VS. OVERLAPPING DEVELOPMENT Sequential Requirements Design Code Test Rather than doing all of one thing at a time...Overlapping ...Our team does a little of everything all the time
OUR AGILEDEVELOPMENT MODEL Behavior Driven Development (BDD)
CODE MOVES FORWARD, CONTENT MOVES BACKWARDWhile the content and thesoftware lifecycles will be de-coupled, they are alsointerdependent. Developers needrealistic content to develop andtest code with, and the contentrelies on the code to deﬁne anddisplay itself.The standard approach for managing theseinterdependencies is “code forward, content Functionality and conﬁguration is developed in Development Environmentbackward.” Content and conﬁguration isdeveloped in the Development environment andtested in the QA environment. When it is ready, it Functionality and conﬁguration is tested in QA Environmentis deployed to the Live environment for UAT andProduction instances. Content is developed andpreviewed in the Production instance that contains Content is developed and previewed in Staging (UAT) Environmentthe Staging (or preview/UAT) and Live contentstates. Periodically, content should be publishedbackwards to the Development and QA instancesso that testing can be realistic as possible. Production contains the Staging (UAT) and Live Content and Code states
TEST. DEVELOP. TEST. REVIEW. DEPLOY. Please note the complete lack of developers in this picture...only Hudson, or similar automated build service, can deploy code, directly from the code revision control system. Continuous IntegrationEnvironment Test Automation Build & Release (UAT/Prod) Deploy Deploy Deploy Dev QA Live Code Quality Control Source Code Code Revision Control
SOURCE CODE MANAGEMENT: DISTRIBUTED VERSION CONTROL SYSTEMS (DVCS)In a DVCS (such as Git, Mercurial, Bazaaror Darcs), clients don’t just check out thelatest snapshot of the ﬁles: they fullymirror the repository. Thus if any serverdies, and these systems werecollaborating via it, any of the clientrepositories can be copied back up to theserver to restore it. Every checkout is reallya full backup of all the data.Git Basics:•Snapshots, Not Differences•Nearly Every Operation Is Local•Git has integrity (checksum everything)•Git Generally Only Adds Data•Operates in three states (working,staging, local) http://progit.org/book/
RELEASE EARLY, RELEASE OFTEN… RELEASE QUALITY. Although our web developers will be tempted to keep their code under wraps until it is perfect, this process will encourage them to release their code as soon as it passes its intended tests. The longer the code is visible to the community (other developers, team leads, testers, managers, clients, etc.) the better integrated it will be. All developers agree to keep their software defect free. Although everyone is encouraged to submit their code early, the code must compile and pass nightly tests A continuous build process sends e-mails to developers who check in code that does not compile More effectively, the development group enforces the commitment though peer pressure
UNIT TESTS + ACCEPTANCE TESTSBuilding The Code Right Building The Right Code Automated unit tests are white-box tests that Automated tests are about building the right describe and verify very low-level behavior. code in the ﬁrst place: the code with the most Especially when working test-ﬁrst, developers business value. While developers may be rely on such tests to make sure that the involved in writing such tests, they are not just design is sound, and that there are no defects. for our developers. Tests are especially for Such tests are built by and largely for customers, business analysts, testing/quality developers, and can be independent of professionals, support staff, management, and functionality. all other stakeholders who need evidence that the project is producing solid return on investment, every step of the way. Our web team uses SimpleTest to perform test-driven development. However, if our TDD effort is still hampered by a broken requirements process, we can easily end up building robust, lean, well-factored code with no business value.
AGILE TOOLSEngineering• Revision control• Story Mgmt (& Issue Tracking)• Continuous Builds• Data Quality (Schema Analysis)• Code Quality (Code Analysis)Documentation and Product Management• Project Management (& Issue Tracking)• Community Communication & Collaboration• Technical Documentation• Product Documentation (End User)Collaboration & Communication • Group Wiki• Live Chat• Video Chat• File Sharing
APPLICATION DEVELOPMENT STACK Application Development Stack (ADS): Architectural Patterns
ADS LAMP PATTERN: DRUPAL STACK Through several rounds of benchmarking internally as well as external benchmarks performed byindustry leaders, we have found that our previous web application stack for Drupal requires signiﬁcanthardening for large scale implementations. In light of this, architectural analysis and our commitment toopen source, Drupal, LAMP and of course, growth, the below grid outlines our standard Drupal Stack, hardened by the enterprise scale and commercially supported PHP stack, Zend Server. Varnish Pressﬂow Drupal MemCached Zend PHP + APC Apache Solr MySQL Apache CentOS/RHEL Linux
PHP WEB APPLICATION SERVER: ZEND Zend Server is a complete, enterprise-ready Web Application Server for running and managing PHP applications that require a high level of reliability, performance and security on Linux, Windows or IBM i. Business-Grade PHPAn up-to-date, tested and supported PHP stack ensures high reliability, enhances security and increases staff productivity Deploy with conﬁdence Rapid response to problems Optimal application performance A complete and consistent Advanced application monitoring Built-in optimization andenvironment used in development, and diagnostics enable early acceleration ensures high testing and production eliminates problem detection and quick root performance and low resource many of the problems cause analysis utilization encountered during deployment Highlights: Enhance PHP Application Reliability and Security Quickly Pinpoint Root Cause of Problems With Code Tracing Ensure Successful Deployments Boost Application Performance Detect Problems Before the Phone Rings Job Queue: Ofﬂoad Execution of Long-running PHP Scripts
APPLICATION PERFORMANCE + MONITORING What is Application Performance Monitoring?Application performance monitoring (APM)detects slow response times, poor throughput,user errors, and other performance problems, andthen guides us to diagnose the cause. It alsoprovides powerful features for tuning andoptimizing our apps such as capacity planning andscalability reporting.APM helps to ensure the health and availability ofour production and lab PHP web apps — such asDrupal. We can monitor overall applicationresponse time and throughput includingmemcached, database and external services. Wecan go deeper into individual web transactionsand database calls to MySQL and all the waydown to SQL statements. Zend Server(commercial) + New Relic RPM provides this tous, on demand.
CONFIGURATION MANAGEMENTTaking conﬁguration management to the automation level allows us to create a consistent environment spec for ourapplications, albeit a lab, test, or even production environment. We utilize Chef, an open source systems integrationframework, to bring the beneﬁts of conﬁguration management to our entire web infrastructure.The result is a fully automated infrastructure: when a new server comes on line, the only thing we have to do is tellChef what role it should play in the architecture.
IMPROVING QOS THROUGH A SIMPLE SERVICE ORIENTATED ARCHITECTURE (SSOA) To provide a more agile, autonomous, scalable, integrated and quality driven system for Enterprise Architectures and their applications and clients to operate within, the application stack used for web enabled applications requiring services from back ofﬁce systems, such as legacy systems, should consume systems as common services.Service Creation & Mgmt• Expose end-points, legacy interfaces, and objects as services• Host these services as a lightweight service containerService Mediation• Completely separate business logic from messaging• Shield services from message formats and protocols• Enable location-independent service calls and provides protocol bridgingMessage Routing• Route messages based on content and rules• Filter, aggregate and re-sequence messagesData Transformation• Exchange data across applications with varying data formats• Manipulate message payload, including encryption, compression and encoding transformations• Format messages across heterogeneous transport protocols data types
TYING THE KNOTS: AGILE TOOLS We bring our Agile Practice, Development Model, and of course...People to tie this knot. And by people, we mean everyone involved in the product development life cycle, be it agile developers, business and product managers, scrum masters (or project managers), analysts or users. We bring this all together by using several tools to provide us integrated agile project management, issue tracking, wikicollaboration, source code analysis and reviews, code version control to improve release planning, team communication and customer feedback gathering.Atlassian JIRA + Atlassian Cloudbees Nectar Sonar Crucible forGreenHopper for Conﬂuence for or Atlassian code quality andagile project agile collaboration Bamboo for coveragemanagement Conﬂuence is designed continuous Sonar integrates peer codeGreenHopper adds agile to help us as, agile integration review and code coverageproject management to any developers, to plan As agile developers we directly into our agileJIRA project with one click. requirements, use continuous developer’s work habits.GreenHopper is the perfect collaborate on changes integration to get the We use Sonar to get timelytool for managing our and display metrics. most from our unit tests. feedback on our code,product and sprint We use CI to get maintain quality metrics,backlogs, planning sprints instantaneous feedback assesses the impact of ourand tracking our team on the impact of each tests and notiﬁes us of testthrough the entire release commit and monitor coverage gaps before it isprocess. build metrics. too late. Best of breed