why agile?
  Tathagat Varma
    Sr. Director
     Yahoo!
the world around
 us…yesterday!
Microsoft Windows timeline




https://en.wikipedia.org/wiki/Timeline_of_Microsoft_Windows
Other major OS and tools
                              timeline




                                                  https://en.wikipedia.org/wiki/Bugzilla
https://en.wikipedia.org/wiki/Solaris_(operating_system)
A typical support timeline




http://itconvergence.blogspot.in/2012/10/oow-12-elison-extradata-oracle-r122.html
As a contrast, what are consumer
       internet companies doing?

 Continuous Integration -> Continuous Delivery ->
    Continuous Deployment
   On ‘good days’, Flickr releases a new version every
    half an hour (Jun 20, 2005)
   IMVU pushes a revision of code to the website every
    nine minutes (Feb 10, 2009)
   The other day we passed product release number
    25,000 for WordPress. That means we’ve averaged
    about 16 product releases a day, every day for the
    last four and a half years! (May 19, 2010)
   A new version of Google Chrome now due every six
    weeks (Jul 22, 2010)
   Facebook does code push twice a day (Aug 4, 2012)
adoption @ net speed!!!




https://lh4.googleusercontent.com/-SQ1ugOgu8Ds/TimaHjH0VpI/AAAAAAAAApU/za0BhEjhoio/w402/google%2Bplus%2Bgrowth%2B20mil.png   http://thesamerowdycrowd.wordpress.com/2011/12/26/a-moment-of-our-time/
darwin at work on internet
                           2006                                                                            2009




http://www.flickr.com/photos/stabilo-boss/93136022/sizes/o/in/photostream/   http://rossdawsonblog.com/weblog/archives/2009/05/rapid_turnover.html
yet, our product development
                                  sucks!




http://blog.amplifiedanalytics.com/2011/07/musing-on-difference-between-successful-product-innovation/   http://www.nickblack.org/2009/10/how-brand-trust-affects-new-products.html
Let’s understand the ‘craft’
                        first…
              Sheer joy of making things

              Pleasure of things that are useful to other
                   people

              Fascination of fashioning complex puzzle-
                   like objects of interlocking moving parts and
                   watching them work in subtle cycles

              Joys of always learning, which sprints from
                   the non-repeating nature of the task

              Delight of working in such a tractable
                   medium
The Mythical Man Month – Fred Brooks, 1975
software development life
          cycles
 Ad-hoc

 Serial

 Iterative

 Incremental

 Iterative/Incremental
risk management in SDLCs
notion of ‘progress’ vs. time
Waterfall Model
 Wrongly inspired by assembly-line manufacturing
   processes of the day

 Economics supported ―measure twice, cut once‖
   leading to up-front planning and BDUF

 Single-pass, sequential process with hand-offs and
   feedback loops between adjoining phases

 Transition to next phase only upon completion of
   current phase
Waterfall Software Development




                                     Limitations and Assumptions

      1. Wrong analogy: Software development ≠ Production
      2. Customers know EVERYTHING upfront and that requirement won‘t change
      3. Legacy from the past: implicitly assumes CPU time is costly, so focuses on
         doing everything upfront to minimize ‗machine time‘ for trial and error
      4. ―Wicked Problem‖: Designers and developers know how exactly how to build
      5. Very long feedback cycles not suitable for today‘s pace of innovation

Picture from http://damonpoole.blogspot.in/2009/07/traditional-development-game-of.html
As a result, software is…



      Costly

               Buggy

       Late
and the costs…?




http://leadinganswers.typepad.com/leading_answers/estimating/
http://www.agileforall.com/dyk/
Holy Grail of Software
    Development

         Better: higher
          quality, more
          reliability, higher
          performance, more
          usable…
         Faster: speedier
          development
         Cheaper: no budget
But the reality?
Preamble to Agile
            Movement
Software Crisis, 1965-85: The major cause of the
software crisis is that the machines have become
several orders of magnitude more powerful! To put it
quite bluntly: as long as there were no
machines, programming was no problem at all; when
we had a few weak computers, programming
became a mild problem, and now we have gigantic
computers, programming has become an equally
gigantic problem. — Edsger Dijkstra, The Humble
Programmer
Software Crisis
The causes of the software crisis were linked to
the overall complexity of hardware and the
software development process. The crisis
manifested itself in several ways:

 Projects running over-budget.
 Projects running over-time.
 Software was very inefficient.
 Software was of low quality.
 Software often did not meet requirements.
 Projects were unmanageable and code difficult to
  maintain.
 Software was never delivered.
and the response?




Frameworks, Standards and Certifications
…and the result?…good
       start…
…but poor finish!
and sadly, none of these came
  out of ‘process factories’…



                                                                 2011 -
                                                                 • Instagram
                                2000-2010
                                •2010: Pinterest, SnapDeal,
                                •2009: Square, Quora, Sina Weibo
 1975-2000
                                •2008: Groupon, AirBnB, GoGo
 •2000: Baidu
 •1997: Yandex
                                •2007:
 •1994: Yahoo!, Amazon,          Dropbox, Zynga, Flipkart, InMobi, Hul
  NetScape,                      u, Tumblr,
 •1986: Pixar                   •2006: Twitter, SlideShare, Badoo
 •1984: Sybase
                                 Spotify
 •1983: Intuit, Borland,
 •1982: Sun, Symantec, Adobe,   •2005: YouTube, Renren
  EA                            •2004: Facebook
 •1980: Informix
 •1979: EMC                     •2003:
 •1977: Oracle                   Myspace, Skype, Rovio, Gameforge,
 •1976: CA, Apple               •2002: LinkedIn
 •1975: Microsoft
                                •2001: StumbleUpon, Mail.ru
Why?
 Process: Long-lead development process
  ineffective in a dynamic and global world
 Management: Command and control model
  unsuitable for fostering collaboration required
  to solve complex problems
 Technology: Advancements in
  computers, compiler technology and
  debugging and testing tools greatly improved
  the economics of software development
 Innovation: in the age of hyper-innovation, old
  processes were simply ineffective
What is the most important part
  in these two machines?




      ―The Brakes!!!‖
  They let you go faster…
Agility vs. Discipline?




http://www.ibm.com/developerworks/rational/library/edge/08/feb08/lines_barnes_holmes_ambler/
Advent of Agile and Lean
               Methodologies
   1970: Royce critiques Waterfall and offers improvement ideas
   1986: Barry Boehm proposes Spiral Model
   1971: Harlan Mills proposes Incremental Development
   1987: Cleanroom Software engineering
   1991: Sashimi Overlapping Waterfall Model
   1992: Crystal family of methodologies
   1994: DSDM
   1995: Scrum
   1996: Rational Unified Process framework
   1997: Feature Driven Development
   1999: Extreme Programming Explained
   2001: Agile Manifesto is born
   2003: Lean Software Development
   2005: PM Declaration of Interdependence
   2007: Kanban-based software engineering
   2008: Lean Startup
   2009: Scrumban
   20xx: Something new !?! (hopefully!)
What is agile really all
            about?
                               • Empowered individuals
                               • Collaboration
             Motivated         • Democratic decision-
             Individuals         making and transparency



   Self-                       • Shorter feedback cycle
organizing                     • Manage changing
    x-
functional                       priorities
  Teams                        • Increased productivity
                    Agile
                  Businesses
                               • Higher ROI
                               • Faster time to market
                               • Better User Experience
Why is it so hard?



52 39 34                  %              %
          %
Organizational   Resistance to   Management
       Culture        Change         Support
feedback loop in agile
      lifecycles
from daily builds to
      project
Scrum
What’s happening here?




http://ayagebeely.blogspot.in/2008_08_01_archive.html
Feedback Loops in Traditional
Techniques vs. Agile Techniques
Agile Development Value
                       Proposition




http://www.versionone.com/Agile101/Agile_Benefits.asp
Does Agile work?




http://www.bigvisible.com/2009/12/taking-agile-beyond-faster/
http://www.testingthefuture.net/page/2/
does iterating help?




http://viniciusvacanti.com/2011/12/12/when-do-you-throw-in-the-towel-on-your-struggling-project/
are small teams more
                                                          productive?




http://drewcrawfordapps.com/2.0/the-agility-of-small-teams/
does colocation impact team
                                             performance?




http://sloanreview.mit.edu/the-magazine/2009-summer/50412/how-to-manage-virtual-teams/
is small batch size
                                                     faster?




http://www.andrejkoelewijn.com/wp/2011/06/30/is-team-productivity-a-responsibility-of-the-product-owner/
Let’s build a car


          …and I
          need it
          delivered…
          next week!
www.wikispeed.com
The Wikispeed Process
   At WIKISPEED, some of our projects move more than 10,000% faster than
    industry norms because of our blend of Agile, Lean, Scrum, and Extreme
    Programming/Manufacturing practices.

   Team WIKISPEED uses methods developed by the fastest-moving software
    companies. In fact, in many ways we have more in common with Google or
    Twitter than with GM or Toyota.

   Manufacturing and old-thought software teams gather requirements, design
    the solution, build the solution, test the solution, then deliver the solution. In
    existing automotive companies, the design portion of that process alone
    takes 3 to 12 years, and then the vehicle design is built for 5 to 14 years. This
    means it is possible to buy a brand new car from a dealer and that car
    represents the engineering team's understanding of what the customer
    might have wanted 26 years ago!

   Team WIKISPEED follows the model of Agile software teams, compressing the
    entire development cycle into one-week "sprints." We iterate the entire car
    every 7 days, meaning that every 7 days we reevaluate each part of the
    car and reinvent the highest-priority aspects, instead of waiting 8 to 26 years
    to upgrade.
Wikispeed uses…
 Lean Software Design: Use less stuff

 XP: Pairing and Swarming

 Agile: Reducing costs to make
  changes
 Scrum: Clearly defined team roles
  and responsibilities
 TDD: start with failing tests and
  develop solutions
 OOP: contract-first development
Recap
 agile ≠ Faster, but Sooner

 agile ≠ No planning, but Adaptive Planning

 agile ≠ More work, but ‗Done‘

 agile ≠ No documentation, but Just Enough

 agile doesn‘t just change the development
   process, but bring a radical change in
   organizational culture, leadership and
   management practices that is more in line with
   business needs and social values and norms of
   today
It’s not about the
                 method!
A photographer went to a
socialite party in New York. As
he entered the front door, the
host said ‘I love your pictures
– they’re wonderful; you must
have a fantastic camera.’
He said nothing until dinner
was finished, then: ‘That was
a wonderful dinner; you must
have a terrific stove.’
                 – Sam Haskins

                           http://www.haskins.com/ImageShop/Image_Shop_60s/60s_Books_A.Image_01.html
Connect




         Blog: http://managewell.net

  Twitter: http://twitter.com/TathagatVarma

Presentations: http://slideshare.net/managewell

why agile?

  • 1.
    why agile? Tathagat Varma Sr. Director Yahoo!
  • 2.
    the world around us…yesterday!
  • 3.
  • 4.
    Other major OSand tools timeline https://en.wikipedia.org/wiki/Bugzilla https://en.wikipedia.org/wiki/Solaris_(operating_system)
  • 5.
    A typical supporttimeline http://itconvergence.blogspot.in/2012/10/oow-12-elison-extradata-oracle-r122.html
  • 6.
    As a contrast,what are consumer internet companies doing?  Continuous Integration -> Continuous Delivery -> Continuous Deployment  On ‘good days’, Flickr releases a new version every half an hour (Jun 20, 2005)  IMVU pushes a revision of code to the website every nine minutes (Feb 10, 2009)  The other day we passed product release number 25,000 for WordPress. That means we’ve averaged about 16 product releases a day, every day for the last four and a half years! (May 19, 2010)  A new version of Google Chrome now due every six weeks (Jul 22, 2010)  Facebook does code push twice a day (Aug 4, 2012)
  • 7.
    adoption @ netspeed!!! https://lh4.googleusercontent.com/-SQ1ugOgu8Ds/TimaHjH0VpI/AAAAAAAAApU/za0BhEjhoio/w402/google%2Bplus%2Bgrowth%2B20mil.png http://thesamerowdycrowd.wordpress.com/2011/12/26/a-moment-of-our-time/
  • 8.
    darwin at workon internet 2006 2009 http://www.flickr.com/photos/stabilo-boss/93136022/sizes/o/in/photostream/ http://rossdawsonblog.com/weblog/archives/2009/05/rapid_turnover.html
  • 9.
    yet, our productdevelopment sucks! http://blog.amplifiedanalytics.com/2011/07/musing-on-difference-between-successful-product-innovation/ http://www.nickblack.org/2009/10/how-brand-trust-affects-new-products.html
  • 10.
    Let’s understand the‘craft’ first…  Sheer joy of making things  Pleasure of things that are useful to other people  Fascination of fashioning complex puzzle- like objects of interlocking moving parts and watching them work in subtle cycles  Joys of always learning, which sprints from the non-repeating nature of the task  Delight of working in such a tractable medium The Mythical Man Month – Fred Brooks, 1975
  • 11.
    software development life cycles  Ad-hoc  Serial  Iterative  Incremental  Iterative/Incremental
  • 12.
  • 13.
  • 14.
    Waterfall Model  Wronglyinspired by assembly-line manufacturing processes of the day  Economics supported ―measure twice, cut once‖ leading to up-front planning and BDUF  Single-pass, sequential process with hand-offs and feedback loops between adjoining phases  Transition to next phase only upon completion of current phase
  • 15.
    Waterfall Software Development Limitations and Assumptions 1. Wrong analogy: Software development ≠ Production 2. Customers know EVERYTHING upfront and that requirement won‘t change 3. Legacy from the past: implicitly assumes CPU time is costly, so focuses on doing everything upfront to minimize ‗machine time‘ for trial and error 4. ―Wicked Problem‖: Designers and developers know how exactly how to build 5. Very long feedback cycles not suitable for today‘s pace of innovation Picture from http://damonpoole.blogspot.in/2009/07/traditional-development-game-of.html
  • 16.
    As a result,software is… Costly Buggy Late
  • 17.
  • 18.
    Holy Grail ofSoftware Development  Better: higher quality, more reliability, higher performance, more usable…  Faster: speedier development  Cheaper: no budget
  • 19.
  • 20.
    Preamble to Agile Movement Software Crisis, 1965-85: The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem. — Edsger Dijkstra, The Humble Programmer
  • 21.
    Software Crisis The causesof the software crisis were linked to the overall complexity of hardware and the software development process. The crisis manifested itself in several ways:  Projects running over-budget.  Projects running over-time.  Software was very inefficient.  Software was of low quality.  Software often did not meet requirements.  Projects were unmanageable and code difficult to maintain.  Software was never delivered.
  • 22.
    and the response? Frameworks,Standards and Certifications
  • 23.
  • 24.
  • 25.
    and sadly, noneof these came out of ‘process factories’… 2011 - • Instagram 2000-2010 •2010: Pinterest, SnapDeal, •2009: Square, Quora, Sina Weibo 1975-2000 •2008: Groupon, AirBnB, GoGo •2000: Baidu •1997: Yandex •2007: •1994: Yahoo!, Amazon, Dropbox, Zynga, Flipkart, InMobi, Hul NetScape, u, Tumblr, •1986: Pixar •2006: Twitter, SlideShare, Badoo •1984: Sybase Spotify •1983: Intuit, Borland, •1982: Sun, Symantec, Adobe, •2005: YouTube, Renren EA •2004: Facebook •1980: Informix •1979: EMC •2003: •1977: Oracle Myspace, Skype, Rovio, Gameforge, •1976: CA, Apple •2002: LinkedIn •1975: Microsoft •2001: StumbleUpon, Mail.ru
  • 26.
    Why?  Process: Long-leaddevelopment process ineffective in a dynamic and global world  Management: Command and control model unsuitable for fostering collaboration required to solve complex problems  Technology: Advancements in computers, compiler technology and debugging and testing tools greatly improved the economics of software development  Innovation: in the age of hyper-innovation, old processes were simply ineffective
  • 27.
    What is themost important part in these two machines? ―The Brakes!!!‖ They let you go faster…
  • 28.
  • 29.
    Advent of Agileand Lean Methodologies  1970: Royce critiques Waterfall and offers improvement ideas  1986: Barry Boehm proposes Spiral Model  1971: Harlan Mills proposes Incremental Development  1987: Cleanroom Software engineering  1991: Sashimi Overlapping Waterfall Model  1992: Crystal family of methodologies  1994: DSDM  1995: Scrum  1996: Rational Unified Process framework  1997: Feature Driven Development  1999: Extreme Programming Explained  2001: Agile Manifesto is born  2003: Lean Software Development  2005: PM Declaration of Interdependence  2007: Kanban-based software engineering  2008: Lean Startup  2009: Scrumban  20xx: Something new !?! (hopefully!)
  • 31.
    What is agilereally all about? • Empowered individuals • Collaboration Motivated • Democratic decision- Individuals making and transparency Self- • Shorter feedback cycle organizing • Manage changing x- functional priorities Teams • Increased productivity Agile Businesses • Higher ROI • Faster time to market • Better User Experience
  • 32.
    Why is itso hard? 52 39 34 % % % Organizational Resistance to Management Culture Change Support
  • 33.
    feedback loop inagile lifecycles
  • 34.
  • 35.
  • 36.
  • 37.
    Feedback Loops inTraditional Techniques vs. Agile Techniques
  • 38.
    Agile Development Value Proposition http://www.versionone.com/Agile101/Agile_Benefits.asp
  • 39.
  • 40.
  • 41.
    are small teamsmore productive? http://drewcrawfordapps.com/2.0/the-agility-of-small-teams/
  • 42.
    does colocation impactteam performance? http://sloanreview.mit.edu/the-magazine/2009-summer/50412/how-to-manage-virtual-teams/
  • 43.
    is small batchsize faster? http://www.andrejkoelewijn.com/wp/2011/06/30/is-team-productivity-a-responsibility-of-the-product-owner/
  • 44.
    Let’s build acar …and I need it delivered… next week!
  • 45.
  • 46.
    The Wikispeed Process  At WIKISPEED, some of our projects move more than 10,000% faster than industry norms because of our blend of Agile, Lean, Scrum, and Extreme Programming/Manufacturing practices.  Team WIKISPEED uses methods developed by the fastest-moving software companies. In fact, in many ways we have more in common with Google or Twitter than with GM or Toyota.  Manufacturing and old-thought software teams gather requirements, design the solution, build the solution, test the solution, then deliver the solution. In existing automotive companies, the design portion of that process alone takes 3 to 12 years, and then the vehicle design is built for 5 to 14 years. This means it is possible to buy a brand new car from a dealer and that car represents the engineering team's understanding of what the customer might have wanted 26 years ago!  Team WIKISPEED follows the model of Agile software teams, compressing the entire development cycle into one-week "sprints." We iterate the entire car every 7 days, meaning that every 7 days we reevaluate each part of the car and reinvent the highest-priority aspects, instead of waiting 8 to 26 years to upgrade.
  • 47.
    Wikispeed uses…  LeanSoftware Design: Use less stuff  XP: Pairing and Swarming  Agile: Reducing costs to make changes  Scrum: Clearly defined team roles and responsibilities  TDD: start with failing tests and develop solutions  OOP: contract-first development
  • 48.
    Recap  agile ≠Faster, but Sooner  agile ≠ No planning, but Adaptive Planning  agile ≠ More work, but ‗Done‘  agile ≠ No documentation, but Just Enough  agile doesn‘t just change the development process, but bring a radical change in organizational culture, leadership and management practices that is more in line with business needs and social values and norms of today
  • 49.
    It’s not aboutthe method! A photographer went to a socialite party in New York. As he entered the front door, the host said ‘I love your pictures – they’re wonderful; you must have a fantastic camera.’ He said nothing until dinner was finished, then: ‘That was a wonderful dinner; you must have a terrific stove.’ – Sam Haskins http://www.haskins.com/ImageShop/Image_Shop_60s/60s_Books_A.Image_01.html
  • 50.
    Connect Blog: http://managewell.net Twitter: http://twitter.com/TathagatVarma Presentations: http://slideshare.net/managewell