Distributed Middleware Factory

2,206 views

Published on

How to do large scale distributed software development

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,206
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
69
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Velkommen Det dere skal få høre er i dag er en historie som heter Distributed Middleware Factory Før vi begir i eventyret så skal vi presentere oss selv
  • Distributed Middleware Factory

    1. 1. Distributed Middleware Factory Ramón Arellano (Accenture) and Paul René Jørgensen (Telenor)
    2. 2. About us <ul><li>Ramón Arellano </li></ul><ul><li>Work experience </li></ul><ul><li>6 years of experience as a technical consultant from Accenture </li></ul><ul><li>E-mail </li></ul><ul><li>[email_address] </li></ul><ul><li>Paul René Jørgensen </li></ul><ul><li>Work experience </li></ul><ul><li>5 years of experience from Telenor </li></ul><ul><li>E-mail </li></ul><ul><li>[email_address] </li></ul>
    3. 3. Distributed Middleware Factory - definition <ul><li>Distributed </li></ul><ul><ul><li>In the geographical sense of the word, i.e. various locations spread around the world </li></ul></ul><ul><li>Middleware </li></ul><ul><ul><li>The communication layer that lets applications interact across hardware and network environments </li></ul></ul><ul><li>Factory </li></ul><ul><ul><li>A facility used to manufacture goods in an effective way, using standardized industrial processes and specialized equipment </li></ul></ul>
    4. 4. Distributed Middleware Factory – definition continued <ul><li>Multiple teams in different locations, producing software for system integration, on the same platform, using standardized architecture and tools </li></ul>
    5. 5. So what are we going to talk about? <ul><li>Middleware platform for Telenor Fixed Line Services </li></ul><ul><li>Metro - A distributed middleware factory </li></ul><ul><li>In this talk we’ll share our experiences from its evolution and some secrets from the past 6 years </li></ul>
    6. 6. Epochs in Metro history <ul><li>Epoch 0 – Medieval Age </li></ul><ul><li>Epoch 1 – Gunpowder Age </li></ul><ul><li>Epoch 2 – Enlightenment </li></ul><ul><li>Epoch 3 – Industrial Age </li></ul><ul><li>Epoch 4 – Modern Age </li></ul>
    7. 7. Epoch 0 – Medieval Age <ul><li>Year 2000 </li></ul><ul><li>Challenges </li></ul><ul><ul><li>Where is the vision for the future? </li></ul></ul><ul><ul><li>Use of the existing middleware platform is not mandatory </li></ul></ul><ul><ul><li>Kings and kingdoms and their point to point integrations </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    8. 8. The Metro Vision <ul><ul><li>All integration between systems for Telenor Fixed Line services to be implemented through Metro platform </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    9. 9. Metro platform goals <ul><li>To offer test and production environments to all development project </li></ul><ul><li>To enforce architecture and development standards </li></ul><ul><li>To supply a common set of core components </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    10. 10. Distributed development model <ul><li>The platform is managed and controlled by a small, centralized team </li></ul><ul><li>The Telenor employer selects software supplier </li></ul><ul><li>The software supplier performs the implementation on the platform </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    11. 11. Distributed vs Centralized <ul><li>Employer chooses software supplier </li></ul><ul><li>Development is done where the supplier is </li></ul><ul><li>Easier to get new developers up and running </li></ul><ul><li>Requires a stronger regime and documented standards </li></ul><ul><li>Platform team chooses software supplier </li></ul><ul><li>Development is done in-house </li></ul><ul><li>Easier communication between all teams involved </li></ul><ul><li>Does not scale so easily </li></ul>Platform Platform Employer Employer Employer Employer Employer Supplier Employer Employer Employer Employer Employer Supplier Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    12. 12. Epoch 1 – Gunpowder Age <ul><li>Year 2001 </li></ul><ul><li>Challenges </li></ul><ul><ul><li>The volume and complexity of Telenor </li></ul></ul><ul><ul><li>Common architecture </li></ul></ul><ul><ul><li>How do we organize SI-testing? </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    13. 13. Statistics <ul><li>In an average hour the platform is processing </li></ul><ul><ul><li>61 200 web requests from the internet </li></ul></ul><ul><ul><li>115 200 EJB invocations </li></ul></ul><ul><ul><li>2 880 000 database queries </li></ul></ul><ul><ul><li>65 MB log messages (1,5GB in a day) </li></ul></ul><ul><li>Volume </li></ul><ul><ul><li>Over 50 systems integrated </li></ul></ul><ul><ul><li>Average 200 deployments a year </li></ul></ul><ul><ul><li>Over 250 different developers from all over the world have written code </li></ul></ul><ul><ul><li>Metro platform team guides up to 30 teams at the same time </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    14. 14. Name a technology, and we’ve got it EBXML Tibco B2B SOAP, Proprietary HTTP/XML Web Services Proprietary XML (JItegra Java COM bridge) IIS / ASP J2EE, RMI, Corba Java Terminal, Screen Scraping, MQ, CICS IBM Mainframe (MVS) Siebel XML Siebel JDBC, ODBC, DB2Connect Oracle, Sybase, MySQL, DB2 Protocol Technology Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    15. 15. Technology at the base <ul><li>J2EE is the chosen technology </li></ul><ul><li>IBM WebSphere is the chosen application server </li></ul><ul><ul><li>Started off with IBM WebSphere 3.5 and J2EE 1.1 </li></ul></ul><ul><li>WSAD as IDE (based on Eclipse) </li></ul><ul><li>IBM MQ Server for messaging </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    16. 16. Layered architecture Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    17. 17. Informal roles <ul><li>Experience has shown us the importance of having roles in a team </li></ul><ul><li>The roles are more or less self imposed </li></ul><ul><li>Examples </li></ul><ul><ul><li>The hardware geek </li></ul></ul><ul><ul><li>The Maven guru </li></ul></ul><ul><ul><li>The support girl </li></ul></ul><ul><ul><li>The documentation dude </li></ul></ul><ul><ul><li>The tool guy </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    18. 18. Test Environments <ul><li>Implementation, building and unit testing on developer’s laptop </li></ul><ul><li>System integration testing on 60 blade servers running the full J2EE runtime environment </li></ul><ul><ul><li>One for each development project! </li></ul></ul><ul><li>Acceptance Test environment administered by the platform team </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    19. 19. Epoch 2 - Enlightenment <ul><li>Year 2002-2003 </li></ul><ul><li>Challenges </li></ul><ul><ul><li>We want to be more innovative </li></ul></ul><ul><ul><li>How do we get implementation projects to help develop the platform? </li></ul></ul><ul><ul><li>How do we organize our work and control the deliverables? </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    20. 20. The 80/20 rule <ul><li>Innovative and flexible leaders introduced the 80/20 rule </li></ul><ul><li>Developers were allowed to use 20% of their time to learn and play with new technologies </li></ul><ul><li>Highly motivating, and has undoubtedly been fruitful for Metro </li></ul><ul><li>Technologies such as multicasting (Log), FishEye (CVS), were introduced by experimenting </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    21. 21. Boiling soup on stone <ul><li>The platform team is responsible for setting the standards for many projects and developers </li></ul><ul><li>Dave Thomas's book, “The Pragmatic Programmer”, talks about “boiling soup on stone” </li></ul><ul><li>Introducing new technologies and standards by curiosity and envy </li></ul><ul><li>The whole package more valuable than each part </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    22. 22. Metro Roadmap <ul><li>Our common workflow and methodology for all projects </li></ul><ul><li>A set of phases with corresponding deliverables </li></ul><ul><li>Waterfall model (may jump backwards and iterate) </li></ul><ul><li>Each project gets their own supervisor responsible for QA </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    23. 23. Roadmap control in Jira <ul><li>We’ve got a tool to enforce Metro Roadmap </li></ul><ul><li>Atlassian’s inexpensive, but excellent tool Jira provides workflow functionality </li></ul><ul><li>Controls the flow and stores all document deliverables </li></ul><ul><li>It’s the platform team’s communication channel with development projects </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    24. 24. Epoch 3 – Industrial Age <ul><li>Year 2004-2005 </li></ul><ul><li>Challenges </li></ul><ul><ul><li>Lack of documented standards </li></ul></ul><ul><ul><li>New developers must get up and running fast </li></ul></ul><ul><ul><li>We feel we could be more efficient in getting new applications out </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    25. 25. Document everything <ul><li>Roadmap workflow </li></ul><ul><li>Architecture principles </li></ul><ul><li>Development environment setup </li></ul><ul><li>Naming standards </li></ul><ul><li>Step-by-step guidelines </li></ul><ul><li>Testing and debugging </li></ul><ul><li>Building and deployment </li></ul><ul><li>FAQ </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    26. 26. Documentation in Confluence <ul><li>Atlassian’s professional wiki, Confluence, for all documentation </li></ul><ul><li>Wiki: easily updated, everybody gets the current document version </li></ul><ul><li>Ported existing documents from MS Word </li></ul><ul><li>Also used for blogging news and alerts through mailing lists and RSS feeds </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    27. 27. We use Open Source <ul><li>Result of the 80/20 rule </li></ul><ul><ul><li>We’ve found free tools to improve our platform </li></ul></ul><ul><li>We love open source, and we give back </li></ul><ul><ul><li>Maven plugins </li></ul></ul><ul><ul><li>Confluence plugins </li></ul></ul><ul><ul><li>XRadar </li></ul></ul><ul><ul><li>Jira plugins </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    28. 28. Leading Star Example <ul><li>Leading Star Driven development </li></ul><ul><li>Improves application consistency </li></ul><ul><li>TelenorShop – a fictitious application </li></ul><ul><ul><li>It shows live examples of all our common components and frameworks in use </li></ul></ul><ul><li>Source code for all Metro applications is openly available internally </li></ul><ul><li>“Take a look at TelenorShop, and see how we’ve done it there” </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    29. 29. Automation through 3rd party tools <ul><li>Automatic builds with unit testing using Maven </li></ul><ul><li>Generating web-site with reports and JavaDoc using Maven </li></ul><ul><li>Continuous integration using CruiseControl </li></ul><ul><li>Mapping between data models using Dozer </li></ul><ul><li>Support request through Jira </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    30. 30. Automation through self-made tools <ul><li>Applying for deployment slots to acceptance test environment </li></ul><ul><li>Generating overview of dependencies between all J2EE applications on server </li></ul><ul><li>Searching logs in all environments </li></ul><ul><li>Updating application configuration </li></ul><ul><li>Backing up WebSphere Application server configuration </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    31. 31. Metro Certification <ul><li>A good factory scales easily </li></ul><ul><li>New resources must be brought up to speed – fast </li></ul><ul><li>No development without certification </li></ul><ul><li>J2EE is a prerequisite </li></ul><ul><li>Certification courses for </li></ul><ul><ul><li>Developers (2 day course) </li></ul></ul><ul><ul><li>Project Leads (1 day course) </li></ul></ul><ul><li>100 certified developers and project leads </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    32. 32. Other certification benefits <ul><li>Efficient development </li></ul><ul><li>Consistent solutions </li></ul><ul><li>Stable solutions </li></ul><ul><li>Reusable services </li></ul><ul><li>Better developer attitude </li></ul><ul><ul><li>“ Ah, so that’s why we have to...” </li></ul></ul><ul><li>Externals pay a course fee ($) </li></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    33. 33. Metro Certification content Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    34. 34. Epoch 4 – Modern Age <ul><li>Right now, right here </li></ul><ul><li>Challenges </li></ul><ul><ul><li>Middleware is only noticed when something goes wrong </li></ul></ul><ul><ul><li>How do we govern dependencies? </li></ul></ul><ul><ul><li>Deployments are held back due to poor code quality </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    35. 35. Commercialization <ul><li>We must learn how to better sell ourselves internally </li></ul><ul><ul><li>We want more applications on the platform and everyone to be aware of what Metro has to offer </li></ul></ul><ul><li>How </li></ul><ul><ul><li>Create a logo </li></ul></ul><ul><ul><li>Offer introduction presentations </li></ul></ul><ul><ul><li>Show off statistics of usage </li></ul></ul><ul><ul><li>Present at JavaZone </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    36. 36. Service Level Agreements <ul><li>SLAs between API owners (the kings) </li></ul><ul><ul><li>Wanted since the medieval age! </li></ul></ul><ul><ul><li>No access to remote API without a signed SLA </li></ul></ul><ul><li>It will give us valuable information about inter-application dependencies </li></ul><ul><li>We must work on enforcing this in the modern age! </li></ul><ul><ul><li>Requires a new tool (Any suggestions?) </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    37. 37. Code Quality <ul><li>Step-up the code QA work </li></ul><ul><li>Make it easier for the reviewers to focus on code that is being altered </li></ul><ul><ul><li>Reviewers are notified immediately on CVS commit </li></ul></ul><ul><li>We’ll introduce Crucible into the review process </li></ul><ul><ul><li>Defines a workflow, creates an arena for dialog </li></ul></ul>Medieval Age – Gunpowder Age – Enlightenment Age – Industrial Age – Modern Age
    38. 38. Architects must write code <ul><li>“ Real insight comes from active coding” </li></ul><ul><ul><li>Practices of an Agile Programmer </li></ul></ul>
    39. 39. Summary <ul><li>Key tools </li></ul><ul><li>Key lessons </li></ul>
    40. 40. Key tools <ul><li>Confluence (wiki) for documentation </li></ul><ul><ul><li>No delay from writing to publication </li></ul></ul><ul><li>Jira </li></ul><ul><ul><li>No more forgotten issues in inboxes all over the place </li></ul></ul><ul><ul><li>Issue management and process (roadmap) workflow </li></ul></ul><ul><li>Maven </li></ul><ul><ul><li>Stable and proven build system </li></ul></ul><ul><li>Eclipse based IDE </li></ul><ul><ul><li>Allow only one IDE </li></ul></ul><ul><ul><li>Be an expert to support the developers </li></ul></ul>
    41. 41. Key Lessons <ul><li>Easy to understand workflow for everybody – Roadmap </li></ul><ul><li>Documented standards and guidelines – Wiki </li></ul><ul><ul><li>And a way to teach them (Certification) </li></ul></ul><ul><li>Decision from the top : All integration between systems to be implemented through the platform </li></ul>
    42. 42. Any questions? paul-rene.jorgensen@telenor.com / ramon.arellano@accenture.com

    ×