Lean data center, a telco experience by Andrea Pinnola, Telecom Italia

6,017 views

Published on

How does lean apply to an IT environment and for what results? Discover the lean data center case study presented by Andrea Pinnola from Telecom Italia at the Lean IT Summit 2013.
More Lean IT presentations and videos on www.lean-it-summit.com

Published in: Technology, Business
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
6,017
On SlideShare
0
From Embeds
0
Number of Embeds
3,903
Actions
Shares
0
Downloads
79
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Lean data center, a telco experience by Andrea Pinnola, Telecom Italia

  1. 1. Copyright © Institut Lean France 2012 3 & 4 October, 2013 Paris, France European Lean IT Summit 2013 Lean Data Center: A Telco Experience Andrea Pinnola - Telecom Italia
  2. 2. Copyright © Institut Lean France 2012 Agenda • • • • TI Background Bologna Data Center Operations Pilot Extending to the Delivery Value Streams Lessons Learned Lean Data Center: A Telco Experience - Andrea Pinnola 2
  3. 3. Copyright © Institut Lean France 2012 Telecom Italia Group Figures and Organization (*) Telecom Italia is the leading Italian Telecommunications and ICT provider, one of the most important Telcos players in LATIN AMERICA and one of the Top Ten global telcos players As of March 31th, 2013 Lean Data Center: A Telco Experience - Andrea Pinnola 3
  4. 4. Copyright © Institut Lean France 2012 Domestic IT Infrastructure Milano (Virtual Data Center) IDC Cesano Maderno IDC Rozzano VDCN Network 2,5 Gbps 155 Mbps DC Padova Torino DC Bologna Firenze DC Oriolo Romano DC Bari IDC Pomezia National Data Center Sites Local Service Centers Palermo • • • • • • 7 National Data Center Sites 4 Local Service Centers 24.000 Servers 21.000 TB Storage 20.000 TB Backup 12.000 TB Backup Transferred/month Napoli Organization Engineering & Delivery coordination plus 3 Data Centers Areas (North East, North West & Center South) Lean Data Center: A Telco Experience - Andrea Pinnola 4
  5. 5. Copyright © Institut Lean France 2012 Lean DC Bologna Pilot The case • Difficulty to answer to all the incoming requests • Feeling overwhelmed • Not clear work loads • Customers complains Focus on Operations activities: Reducing Problems and Improve Effectiveness Lean Data Center: A Telco Experience - Andrea Pinnola 5
  6. 6. Copyright © Institut Lean France 2012 Bologna Data Center Delivery & Operations Teams • Data Management:  Storage  Backup • Systems:  Windows e Linux  UNIX • Networking • Plant & Logistics Sponsorship from the Head of Infrastructure Head of DC Bologna willing to improve Lean Data Center: A Telco Experience - Andrea Pinnola 6
  7. 7. The Improvement Cycle Process Identification Activity Tracing Data Analysis Improvement Identification and Proposal Improvement Execution and Check Lean Data Center: A Telco Experience - Andrea Pinnola 7
  8. 8. Copyright © Institut Lean France 2012 Make Process & Problems Visible: e.g. Management of Backup Policies Over 1200 Backup Policies for Systems and Applications Requests to know Policies in place Request Documentation Rework Problems No access to the Policies DB Difficult Password Retrieval OOPS … No Data ? Problem! Lean Data Center: A Telco Experience - Andrea Pinnola 8
  9. 9. Make process visible: tracking requests Operation requests Tracking Track every Backup Policy Request received by Mail, IT System, Phone call, coffee chat … From 7-8 main activities / month to 31 activities tracked / week ! 100% 10 80% 8 60% 6 40% 4 Impl. Policy Informazioni 0% Modifica SW 20% 0 Fault 2 Percentuale cumulata 120% 12 Numerosità 14 Review of backup controls in order to prevent faults incidence Lean Data Center: A Telco Experience - Andrea Pinnola 9
  10. 10. Make process visible: tracking problems Disk Drivers Problems • Backup Faults: highlighted recurring errors on 4 drives • New (type of) Fault not described within the Fault Management procedures Lean Data Center: A Telco Experience - Andrea Pinnola 10
  11. 11. Copyright © Institut Lean France 2012 Gemba Walk: Critical cabling In one room: • Longer cable plug in • Old cables not removed • Difficulty and risk of faults during interventions Why ? • Continue escalating requests of new connections done in a hurry Lean Data Center: A Telco Experience - Andrea Pinnola 11
  12. 12. Copyright © Institut Lean France 2012 Flow of improvements proposals 40 By Team 30 20 10 0 106 40 30 20 By Type • All teams involved in a 5 months timeframe • Created one sharing point for problems, countermeasures proposal and approval • Weekly review meeting with the head of DC, Team Leaders and Bologna Facilitator 10 0 Lean Data Center: A Telco Experience - Andrea Pinnola 12
  13. 13. Copyright © Institut Lean France 2012 Actions and Results (Examples) Actions Results • Storage size controls contemporary to • Effectiveness of storage assignment and storage assignment reduced monthly reporting (4d to 2h) • Semi-automated procedure to OS patching • Reduction of OS patch time and of global delivery time of an updated server • Simplification of systems /servers password management • Eliminated password renewals delay • Reduced time to interventions • Backup policy requests simplified templates • Reliable policies update and reduction of info request • Enriched scripts to better signaling backup failures • Reduced errors and increased reliable operations People feeling change and the effects of their improvements 13
  14. 14. Copyright © Institut Lean France 2012 Results: Critical cabling recovery • Clean situation restored • New interventions simplified • Recovered fibers for 1 year of consumption (variations) Lean Data Center: A Telco Experience - Andrea Pinnola 14
  15. 15. Copyright © Institut Lean France 2012 Moving Forward: involving the other DCs • • • • Lean DCs Steering Committee Biweekly meetings to share improvements 3 Lean DCs Facilitators appointed Involved all the DCs Team Leaders • … move to a wider Lean initiative and joint DCs responsibility Lean Data Center: A Telco Experience - Andrea Pinnola 15
  16. 16. Copyright © Institut Lean France 2012 Moving Forward: the Delivery case • Motivation  Growing number of installations & requests  Long time to deliver  Need for better delivered quality • Three Cases on:  Infrastructure Creation  Virtual Machines Delivery  Market Infrastructure Delivery Workshops on Lean Basics and VSM Lean Data Center: A Telco Experience - Andrea Pinnola 16
  17. 17. Copyright © Institut Lean France 2012 Teamworking & Common Knowledge: the actual Value in Value Stream Mapping Rome Infrastructure Creation Virtual Infrastructure Delivery to External Market Main Goals: Lead Time Reduction and reliable end date Lean Data Center: A Telco Experience - Andrea Pinnola 17
  18. 18. Copyright © Institut Lean France 2012 5S in physical spaces • • • • Before Time spent to look for materials Disordered Workspace No space to act Access forbidden to the customer • • • • After Phase out of obsolete and not used Space to mount racks & shelfs Visual control of quantities Better Customer perception Lean Data Center: A Telco Experience - Andrea Pinnola 18
  19. 19. 5S on IT Shared Servers • - 50% of the “space quota” assigned to the Shared Server • - 83% of documents space, complete and accurate information mantained Lean Data Center: A Telco Experience - Andrea Pinnola 19
  20. 20. Copyright © Institut Lean France 2012 OPL: standardization and knowledge transfer People able to manage the application since their first access Lean Data Center: A Telco Experience - Andrea Pinnola 20
  21. 21. Results: Visual Mng & Server Farm Creation •Physical Flow •Information Flow • Organization ‘Flow’ Lead Time reduced by 15% Effort reduced by 5% Lean Data Center: A Telco Experience - Andrea Pinnola 21
  22. 22. Results: Virtual Server Delivery SERVICE REQUEST BACKLOG TREND SERVICE REQUEST AGING TREND Improvement Lead Time - 40% Savings 4 hours for each activity Quality - 50 % reworks on docs Lean Data Center: A Telco Experience - Andrea Pinnola 22
  23. 23. Copyright © Institut Lean France 2012 Results: External Market Improvement Lead Time - 50% Operation Tests Definition - 50% Elapsed Time Rework on project docs - 75% Effort Spent Organization One single person as DC reference assigned for each delivery Lean Data Center: A Telco Experience - Andrea Pinnola 23
  24. 24. Copyright © Institut Lean France 2012 Lessons Learned • Make Process and problems visible • Adapt Lean to IT (e.g. 5S in physical and ‘virtual’ spaces) • 80% of the knowledge already there (people are experts) • Act readily on problems to get people involvement • Management commitment from the start • Customers involvement after a while Lean Data Center: A Telco Experience - Andrea Pinnola 24
  25. 25. Copyright © Institut Lean France 2012 Acknowledgements • To all the colleagues of the IT Infrastructure Department that initiated and participated to the Lean DC initiative Lean Data Center: A Telco Experience - Andrea Pinnola 25
  26. 26. Copyright © Institut Lean France 2012 andrea.pinnola@telecomitalia.it Lean Data Center: A Telco Experience - Andrea Pinnola 26
  27. 27. Copyright © Institut Lean France 2012 More Lean IT videos and presentations on www.lean-it-summit.com

×