Recommended
More Related Content
What's hot
What's hot (18)
Similar to DevOps CTO Masterclass Tools Methodologies
Similar to DevOps CTO Masterclass Tools Methodologies (20)
More from Cyber-Duck
More from Cyber-Duck (20)
Recently uploaded
Recently uploaded (20)
DevOps CTO Masterclass Tools Methodologies
- 1. DevOps CTO Masterclass October 2020 Presented by #DevOpsDuck
- 2. Agenda House rules Meet the experts / intro to Cyber-Duck What is DevOps? When do you it? (case study) Team culture DevOps tools and methodologies Summary Q&A
- 3. House Rules The webinar will run for 1 hour. The webinar will be streamed on Zoom and YouTube. There will be time at the end for Q&A session.
- 4. House Rules To submit a questions: Zoom - Open the chat window by clicking the Q&A button located at the bottom of your screen YouTube - Please use the chat box on the right of your screen You can also post questions on Twitter after the webinar using the hashtag #DevOpsDuck
- 5. Quick intros
- 6. Sylvain Reiter Client Services Officer
- 8. Indira Reyes Senior Project Manager
- 10. Together, we transform digital For over 15 years, clients have trusted us to transform complex websites, content and applications. Our ISO certified UX and Web & Mobile Development methodology includes user research and change management. We also undertake data and technology auditing and strategy. Established in the UK, our remote-first team gives us unlimited scale, breadth and depth.
- 13. What are users saying?
- 14. What is DevOps?
- 15. Definition Set of tools, principles and processes Unifies the entire lifecycle Surrounds development team Integrates with other departments Agile by nature
- 16. History Very vague ;-) Rise of cloud-base technology Importance of software Need to improve the speed to production… … whilst maintaining quality
- 17. The DevOps loop
- 18. When do you need DevOps?
- 19. Background story I am in charge of the development team, but need to manage: Internal IT stakeholders External development agency Product owners from the Marketing team I need to build trust with all
- 20. Building trust Trust is earned with confidence but I was facing: Many discrepancies in environments (config, jobs…) Delays in releases Technology used incorrectly Simple questions turned into complex issues
- 21. Making sense of the tech Ubuntu servers (v14 to v18, patches) Webservers (Apache / Nginx) Databases (different versions, configurations) PHP versions (v5.6 to 7.2) Drupal CMS (mix of versions 6/7/8)
- 22. Our digital estate was creaking Challenges in applying Drupal security patches Bug fix taking longer Difficult to add new features Everything became very inefficient
- 23. DevOps to the rescue The solution came in the form of DevOps. Cyber-Duck was onboarded to: Review the whole lifecycle Rebuild a plan with the marketing team Take over the code repo and deployment pipeline Migrate to AWS Follow Agile sprints to improve the platforms
- 24. Team culture
- 25. Culture & Communication Everyone needs visibility on the whole process Client-side on requirements and UAT, progress throughout Internal project team Internal stakeholders Sharing knowledge - EVERYONE owns the delivery process
- 26. DevOps in Sprints Rapid iteration in Sprints Need some upfront work to configure the entire pipeline Once tools are set, alerts and notifications brings visibility to the entire team
- 27. What success looks like “Deployment celebrations should be about the value of new features, not joyful relief that nothing went wrong” Rebecca Parsons, CTO @ ThoughtsWorks
- 29. A whole sea of options Image source : Continuous delivery tool landscape · James Bowman
- 30. Plan
- 31. Plan - Requirements Identify large modules Document user stories Focus on user value Plan early
- 32. Plan - Infrastructure List of services Redundancy level Costs options Scaling requirements
- 34. Code
- 35. Code - Repository Shared repository Default settings Readme templates
- 36. Code - GitFlow Naming convention Setup rules Permissions on PR Image source : Atlassian
- 37. Code - Local config Docker setup Reduces compatibility risks Faster staff onboarding
- 39. Build
- 40. Build - Automated pipeline Install dependancies Compile assets Consistent environments
- 41. Build tools
- 42. Test
- 43. Test - Continuous Integration (CI) Triggered on code updates Build and run automated tests Automated regression Unit to integration testing Improve developer productivity Find more bugs Deliver updates faster
- 44. Test - Browser testing Automated tests in browser Compare screenshots
- 45. Test tools
- 46. Deploy
- 47. Deploy - Methods Continuous deployments (CD) Infrastructure as code Platform as a Service (PaaS) Testing sandboxes Consistent environments
- 48. Deploy - Release plans Git workflow Run book Rollback plans Change control
- 49. Deploy tools
- 50. Operate
- 51. Operate - Infrastructure monitoring Triggers and notifications Review performance Security
- 52. Operate - App monitoring Application errors Aggregate logs
- 53. Monitoring tools
- 54. To conclude
- 55. 1. Plan & Communicate Automate & Measure Continuous improvement 3 take-aways
- 56. If not done so, download our DevOps white paper on: www.cybrd.uk/dvops_wp If there is anything that triggered your attention and would like to know more, ping us on #DevOpsDuck Next steps
- 57. Gareth and Sylvain are available to chat about your specific DevOps or technical challenges. We’d love to discuss them over a virtual coffee ☕️. You can reach us at... Email: hello@cyber-duck.co.uk Twitter: @cyberduck_uk LinkedIn: linkedin.com/company/cyber-duck What is your DevOps challenge?
- 58. Q&A Session Let’s answer your questions from Zoom and YouTube Email: hello@cyber-duck.co.uk Twitter: @cyberduck_uk LinkedIn: linkedin.com/company/cyber-duck
- 59. Thank you Download our white paper! cybrd.uk/dvops_wp
Editor's Notes
- Dan W
- Dan W
- Dan W Quickly cover house rules Questions coming in from YouTube and ZOOM Q&A Please submit your questions as we’ll answer in the Q&A at the end. You can also chat to us on Social media (linkedin, twitter and facebook) - on Twitter you can either @mention us or use the DevOpsDuck hashtag
- Dan W Please submit any questions you may have throughout the webinarQuestions coming in from YouTube and ZOOM Q&A Please submit your questions as we’ll answer them as we go and in the Q&A at the end. You can also chat to us on Social media (linkedin, twitter and facebook) - on Twitter you can either @mention us or use the DevOpsDuck hashtag
- Dan W
- Dan W Sylvain is the co-founder and Client Services Officer at Cyber-Duck. Coming from a strong tech background, he turns our clients requirements into digital solutions that are secured, scalable and compliant. He used to manage our VPS servers back in 2006 when everything was done manually and has supported our DevOps journey to where we are today!
- Dan W Matt is joining us from Fitch Learning, part of the Fitch Group. He is one of our clients and we started working together almost 1y ago. He will share some of the painpoints he experienced and how adopting DevOps has helped him have better peace of mind!
- Dan W Finally Indira is our senior project manager and she will explain how she uses DevOps to deliver better projects for clients, on time and on budget. Enable handoff to SYLVAIN OR DANNY and give screen control access.
- Dan W Gareth is our Technical Director, at the company for 10 years. He oversees the development, quality assurance and DevOps teams as well as defining our tools, processes and training to ensure overall quality of what we deliver.
- DAN : (handoff to Danny Bluestone, CEO of Cyber-Duck) Danny: - Intro to Cyber-Duck - who we are, what we do.
- Danny Cyber-Duck consolidate Sport England’s multiple websites, with a wealth of content and audiences. Our dev DevOps role involved setting up multiple Sport England applications on AWS. We utilised automation tools such as Terraform and Buddy pipelines.
- Danny College sites need to be modernised to reduce the overheads of maintaining legacy systems. Improve website accessibility by meeting WCAG 2.1 guidelines. Provide Enterprise Search Functionality (optional work that may not proceed through this contract). The DevOps work included automated deployment to GOV.UK’s Platform as a Service including monitoring
- DANNY: - Talk about work done for Cabot Cabot Financial is a group of leading credit management service organisations covering several services, including debt purchasing and business process outsourcing. They required new digital channels to give users more control of their experience DevOps - Multiple web applications configured via Terraform in Microsoft Azure utilising Azure DevOps pipelines and Azure’s App Services
- Danny - We’ve been running a poll to see what people are interested in hearing about with over 2000 responses, and we hope you enjoy the webinar. - Handoff to Sylvain to tell us more about the DevOps process.
- SYLVAIN So let’s start with the basics of What is Devops?
- DUNCAN - I talk quite a bit after I cover these 3 sections. Hold fire on the next slide.
- DUNCAN
- DUNCAN The migration of data from your old site to your new site starts with the new Drupal 9 site. You place your old site’s database alongside the database for your Drupal 9 website and perform the migration direct from one database to the other. There is a web-based user interface for this, but to prevent time-outs a command line approach is recommended using Drupal’s drush command line interface.
- Sylvain Dev Ops comes from Development Operations and describes a set of tools, principles and processes to unify the entire software development lifecycle. It usually surrounds the tech teams with programmers at the heart of it, but the critical part is the link and interactions with other departments: project managers, business analysts, testers and the IT Operations team usually in charge of the infrastructure. DevOps is strongly embedded in the Agile methodology, it follows best practices of rapid iterations and continual improvements. Where agile bridges the communication gap between clients and developers, DevOps can be seen as the next extension, bridging the gap between developers and IT operations teams.
- Sylvain A bit vague - term first coined at a conference in Belgium in 2009 It’s at that time that the rise of cloud based technologies enabled new systems to be put in place. who remembers that time when we had to wait 2 weeks to provision a new dedicated server in a datacenter (?!), In the last decade, we can now spin up new VM in seconds. That opens a lot of new possibilities. Remember, the main goal of DevOps is to reduce the time between requesting a change to a system and that change being deployed into production. With the importance of software running mission critical applications, we all need to ensure the highest possible quality For some part, it means removing human errors and this is where automation comes in. Also important to touch on security and compliance which are a key part of the process.
- Sylvain You might have heard of terms such as Continuous integration / continuous delivery. It’s all in this notion of permanent loop highlighted here, never finished ;-) Even when we talk about automation in DevOps, with so many moving parts, there is always plenty of work for us to do, improve, optimised, upgrade…! As we have seen earlier, to ensure quality, everything in that chain needs to be optimised This Loop is also known as toolchain . It covers several disciplines, involve multiple teams… We will explore in more details in a few minutes.
- Sylvain But first, let’s look at a real life case study from the perspective of one of our client, Matthew from Fitch Learning. Matt, over to you.
- MATT from Fitch
- MATT from Fitch
- MATT from Fitch On top of the confidence, I had to manage the tech… A simple “LAMP” stack? Sounds simple enough! But it was far from that: [list the slide]
- MATT from Fitch Requiring more and more expert support and interventions not just with the normal day-to-day operations but with the fast changing digital landscape Requiring more and better security hardening along with monitoring to keep the services operational The whole Marketing team required upskilling to help with spreading this workload
- MATT from Fitch [list on the slide] And in the future planning the upgrade from Drupal 7 to Drupal 8
- Sylvain Like with most things in IT, the tech behind the scene does not really matter, it just needs to work. Everything we do surrounds people. The software is SERVING us but it’s real persons who make things happen and need to collaborate. Let’s hear from Indira, our most senior project manager, see how she integrates DevOps in her day to day job
- Indira As PM, it’s my role to ensure good delivery of projects, on time and on budget It starts with good communication at all levels, both with the clients and internal teams The DevOps loop actually mirrors cycle activities from brief/requirements > Scoping > Planning > Developing > Testing > Deploying > UAT > Sign-off Sometimes have to “fight” with clients to explain why we have such processes: why we do things a certain way, why we follow agile rituals for good reason, and why we bring in multiple people or document these processes to avoid silos and improve communication Ensure good delivery at the end requires a good brief. We worked a lot recently on improving user stories template, covering acceptance criteria. It’s a collaborative team review, shared knowledge At the end of the day, it’s not one person’s job to just press the launch button. It’s important to build a culture where EVERYONE owns the delivery process from concept through dev, UAT, launch and production monitoring.
- Indira The entire DevOps chain actually happens in ONE sprint, and one environment. Making sure the quality you want live you have it at every stage of the cycle (Sometimes tricky on 2-weeks sprint?) On a new project, I work upfront (during UX stages) to set up the scene, book the developers and server team to configure Once automated, some of those tests are fully automated, generate reports, send alerts so we can all keep track of what is going. Gareth later on will give you a few pointers and tools on how to best approach these. “Increased communication and collaboration in an organization is one of the key cultural aspects of DevOps. The use of DevOps tooling and automation of the software delivery process establishes collaboration by physically bringing together the workflows and responsibilities of development and operations. Building on top of that, these teams set strong cultural norms around information sharing and facilitating communication through the use of chat applications, issue or project tracking systems, and wikis. This helps speed up communication across developers, operations, and even other teams like marketing or sales, allowing all parts of the organization to align more closely on goals and projects.”
- Indira Our common goal is to enjoy releases… read quote from Rebecca Parsons is ThoughtWorks' Chief Technology Officer.
- Sylvain Thanks Indira. Let’s now look in more details about the various tools and how we apply them
- Sylvain It can be very intimidating - some tools are very specific for one piece of the puzzle Some others try to encapsulates several steps of the process Different configurations format Overall you need the information to flow through that loop. Like we have seen with the team, avoid silos of information so pick the tools that can talk to each others. Need to understand how things work before you can automate them. All of us at CD have done it manually on physical servers 10y ago, easier to pick up now! Go with what your team is comfortable with, or work with an agency like us who takes care of the config and you see the results on dashboards for your board reports
- Sylvain First on the loop…
- Sylvain Identify large modules / components. Again, its critical to spread knowledge and domain expertise for future team members who might join the project at a later stage Break down user stories This is one of the most difficult but important step in the process as serve as specification of what is build, drives the entire process up until sign-off post deployment “make sure we delivered what was asked” Some lessons learnt; allow enough time BEFORE a sprint starts, have enough tech details for estimation, clarify UX as part of previous spike…
- Sylvain Break down large module into user stories - collaborative with the client, business analyst role but also plan UX, QA acceptance criteria. Dev need to know what to build, avoid any guesswork or assumptions.
- Sylvain We’re not going to get into a lesson on Scrum or Agile but a backlog is a key components of the DevOps chain When developers work on features, they need to be priorities with what will bring the most value, as this is what will be DEPLOYED next It’s all good to have great user stories, but you also need to “make them move” through the process. This is when the use of kanban or Agile board helps gain visibility of progress and status. Make sure every task is assigned to someone Very visual to identify blocker, quick drag & drop to change status, very useful during the agile standup. Again, importance on communication and sharing the plan: everyone from clients to internal stakeholder to see at a glance what is going on, who is working on what
- Sylvain It’s all good to have great user stories, but you also need to “make them move” through the process. This is when the use of kanban or Agile board helps gain visibility of progress and status. Make sure every task is assigned to someone Very visual to identify blocker, quick drag & drop to change status, very useful during the agile standup. Again, importance on communication and sharing the plan: everyone from clients to internal stakeholder to see at a glance what is going on, who is working on what
- Sylvain Tools like CloudCraft or draw.io helps you map out the server, database and other services where your code would be deployed This is validated by InfoSec and IT teams for redundancy, avoiding single point of failure Need to balance redundancy VS the running costs, looking a Recovery Time Objective and if we use multi-availability zone or multi-regions
- Sylvain Tools like JIRA, Asana, ClickUP, Teamwork, Aha, Azure Devops… there are plenty of backlog management tool but we find JIRA and ClickUP the most useful, as they integrate with our code repository.
- Sylvain Next on the loop…
- Sylvain Next in the loop is manage the source code. For team collaboration, the standard is git repository Allows multiple developers to work on the same application. Includes some default settings and documentation in ReadMe files
- Sylvain Once you had the repo setup, we recommend following some-version of gitflow. Gitflow defines a strict branching model designed around the project release. It is ideally suited for projects that have a scheduled release cycle, with multiple team members working on build new “features”. [ … Explain each one … ] We can also set permissions to ensure a lead reviews each PR before it can be merged to Master
- Sylvain With so many version of software, multiple stack, multiple version of PHP/MySQL, Redis… as Matt explained earlier. This can be problematic as the code you write might be fine locally but not on your colleague’s setup or the hosting server. To resolve this, we use containers such as docker to have the shared config as part of the code
- Sylvain Multiple versions of the tools If you are on a full MS Azure stack, Azure repos might makes sense as you have closer integration with the rest of their services. Or CodeCommit on AWS. However, both BitBucket and GitHub are the leading platform with integrations to hundreds of other tools For environments, you can use local tools like MAMP or framework specific but Docker allows you to really mirror a production server so would be our recommendation. There are some docker alternative like ROCKET but Docker is the most popular and well maintained online.
- Sylvain Over to Gareth who will take you through the next few exciting phases ;-) Gareth …
- Gareth Used to build locally Install dependancies - composer, npm packages Compile assets - compress images, build JavaScript components, minify CSS. Consistent environment - Developer machines can all be set up differently so avoid compiling there
- Gareth
- Gareth
- Gareth Continuous integration is the process of building and testing your application. Tools can trigger tests and automated feedback to the team on success or failure. Automated tests can be formulated in a number of different ways from small units of code to full features. As your application gets more complex testing all previous functionality is important. Free up developers from manual tasks which reduces human error and improves productivity which in turn means delivering updates faster to customers
- Gareth For web projects testing in browsers and devices is important. QA engineers have a variety of tools, mainly based on a Selenium WebDriver to perform such tests in parallel. Screenshot comparison can be useful for testing variances on pages between releases. Automated video playback of issues without having to reproduce manually.
- Gareth
- Gareth
- Gareth Deploy: Hosting setup: no more build manually as if goes wrong or scale, you need infra as code: terraform, cloudformation, Azure pipeline, chef, puppet, ansible or from the cloud platform (Heroku, PaaS). Auto deploy on merge approval in code, no manual step. Create sandboxes to test in isolation by sharing a link (disposable cloud, don't need a testing/staging env at night) Release: gitflow with branching but more comms plan: runbook, rollback, approval / CAB, change control
- Gareth Organise your code using the tools available. Have a well documented work flow which the team follow. Tag your production releases. Create deployment a run book and share with all parties in advance. Evolve from previous deployments to improve your plans and communications. If things don’t work out, what is your plan to back out of a release. Use the tools for creating change control documentation
- Gareth
- Gareth
- Gareth - Uptime - nagios, datadog, AWS cloudwatch. Setup alerts and triggers before it goes down. Need a support process and SLA to handle. - Performance Optimisation: cost saving, scaling up & down (usage or timing) - Security: regular pen-test. on release or major changes. Scan on docker containers. Use 3rd party pro. Alerts on CMS patches. Policy as code,.
- Gareth - Application errors - More than just homepage working, lots can go wrong behind the scene, even without error noticing or not reporting, affect their experience. Know before your users report the issue or in most cases go to a competitor’s solution.
- Gareth
- Sylv We are wrapping up now
- Sylvain Through automation, the whole team can be informed on what is happening on the entire product lifecycle
- Sylv The goal of using all these tools is increasing collaboration through communication Use the right tools to automate the alerts and measure performance The ultimate aim is for everyone to be aligned on the goals of the product and increase efficiency. Never “done”, mindset and culture change will be the biggest hurdle but it’s time to start your journey
- DAN B or Dan W: - handoff to Danny - We will be running several other events and releasing new materials in the coming weeks, so If there is anything that triggered your attention and would like to know more - Danny talks about concluding remarks, how to get in contact with CD and, download the white paper and join the next webinar.
- DAN B or Dan W: - Get in touch with us if you want to speak through a specific business challenge you have. - Mention consultancy / working with CD
- DAN W: (send/show Qs to DanW to ask panelists) - Questions from YouTube and Zoom