DevOps aims to bring development and operations teams closer together through automation, shared tools and processes. Automating builds improves consistency, reduces errors and improves productivity. Common issues with builds include them being too long, handling a large volume, or being too complex. Solutions include improving build speed, addressing long/complex builds through techniques like distributed builds, and using build acceleration tools. Automation is a key part of DevOps and enables continuous integration, testing and deployment.

1. Introduction to
DevOps
&
Build Automation
Update
WAYI
27 February 2017

2. Introduction to
DevOps

3. How can we release on-time?
How can we have a stable
release?
How can we address the
problem before end users get
impacted?
Top Concerns

4. Embrace it.

5. DevOps = Dev + Ops
Bring Dev & Ops closer together
- Ideally, unified
Culture and Mindset
Set of practices centered on
automation
- Reliability
- Repeatability
- Traceability
- Minimal or no manual intervention

7. What is
happening in
software delivery
industry right
now?

8. DevOps adoption increased from 66
percent in 2015 to 74 percent in
2016.

9. DevOps adoption is strongest in the
enterprise (81 percent of enterprises
adopting DevOps compared to 70 percent
in SMBs).

10. Enterprises are adopting DevOps from the bottom
up: Adoption is by projects or teams (29 percent)
and business units or divisions (31 percent) vs.
company-wide (21 percent).

11. Based on a 2016 Puppet Labs study Highly effective
DevOps organizations have the following impact on the
business.
Source: 2016 State of DevOps Report, Puppet Labs
200X
Frequent
Deployments
24X
Return to
Normal
Operations
3X
Change
Failure
22X
Unplanned
Work and
Rework
50%
Security
Remediation
Time
High performing teams offer secure and reliable products, services and
capabilities quicker, faster.

12. 2017?
This year, DevOps adoption is expected to
increase up to 85%.

13. Automation is a
part of DevOps
patterns
“Infrastructure as Code”

14. Standardize development, test and
production environments.
Effectively deploy and manage cloud
resources.
Eliminate error-prone, time-consuming
manual tasks.
Improve cooperation between
development and operations.
Implement automated release pipelines.

16. Benefits of automation
Automation enables velocity, scale, consistency and
feedback. All of these qualities are of a piece. Any one
depends on the other three – for example;-
 you can’t scale unless you are able to quickly add servers
with consistent configurations.
 you can’t get feedback automatically without being able to
support different real-time testing and monitoring
techniques.
 you can’t respond to feedback effectively unless you have a
high-velocity way to deliver incremental changes safely.

17. Why Should I
Automate?

18. Repeatability
Scripts can be repeated and unless your
computer is having a particularly bad
day, you can be reasonably certain that
the same instructions will be executed in
the same order each time the same
script is run.

19. Reliability
Scripts reduce changes for human error.

20. Efficiency
Automated tasks will often be faster than
the same task performed manually.

21. Testing
Scripted processes undergo testing
throughout the development cycle, in
much the same way the system code
does. This greatly improves changes for
successful process execution as the
project progresses. Automated scripts
eventually represent a mature, proven
set of repeatable processes.

22. Versioning
Scripts are artifacts that can be placed
under version control. With manual
processes, the only artifacts that can be
versioned and tracked are procedure
documents. Versioning of human beings
– the other factor in the manual process
equation – is unfortunately not
supported by typical source control
system.

23. Leverage
Another big benefit to automating is that
developers and testers can focus on the
areas where the add real value to a
project – developing and testing new
code and features – instead of worrying
about the underlying development
infrastructure issues.

24. What Should
I Automate?

25.  Build and deployment of the system
under design.
 Unit test execution and report
generation.
 Code coverage report generation
 Functional test execution and report
generation.
 Load test execution and report
generation.
 Code quality metrics report
generation.
 Coding conventions report
generation.

26. Importance of
Automated
Testing

27. With automated testing, we are
able to catch problem earlier,
before they impact our release
cycle. The earlier we catch a
problem, the easier and less
expensive it is to fix, and this is why
testing is such an important part of
DevOps practice.

28. Architecture
design
Component
design
Coding Unit
testing
Integration
testing
Acceptance
testing
The V-diagram, common in software engineering, illustrate this:
As you can see, each kind of testing activity (on the right side of the V)
checks a particular phase of development (shown on the left side). The
cost of rework rises as defects are discovered later in the project. It’s
better to begin testing at the vertex of the V, with unit tests. Catching a
defect during a unit test is much easier than trying to fix it when it’s
being tested with other components that might make it difficult to
discover where the actual problem lies.

29. DevOps
Toolchains

30. DevOps Toolchain

31. 1. Code – Code development and review,
version control tools, code merging
2. Build – Continuous integration tool, build
status
3. Test – Test and results determine
performance
4. Package – Artifact repository, application
pre-deployment staging
5. Release – Change management, release
approvals, release automation
6. Configure – Infrastructure configuration
and management, Infrastructure as Code
tools
7. Monitor – Application performance
monitoring, end-user experience

32. Examples of DevOps Tools
Continuous Integration (CI)
- Jenkins, Travis, TeamCity
Configuration Management (CM)
- Puppet, Chef, Ansible, CFEngine
Continuous Inspection
- SonarCube, HP Fortify, Coverity
Containerization
- Vagrant, Docker
Virtualization
- Amazon EC2, VMWare, Microsoft Hyper-V

33. Build Automation
Update

34. Most modern development teams have set up some
form of build automation. At minimum, build
automation involves check out from source control,
compilation and linking, and packaging the resulting
binaries and other necessary files. These packages
can be then deployed and tested in downstream
processes. If done correctly, build automation
reduces manual labor by developers, ensures builds
are consistent and complete, and improves product
quality.

35. Critical Issues with Build Automation
1. Long Builds
2. Large Volume of Builds
3. Complex Builds

36. 1. The Problem: Long Builds
Why it matters?
 Developer wait time
Most developers spend between 2-10 hours per week waiting for builds to
complete – lost time
 Context switch
developers switch task, lose context related to the problem they were
working on – reduces productivity
 Build bottleneck
Developers sync changes into the main code line and frequent builds are
needed
 Product quality
The less frequent the build, the less issues can be fixed and verified befre the
release – hurting product quality

37. 2. The Problem: Large Volume of Builds
Why it matters?
 Limited build resources
Limited access to build servers during specific time windows or the servers
are often overloaded and builds will take much longer – hurting productivity
(especially practicing Agile)

38. 3. The Problem: Complex Builds
Why it matters?
 Complex builds are brittle
Interactions between many different components often lead to manual error,
broken builds and worse
 Extensive manual efforts
Executing a complex build and delivering its results requires a substantial
manual effort
 Incremental builds are difficult
An incremental run could break the build due to partially specified
dependencies
 Legacy components and fear of change
Complex builds tend to have legacy components written years ago by staff
members who are no longer at the organization
 Complex builds are long
As the complexity of the build increases, so the time it takes to run

39. What Can Be Done?

40. Solution: Improve build speed:
1. Upgrade our build server
2. Running builds in parallel on one machine
3. Build avoidance / incremental builds – running only those parts of the build
that have changed
4. Using a RAM disk – moving the entire build operation to RAM to eliminate
disk I/O
5. Precompiled headers – precompiling shared dependencies that changes less
frequently
Solution: Address long and complex build:
1. Distributed builds – parallelizing build across a cluster of build servers
2. Manually partitioning Makefiles – breaking the build into smaller components
3. Optimizing Makefiles – rewriting Makefiles to make them run more efficiently
4. Unity builds in C/C++ - combining source files into one file to reduce file
access
Solution: Installing free build acceleration tool – e.g. Huddle from Electric Cloud

41. Version Control
- Team Foundation Version Control (TFVC)
- Git
- Branching & Merging
Project Management
- Scrum, Agile
- Defect Tracking, Bugs
QA Testing / Test Case Management
Automated Builds
Automated Release Pipelines
Team
Foundation
Server

42. Source
Code
Build
automation
Executable
Code
Deployment
automation
Target
environment
Target
environment
Target
environment

43. Source
Code
Build
automation
Executable
Code
Deployment
automation
Target
environment
Target
environment
Target
environment
Team Foundation Server
 Maximize automation with scripts and custom tools.
 Include automated testing.
 Provide solid process and documentation in our
software delivery practices.

44. Advantages of build automation to
software development projects:-
 A necessary pre-condition for continuous
integration and continuous testing
 Improve product quality
 Accelerate the compile and link processing
 Eliminate redundant tasks
 Minimize “bad builds”
 Eliminate dependencies on key personnel
 Have history of builds and releases in order
to investigate issues
 Save time and money – because of the
reasons listed above.

45. Conclusion

46. Future GPMT
DevOps
Ecosystem
Workstations - On-Premises
DEV TEST QA
Environments - On-Premises
Build&Test
Build/CI
Test
Develop
Developer
Workstation
Team
Collaboration
Deploy
Release
Monitor&Learn
Monitor
ALMServices - On-Premises
TFS
Monitoring- On-Premises

47. DevOps Orchestration
Reduce costs and improve productivity and
predictability by standardizing Dev and Ops
toolchains
DevOps teams like to move fast, and ship
frequently. Individual teams often depend on
(and are sometimes fiercely loyal to) a unique,
curated toolchain to help them get things
done. But many organizations want to
consolidate tools to gain economies of scale
and allow cross-team sharing. How can they
orchestrate all of these tools in an efficient,
auditable way.

48. Deployment Automation
Automate deployments so App Support and
Ops teams can keep up with the page of Agile
delivery
Consistently deploying applications into
standardized environments is one of the best
ways to achieve application rollout success,
but most teams still rely on hard-coded scripts
and manual instructions. App Support and Ops
teams want to eliminate manual and error-
prone deployment processes to achieve the
velocity needed to deliver any software into
any environment – especially into production,
where value is delivered.

49. Release Management
Control and track applications as they are
promoted along the path to production
Shipping code is the killer feature. Until code is
deployed and accessible by users, no value has
been created. But coordinating release
activities and approvals for both traditional
and CD pipelines across multiple teams and
shared infrastructure is a logistical challenge.
Organizations want a way to manage their
enterprise traditional releases and CD
pipelines in a scalable, auditable and
predictable way.

50. Agile
Automate deployments so Ops teams can keep
up with the pace of Agile delivery
Iterate and integrate often. Time is money.
Requirements are always shifting. Agility is
hard to achieve if your builds and tests take a
long time or are not automated. Nimble,
iterative, and incremental sprints get bogged
down without a solid process automation
framework in place. Organizations want to
automate provisioning, build and release
activities and handoffs to eliminate manual
errors and wait times, shrinks cycle times and
accelerate the feedback loop.

51. print(“Thank You!”)