Harm Boertien is an engineer at Schuberg Philis who provides a history of automation at the company. They initially used cfengine for configuration management before adopting BMC Bladelogic and later Chef. Currently, they focus on writing software for customers, maintaining open source cookbooks, and evangelizing devops practices. The presentation provides a brief history of early automation technologies like the Anthikythera mechanism and Charles Babbage's analytical machine to argue that our society must intentionally include diversity.
2. WHOAMI
Harm Boertien
engineer @ Schuberg Philis
Father of Kai, Silke and Matt
Volunteer
hackercamps
lego mindstorms education
hour of code (scratch)
devopsdays Amsterdam
several meetup groups
Mostly harmless @Harm_Ops
4. History of automation at Schuberg Philis
Ian Southam started with cfengine
- Usermatrix distributing public ssh keys
- Automated deploy of nagios
- Manage proxy/httpd/postfix settings
- addclient.sh @isoutham
5. BMC Bladelogic for MS Windows
Introduced by our toolkit team
- good at automating windows
- we did not replace cfengine 2 on unix
- it did not scale well over multiple
customers
- no multi-tenancy
- relatively expensive per host
6. Chef at Schuberg Philis
We started somewhere 2012
Sander2 very active in the community
Introduced chef-guard
https://github.com/xanzy/chef-guard
https://www.chef.io/blog/chef-guard/
Maintainers of:
nagios, nrpe cookbook, knife cloudstack gem
sbp_* cookbooks
@_svanharmelen_
@sanderbotman
7. What are we working on now?
NEW -> Writing software for our customers
maintaining and improving CloudStack code
Maintainig and adopting community cookbooks
Chef-plugin for Terraform
Owasp Security Knowledge Framework
Evangelizing devops way of working
8. Adopt a cookbook !
https://supermarket.chef.io/available_for_adoption
We adopted the nagios cookbook
And are adding our sbp_*_cookbooks
11. Joseph Marie Jacquard (1752 – 1834)
The first programmable weaving 1801
Based on this Charles Babbage
constructed his ideas
12. Charles Babbage (1791 – 1871)
Ada Lovelace (1815 – 1852)
The difference engine
The analytical machine
Considered first programming language
13. Herman Hollerith (1860 – 1929)
Storing data with the first tabulating machines.
And yes, this later became IBM
14. Alan Turing (1912 – 1954)
The Bombe used for decrypting the Enigma messages
15. Margareth Hamilton / Apollo 13
One of the founders of modern software design
- Ultra-reliable design
16. So what do we learn from this condensed overview
Our society is harsh to diversity and this is hard to change
“If you do not intentionally include,
you unintentionally exclude.”
Set a “code of conduct” for your meetups
Actively invite women in tech groups
Working at Schuberg since 2006
Living in Amsterdam
Network engineer / Scrum master / technical lead (which basicly means architect)
Started at AT&T 2nd line / Frame Relay networks …
KPN/Qwest
Enertel/Energis
We started of with no automation
Using foglight (what’s in a name) for monitoring
At our 1st big customer Ian decided to start using cfengine
And we started using nagios
Instead of using word dox, we started using an internal wiki which later became confluence
We had a horrendous ticketing system, now Jira
Before we used slack we used IRC/MSN/Lync/jabber etc.
Slack is now widely adopted
For versioning we used cvs and later svn. For the versioning of our cfengine files.
For our windows part of the world we used for monitoring MS System Center
For configuration we did an attempt of using cfengine, lacked registry support
We started using BL for our customers, we could not scale this well.
Hard to exchange “cookbooks” between customers and capture our best practise
Since 2012 we have switched more and more of our customers to Chef.
We had the choice back then between puppet / cfengine3
From the shootout we learned that Chef had the upperhand.
Our Sanders did a big part in evangilizing chef within the company and come to a way of working over all teams.
Our cookbooks are in git, and where we can we use the public cookbooks from the chef supermarket.
If that does not fit, we will try and update the public cookbooks, before making our own “private” cookbooks.
Chef-guard is a layer between you (the knife/berk/gui user), and the chef-server. This awesomeness is created by the awesome Sander van Harmelen!
Key characteristics of chef-guard:
Chef-Guard gives you a wChef-guard is a layer between you (the knife/berk/gui user), and the chef-server. This awesomeness is created by the awesome Sander van Harmelen!
Key characteristics of chef-guard:
Chef-Guard gives you a way to monitoring and audit everything that happens in Chef by commiting *all* (succesful) changes to nodes, roles, environments and data bags into a Git repository. All changes you do with knife/berks/gui will be noticed now, and sent to x-cvs-commits@schubergphilis.com! --> No more need for separate "git commit" or "git update"!
Chef-Guard does quality checks (using foodcritic and rubocop at this moment) of the cookbooks you (try) to upload by doing inline checks *before* actually saving them to your Chef server. --> no more ugly/unreadable code accepted by chef-server!
Chef-Guard enables you to bypass failed checks using --force and at the same time disables the option to overwrite a frozen cookbook using that same --force --> safeguard from yourself...
Chef-Guard does Graphite! (Link to the GaaS dashboard will be provided)
Chef-Guard checks if you add, update or change any dependencies (anywhere) and, if so, checks if the version you depend on are frozen. --> so frozen really means frozen for life of that cookbook v1.2.
When freezing a cookbook Chef-Guard validates if what you are uploading is a cookbook version that actually exists
ay to monitoring and audit everything that happens in Chef by commiting *all* (succesful) changes to nodes, roles, environments and data bags into a Git repository. All changes you do with knife/berks/gui will be noticed now, and sent to x-cvs-commits@schubergphilis.com! --> No more need for separate "git commit" or "git update"!
Chef-Guard does quality checks (using foodcritic and rubocop at this moment) of the cookbooks you (try) to upload by doing inline checks *before* actually saving them to your Chef server. --> no more ugly/unreadable code accepted by chef-server!
Chef-Guard enables you to bypass failed checks using --force and at the same time disables the option to overwrite a frozen cookbook using that same --force --> safeguard from yourself...
Chef-Guard does Graphite! (Link to the GaaS dashboard will be provided)
Chef-Guard checks if you add, update or change any dependencies (anywhere) and, if so, checks if the version you depend on are frozen. --> so frozen really means frozen for life of that cookbook v1.2.
When freezing a cookbook Chef-Guard validates if what you are uploading is a cookbook version that actually exists
From a traditional outsourcer, well traditional, we did a lot of infrasctructure with our 100% promise.
For some of our customers we are now writing
Nagios cookbook adopted by Sander Botman, cleaned up
Several cookbooks from Sander van Harmelen and Sander Botman
So what is all this automation bringing us ?
If we had not invested in this, and we invested quite a lot, we would not have been able to cater our customers.
We are “centralizing” our services and now not only have a private cloud for our customers.
We can cater for public cloud.
We have a multi-tenant backup solution, which all our customer teams can use.
Because we automate our best practises, it is easier to rebuild a customer infrastructure.
It is also easier to move from private to a more hybrid cloud solution.
Now this automation seems a new thing in our industry.
But of course it is not. Mankind has automated throughout the ages.
This artefact was found on a greek ship, and was a complex tool to predict the motion of celestial bodies
https://en.wikipedia.org/wiki/Antikythera_mechanism
The first analogue computer, recovered from a wreck outside Anthikythera.
And it was a weaver and merchant who basicly invented “programming”.
The self-playing organs came later.
In 1801 he constructed a loom that used a series of punched cards to control the pattern of longitudinal warp threads depressed before each sideways passage of the shuttle. Jacquard later developed a machine where the punched cards were joined to form an endless loop that represented the program for the repeating pattern used for cloth and carpet designs.Jacquard's invention allowed patterns to be woven without the intervention of the weaver. At first Jacquard's looms were destroyed by weavers who feared unemployment. The French government took over the invention and Jacquard was given a royalty on every loom sold.
By 1812 there were 11,000 Jacquard looms working in France, and they were also beginning to appear in other countries. The growth of the use of the Jacquard loom in the 1820s gave the textile industry a tremendous boost in Britain. By 1833 there were about 100,000 power-looms being used in this country that had been influenced by Jacquard's invention.Joseph Jacquard died in 1834. Charles Babbage was later to adapt Jacquard's punch-card system to produce a calculator that was the forerunner of today's methods of computer programming.
In the early 1900’s Charles Babbage worked on a mechanical differential machine.
And it was because of the interaction with Ada Lovelace that the Analytical machine came to live.
https://en.wikipedia.org/wiki/Charles_Babbage
https://en.wikipedia.org/wiki/Ada_Lovelace
Ada Lovelace and Italian followers
Ada Lovelace corresponded with him during his development of the Analytical Engine. She is credited with developing an algorithm for the Analytical Engine to calculate a sequence of Bernoulli numbers. Although there is disagreement over how much of the ideas were Lovelace's own, she is often described as the first computer programmer.[151] She also translated and wrote literature supporting the project. Babbage visited Turin in 1840 at the invitation of Giovanni Plana. In 1842 Charles Wheatstone approached Lovelace to translate a paper of Luigi Menabrea, who had taken notes of Babbage's Turin talks; and Babbage asked her to add something of her own. Fortunato Prandi who acted as interpreter in Turin was an Italian exile and follower of Giuseppe Mazzini.[152]
First computer program[edit]
Lovelace's diagram from Note G, the first published computer algorithm
In 1840, Babbage was invited to give a seminar at the University of Turin about his Analytical Engine. Luigi Menabrea, a young Italian engineer, and the future Prime Minister of Italy, wrote up Babbage's lecture in French, and this transcript was subsequently published in the Bibliothèque universelle de Genève in October 1842. Babbage's friend Charles Wheatstone commissioned Lovelace to translate Menabrea's paper into English. She then augmented the paper with notes, which were added to the translation. Lovelace spent the better part of a year doing this, assisted with input from Babbage. These notes, which are more extensive than Menabrea's paper, were then published in Taylor's Scientific Memoirs under the initialismAAL. In 1953, more than a century after her death, Lovelace's notes on Babbage's Analytical Engine were republished. The engine has now been recognised as an early model for a computer and her notes as a description of a computer and software.[61]
Lovelace's notes were labelled alphabetically from A to G. In note G, she describes an algorithm for the Analytical Engine to compute Bernoulli numbers. It is considered the first algorithm ever specifically tailored for implementation on a computer, and Ada has often been cited as the first computer programmer for this reason.[62][63] The engine was never completed so her code was never tested.[64]
In the late 1900’s a solution was found for storing data.
In 1892 Hermann Hollerith moved his fledgling tabulating machine business from downtown Washington, D.C., to a former cooper's shop in the Georgetown section of the city. Seen here the following year, the two-story building (later expanded) housed Hollerith's card manufacturing plant, assembly plant, repair shop and development laboratory. Three years after this photograph was taken, Hollerith incorporated his business as the Tabulating Machine Company. It was consolidated into the Computing-Tabulating-Recording Co. in 1911, and CTR -- with Thomas J. Watson, Sr., at the helm -- was renamed IBM 13 years later. (VV2076)
https://en.wikipedia.org/wiki/Alan_Turing
Died at the age of 42, because he was forced treatment for his homosexuality.
The UK government apologised in September 2009, only 6 years ago
Hamilton then joined the Charles Stark Draper Laboratory at MIT, which at the time was working on the Apollo space mission. She eventually became the director and supervisor of software programming for Apollo and Skylab.[8]
At NASA, Hamilton's team was responsible for helping pioneer the Apollo on-board guidance software required to navigate and land on the Moon, and its multiple variations used on numerous missions (including the subsequentSkylab).[2] She worked to gain hands-on experience during a time when computer science and software engineering courses or disciplines were non-existent.
Her areas of expertise include system design and software development, enterprise and process modelling, development paradigm, formal systems modelling languages, system-oriented objects for systems modelling and development, automated life-cycle environments, methods for maximizing software reliability and reuse, domain analysis, correctness by built-in language properties, open-architecture techniques for robust systems, full life-cycle automation, quality assurance, seamless integration, error detection and recovery techniques, man/machine interface systems, operating systems, end-to-end testing techniques, and life-cycle management techniques.[2]
She was one of those who developed concepts of asynchronous software, priority scheduling, and Human-in-the-loop decision capability, which became the foundation for modern, ultra-reliable software design.
https://en.wikipedia.org/wiki/Apollo_Guidance_Computer
https://en.wikipedia.org/wiki/Margaret_Hamilton_(scientist)
Lean
Kata
Agile
https://en.wikipedia.org/wiki/Mission_Command
Dupont
http://www.forbes.com/sites/ekaterinawalter/2014/01/14/reaping-the-benefits-of-diversity-for-modern-business-innovation/
Diversity brings creativity
Apperently they bring higher earnings
According to devops survey 30% of the people in our industry work in teams without women (that includes me)
Google has accomplished around 15-20% of their tech force
Etsy around the same
https://medium.com/matter/this-is-the-last-thing-youll-ever-need-to-read-about-sexism-in-tech-56b9a3a77af0
57 % end up quitting
Thoughtworks ended up with a whopping 30%
http://www.huffingtonpost.com/2015/06/18/heres-how-you-get-more-wo_n_7613670.html
Host meetups
Inspire others for meetups
Working together with other engineers and developers will improve your work and your company
Embrace diversity and see what it will bring you
What might seem a little thing to you can mean the world for someone who’s waiting for a chance and a change
Look for peope to strengthen your company outside of the tech bubble
More women graduate then men, with higher grades, let’s make this chance.
See you tomorrow at the open space, and hopefully some people will join me in how we can improve this.