Using Go in DevOps

•

1 like•899 views

Natalie Pistunovich Engineering Manager – Fraugster Natalie is an Engineering Manager, Go Developer, Berlin’s Go User Group Lead, GopherCon Europe organizer and Public Speaker. She also describes herself as a Passionate Learner and Professional Questions Asker.

Technology

Using Go
in DevOps
Natalie Pistunovich
@NataliePis

- Intro to Go
- OPS/SRE Tools in Go
- What’s Next?

$GOOS $GOARCH
darwin 386 -- 32 bit MacOSX
darwin amd64 -- 64 bit MacOSX
linux 386 -- 32 bit Linux
linux amd64 -- 64 bit Linux
linux arm -- RISC Linux
windows 386 -- 32 bit Windows
windows amd64 -- 64 bit Windows
And there’s more…

• fmt
• io/ioutil
• net/http
• os/exec
• path/filepath
• regexp
• strings
• text/tabwriter
• time
stdlib

Pointers Exist,
Pointer Arithmetic
Doesn’t

The Go community
  doubled
in the last year

There are approx. 1.5
Million active Go users

New Go users in 2017
=
New Go users in
the last 9 years

invite.slack.golangbridge.org
github.com/golang/go/wiki
play.golang.org
golang.org/blog
twitter.com/golang (also: #golang)
Go Resources

This session will focus on the practicals of building a fully-functional stack of container cluster tools, with different options for stacking those tools from the OS-up. We’ve all seen examples of common technologies stacks, like the good ol’ LAMP and MEAN stacks for apps, but what about lower-level infrastructure? And can we get it without cloud vendor lock in please? Oh and pure containers and infrastructure-as-code too? With Docker, sure thing! This session will cover: Which OS/Distro and Kernel to use VM’s or Bare Metal Recommended Swarm architectures Tool stacks for “pure open source”, “cloud-service based”, and “Docker EE” scenarios Demos of these tools working together including InfraKit, Docker, Swarm, Flow-Proxy, ELK, Prometheus, REX-Ray, and more.

Troubleshooting tips from docker support engineers

Docker, Inc.

Docker makes everything easier. But even with the easiest platforms, sometimes you run into problems. In this session, you'll learn first-hand from someone whose job is helping customers fix these problems. In this session, you'll learn how to: Ask better questions & identify the real problems you want to solve Use command line tools to inspect internal Swarm and Kubernetes components Debug and troubleshoot interactions between Enterprise Edition components Troubleshoot Docker Enterprise Edition Windows nodes Apply these skills to troubleshooting Docker Enterprise Edition

Containerizing Hardware Accelerated Applications

Docker, Inc.

Many applications allow you to use hardware such as GPUs and FPGAs for acceleration. Common examples include media processing and offloading highly parallel work to a GPU. Applications that use accelerators are resource heavy and have stacks spanning kernel and user space; accelerators often have their own requirements for operating system support and kernel versions. While it may not seem intuitive to containerize this type of application, the use of containers provides benefits such as reduced setup time from container reuse, reduction in dependency conflicts and dependency on a specific operating system, and easier updates. In this session I show a media processing stack, making use of containers alongside a GPU. Specifically, I explain the kernel and user space divide of a hardware-accelerated transcode application using a device exposed to the container. This specific stack is an interesting case because of its dependency on hardware, use of a custom kernel and libraries, and operating system requirements. Our investigations have shown the use of containers has minimal performance overhead compared to running natively. Furthermore, we can quickly deploy on other machines with reduced configuration effort. There are some aspects of the application not suited to containerization, however. Since the application relies on a custom kernel, the use of containers does not necessarily increase portability. Improvement in this area would require rethinking how the applications are developed and distributed. Other areas of innovation include things such as Docker plugins to check for compatibility between the container software and host kernel.

DockerCon EU 2015: Stop Being Lazy and Test Your Software!

Docker, Inc.

Presented by Laura Frank, Engineer, Codeship Testing software is necessary, no matter the size or status of your company. Introducing Docker to your development workflow can help you write and run your testing frameworks more efficiently, so that you can always deliver your best product to your customers and there are no excuses for not writing tests anymore. You’ll walk away from this talk with practical advice for using Docker to run your test frameworks more efficiently, as well as some solid knowledge of software testing principles.

Snap CI enables software teams to do Continuous Delivery (CD). When practicing CD, the goal is to automate the deployment process and build software in such a way that it can be deployed to production any time. As a deployment tool, Snap CI cannot have downtime. If it did, our users would not be able to deploy their own software. We had to change Snap CI’s architecture to ensure zero-downtime and we chose to do blue-green deployments to achieve it. In this approach, we had to maintain two instances of our system: one active instance, and one inactive instance. Based on our experiences, we will share some tricks of the trade from the numerous challenges we faced such as: making the application aware of whether it was active or inactive, handling data migrations, and babysitting long-running jobs. These are the slides from Akshay Karle and Fernando Junior's presentation on Agile Brazil 2015.

Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...

Docker, Inc.

What is the right balance between moving fast, innovating, experimenting with new technology, and protecting the personal data of our customers and interests of our stakeholders? How can we safely try new ideas in production without risking costly downtime? Does the utopia where developers are free from lock-in and operators enjoy the calm of a steadily running system exist in the real world? Is it possible to have open platforms with better security? At Kroger Digital we are still working through these questions every day but are redesigning our systems with the goals of true operational maturity and security. Discover how we are building capabilities for monitoring, A/B testing, and continuous delivery with Docker Datacenter, plugins, and open source building blocks such as NGiNX, ElasticSearch, and more.

Immutable Infrastructure: Rise of the Machine Images

C4Media

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/1WlpXHF. Axel Fontaine looks at what Immutable Infrastructure is and how it affects scaling, logging, sessions, configuration, service discovery and more. He also looks at how containers and machine images compare and why some things people took for granted may not be necessary anymore. Filmed at qconlondon.com. Axel Fontaine is the founder and CEO of Boxfuse. Axel is also the creator and project lead of Flyway, the open source tool that makes database migration easy. He is a Continuous Delivery and Immutable Infrastructure expert, a Java Champion, a JavaOne Rockstar and a regular speaker at various large international conferences.

Dev ops with smell v1.2

Antons Kranga

CI/CD on AWS

Bhargav Amin

JavaCro'14 - Continuous delivery of Java EE applications with Jenkins and Doc...

HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association

Continuous delivery is a powerful concept, but hard to achieve. One of the challenges is automating the setup of environments and the deployment of the Java EE applications. We have looked at and used quite some tools like for instance Chef, Puppet, Vagrant and Nolio. All tools had one thing in common: we had never used them. Why should we invest time in mastering those tools? There is a perfect alternative in Jenkins, a tool most developers are familiar with. Besides the basic Jenkins buildserver capabilities it offers quite some useful plugins like the Build Pipeline plugin. To setup environments the popular Docker project is used. Docker allows you to create containers from any application. Only some knowledge is required for the setup of the containers. The rest of the configuration is done through commands most people are quite familiar with.

CI/CD on pure AWS

Andrey Trubitsyn

In this talk I will show you how to build a CI/CD pipeline in AWS with, static code analysis in Sonar, tests and continuous deployment of a dockerized service through several environments by using pure AWS services like CodeStar, CodeCommit, CodeBuild, CodeDeploy and CodePipline. I will do a demo of such CI/CD to reveal all guts of tools and services integration and implementation. So you will see how a commit will be going through all those steps and tools to get production environment.

Java Day Kharkiv - Next-gen engineering with Docker and Kubernetes

Antons Kranga

DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose

Docker, Inc.

Presented by Sandro Cirulli, Platform Tech Lead, Oxford University Press Oxford University Press (OUP) recently started the Oxford Global Languages (OGL) initiative (http://www.oxforddictionaries.com/words/oxfordlanguages) which aims at providing language resources for digitally under represented languages. In August 2015 OUP launched two African languages websites for Zulu (http://zu.oxforddictionaries.com) and Northern Sotho (http://nso.oxforddictionaries.com). The backend of these websites is based on an API retrieving data in RDF from a triple store and delivering data to the frontend in JSON-LD. The entire micro-service infrastructure for development, staging, and production runs on Docker containers in Amazon EC2 instances. In particular, we use Jenkins to rebuild the Docker image for the API based on a Python Flask application and Docker Compose to orchestrate the containers. A typical CI workflow is as follows: - a developer commits code to the codebase - Jenkins triggers a job to run unit tests - if the unit tests are successful, the Docker image of the Python Flask application is rebuilt and the container is restarted via Docker Compose - if the unit tests or the Docker build failed, the monitor view shows the Jenkins jobs in red and displays the name of the possible culprit who broke the build. A demo of this CI workflow is available at http://www.sandrocirulli.net/continuous-integration-with-jenkins-docker-and-compose

DCSF 19 Building Your Development Pipeline

Docker, Inc.

Oliver Pomeroy, Docker & Laura Tacho, Cloudbees Enterprises often want to provide automation and standardisation on top of their container platform, using a pipeline to build and deploy their containerized applications. However this opens up new challenges; Do I have to build a new CI/CD Stack? Can I build my CI/CD pipeline with Kubernetes orchestration? What should my build agents look like? How do I integrate my pipeline into my enterprise container registry? In this session full of examples and how-to's, Olly and Laura will guide you through common situations and decisions related to your pipelines. We'll cover building minimal images, scanning and signing images, and give examples on how to enforce compliance standards and best practices across your teams.

Building a Service Delivery Platform - JCICPH 2014

Andreas Rehn

This talk will walk through the critical parts of a tool chain that forms the service delivery platform, a robust, secure solution with Jenkins as the main orchestrator that scales with many teams and hundreds of pipelines. I will show a tool chain with Git, Jenkins, Jenkins Job Builder, Puppet, Graphite, Logstash and more that is proven in battle. I will share insights and details on good ways of building a platform for pipelines that recognizes the individual teams needs for fast feedback, traceability and visibility in the delivery process.

Drone CI

Thomas Boerger

Rundeck + Nexus (from Nexus Live on June 5, 2014)

dev2ops

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

Docker, Inc.

Mohammed Aboullaite, xHub Nowadays, containers are more than a cool new tool to play with - they've revolutionized the way we develop, architect and ship software and have become part of our day-to-day operations. On the other hand, Java has been around for more than 2 decades, dominating the entreprise world. Both advocate the "RUN anywhere" principal, but, is it that easy? No! Your perfect working local container will most probably fail on production because of MEMORY &/or CPU issues, since jvm apps assume they OWN the server they are running on. This session will look at the gotchas of running JVM apps in containers & how to avoid costly pitfalls. We will cover: _ what updates Java 10 brought to the table to improve container awareness and the issues related to prior versions, including how to address each one of them to avoid the mess. - tips and tricks to obtain smaller images & best practices while writing your DockerFiles - some plugins to helps non-containers experts integrate docker in their development process; and - the concept of "Distroless" Docker Images and how to benefit from it.

Building a Secure Supply Chain with Docker

Docker, Inc.

Creating a Secure Supply Chain of images is vitally important. Every organization needs to weigh ALL options available and understand the security risks. With so many options for images, it is tough to pick the right ones or even to create your own. Ultimately, every organization needs to know the provenance of all the images. Then once the images are imported into the infrastructure, a vulnerability scan is vital. Docker Trusted Registry with Image Scanning will give organizations insight into any vulnerabilities. Better yet, its automated with a succinct audit trail, so you can still take that vacation you had planned and make your security team happy.

DCEU 18: How To Build Your Containerization Strategy

Docker, Inc.

Lee Namba - EMEA Professional Services Manager, Docker The Docker Enterprise container platform helps organizations deploy and manage applications faster and it secures the application pipeline at a lower cost than traditional application delivery models. But it takes more than just great technology to achieve the desired results. The organization and culture of your enterprise directly impacts what you transform, how it’s done, and who does it. Success requires a strategy for how you will govern the container platform environment, how to assess your application estate, what your delivery pipeline will look like, and how to ensure developers, operators, security teams and others play nicely together. In this talk I will cover topics such as different types of workloads (legacy, microservices, FaaS, big data and more), how your org chart can influence whether you deploy CaaS (Containers as a Service) vs CLaaS (Clusters as a Service), how "shifting left" can determine if you can outsource, centralized vs distributed CI/CD and how containers play a role, transforming your pets into cattle, how giant whale balloons are used for onboarding, and a prescriptive and comprehensive methodology for successfully deploying containers into your enterprise.

Deploy and upgrade Docker applications with a single click

Docker, Inc.

Docker has made running complex applications locally a snap, but running applications in production can still be quite challenging. This session will demonstrate how to use Rancher and Docker Compose to deploy and upgrade applications predictably into production environments. Rancher Co-Founder Darren Shepherd will demonstrate how to deploy an application from a Docker compose application catalog into production, and then how to orchestrate an upgrade of that application.

Efficient Parallel Testing with Docker by Laura Frank

Docker, Inc.

Fast and efficient software testing is easy with Docker. We often use containers to maintain parity across development, testing, and production environments, but we can also use containerization to significantly reduce time needed for testing by spinning up multiple instances of fully isolated testing environments and executing tests in parallel. This strategy also helps you maximize the utilization of infrastructure resources. The enhanced toolset provided by Docker makes this process simple and unobtrusive, and you’ll see how Docker Engine, Registry, Machine, and Compose can work together to make your tests fast.

"Using Automation Tools To Deploy And Operate Applications In Real World Scen...

ConSol Consulting & Solutions Software GmbH

Continuous Delivery with Jenkins Workflow

Udaypal Aarkoti

The devops approach to monitoring, Open Source and Infrastructure as Code Style

Julien Pivotto

Beyond QA

gilforcada

What's hot

Continuously serving the developer community with Continuous Integration and...

Thoughtworks

Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...

Docker, Inc.

Immutable Infrastructure: Rise of the Machine Images

C4Media

Dev ops with smell v1.2

Antons Kranga

CI/CD on AWS

Bhargav Amin

JavaCro'14 - Continuous delivery of Java EE applications with Jenkins and Doc...

HUJAK - Hrvatska udruga Java korisnika / Croatian Java User Association

CI/CD on pure AWS

Andrey Trubitsyn

Java Day Kharkiv - Next-gen engineering with Docker and Kubernetes

Antons Kranga

DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose

Docker, Inc.

DCSF 19 Building Your Development Pipeline

Docker, Inc.

Building a Service Delivery Platform - JCICPH 2014

Andreas Rehn

Drone CI

Thomas Boerger

Rundeck + Nexus (from Nexus Live on June 5, 2014)

dev2ops

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

Docker, Inc.

Building a Secure Supply Chain with Docker

Docker, Inc.

DCEU 18: How To Build Your Containerization Strategy

Docker, Inc.

Deploy and upgrade Docker applications with a single click

Docker, Inc.

Efficient Parallel Testing with Docker by Laura Frank

Docker, Inc.

"Using Automation Tools To Deploy And Operate Applications In Real World Scen...

ConSol Consulting & Solutions Software GmbH

Continuous Delivery with Jenkins Workflow

Udaypal Aarkoti

What's hot (20)

Continuously serving the developer community with Continuous Integration and...

Learning the Alphabet: A/B, CD and [E-Z] in the Docker Datacenter by Brett Ti...

Immutable Infrastructure: Rise of the Machine Images

Dev ops with smell v1.2

CI/CD on AWS

JavaCro'14 - Continuous delivery of Java EE applications with Jenkins and Doc...

CI/CD on pure AWS

Java Day Kharkiv - Next-gen engineering with Docker and Kubernetes

DockerCon EU 2015: Continuous Integration with Jenkins, Docker and Compose

DCSF 19 Building Your Development Pipeline

Building a Service Delivery Platform - JCICPH 2014

Drone CI

Rundeck + Nexus (from Nexus Live on June 5, 2014)

DCSF19 Docker Containers & Java: What I Wish I Had Been Told

Building a Secure Supply Chain with Docker

DCEU 18: How To Build Your Containerization Strategy

Deploy and upgrade Docker applications with a single click

Efficient Parallel Testing with Docker by Laura Frank

"Using Automation Tools To Deploy And Operate Applications In Real World Scen...

Continuous Delivery with Jenkins Workflow

Similar to Using Go in DevOps

The devops approach to monitoring, Open Source and Infrastructure as Code Style

Julien Pivotto

Beyond QA

gilforcada

Pipeline as code for your infrastructure as Code

Kris Buytaert

Open Source Monitoring in 2019

Kris Buytaert

Cutting the Kubernetes Monorepo in pieces – never learnt more about git

Stefan Schimanski

Kubernetes uses a monorepo approach for development and testing. For Golang vendoring by 3rdparties we publish subdirectories as separate Github repositories (like k8s.io/client-go, k8s.io/apimachinery, k8s.io/api, etc.) continuously every night, while keeping all history, all Github merge commits and while rewriting commits with Godep-save updates to express dependencies. Sounds easy? It's not! Implementing this was the best learning experience of git and git internals and the topic of this talk.

GitOps Core Concepts & Ways of Structuring Your Repos

Weaveworks

Watch this talk on YouTube here: https://youtu.be/vLNZA_2Na_s Whether you’re new to GitOps or a seasoned pro, this talk is for you! We'll start with the basics of how/where to get started, and then dive into one of the most asked GitOps questions: how to structure your repository! During this talk, Scott & Pinky will review the Core Concepts of Flux including Git Sources, Reconciliation, Helm Releases, Kustomization, and Bootstrapping, to get you ramped up with how to think with a GitOps mindset! Then they’ll dive into and discuss considerations for and demo ways of structuring your repositories: monorepo, repo per environment, repo per team, or repo per app. Resources: - Flux on GitHub: https://github.com/fluxcd/flux2 - Flux docs: https://fluxcd.io/docs - Core Concepts: https://fluxcd.io/docs/concepts/ - Sources: https://fluxcd.io/docs/components/source/ - Helm Releases: https://fluxcd.io/docs/guides/helmreleases/ - Kustomization: https://fluxcd.io/docs/components/kustomize/ Bootstrap: https://fluxcd.io/docs/installation/#bootstrap - Ways of Structuring Your Repos: https://fluxcd.io/docs/guides/repository-structure/ Speaker Bios: Priyanka “Pinky” Ravi is a Developer Experience Engineer at Weaveworks. She has worked on a multitude of topics including front end development, UI automation for testing and API development. Previously she was a software developer at a large insurance company where she was on the delivery engineering team working on GitOps enablement. She was instrumental in the multi-tenancy migration to utilize Flux for an internal Kubernetes offering. Outside of work, Priyanka enjoys hanging out with her husband and two rescue dogs as well as traveling around the globe. Scott is a Brooklyn based interdisciplinary artist and Developer Advocate at Weaveworks. He co-founded the Basekamp art and research group in 1998 and the massively collaborative Plausible Artworlds international network. In technology he enjoys helping develop open source software that anyone can use, most recently projects in the cloud native landscape including co-maintaining Helm and Flux. In daily decisions, large or small, he tries to help make the world a better place for everyone.

Resources For Floss Projects

Jon Spriggs

Monitoring at Cloud Scale

Julien Pivotto

Dev secops opsec, devsec, devops ?

Kris Buytaert

Github for Serious Business Professionalzwheller

Town Hall - Business Implications of Open Source OSGi Implementations - BJ Ha...

mfrancis

Deployment Patterns in the Ruby on Rails World

Nikhil Mungel

Ruby - The Hard Bits

Paul Gallagher

Continuous Delivery of (y)our infrastructure.

Kris Buytaert

Puppet Camp Dallas 2014: How Puppet Ops Rolls

Puppet

2014-08-19 Multiple Approaches to Managing Puppet Modules @ Puppet Camp Chicago

garrett honeycutt

Groovy there's a docker in my application pipeline

Kris Buytaert

OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert

NETWAYS

The DevSecOps Builder’s Guide to the CI/CD Pipeline

James Wickett

All organizations want to go faster and decrease friction in their cloud software delivery pipeline. Infosec has an opportunity to change their classic approach from blocker to enabler. This talk will discuss hallmarks of CI/CD and some practical examples for adding security testing across different organizations. The talk will cover emergent patterns, practices and toolchains that bring security to the table. Presented at LASCON 2018, in Austin, TX.

From Config Management Sucks to #cfgmgmtlove

Kris Buytaert

Similar to Using Go in DevOps (20)

The devops approach to monitoring, Open Source and Infrastructure as Code Style

Beyond QA

Pipeline as code for your infrastructure as Code

Open Source Monitoring in 2019

Cutting the Kubernetes Monorepo in pieces – never learnt more about git

GitOps Core Concepts & Ways of Structuring Your Repos

Resources For Floss Projects

Monitoring at Cloud Scale

Dev secops opsec, devsec, devops ?

Github for Serious Business Professional

Town Hall - Business Implications of Open Source OSGi Implementations - BJ Ha...

Deployment Patterns in the Ruby on Rails World

Ruby - The Hard Bits

Continuous Delivery of (y)our infrastructure.

Puppet Camp Dallas 2014: How Puppet Ops Rolls

2014-08-19 Multiple Approaches to Managing Puppet Modules @ Puppet Camp Chicago

Groovy there's a docker in my application pipeline

OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert

The DevSecOps Builder’s Guide to the CI/CD Pipeline

From Config Management Sucks to #cfgmgmtlove

More from Eficode

Saving money with Consolidations

Using Go in DevOps

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Using Go in DevOps

Similar to Using Go in DevOps (20)

More from Eficode

More from Eficode (20)

Recently uploaded

Recently uploaded (20)

Using Go in DevOps