Packer, Terraform & Jenkins

•Download as PPTX, PDF•

0 likes•191 views

The document discusses using Packer, Terraform, and Jenkins together for infrastructure as code (Iac) and continuous delivery (CD) of small application stacks. It describes goals of having traceable images that record build details, testable images using tools like InSpec, and self-contained pipelines using a monorepo approach. It outlines wrappers for Packer and Terraform to expose build details and map branches to workspaces. A Jenkinsfile is proposed for a simple master/non-master workflow. Tips are provided around unique resources and image build provenance. Links are included for related GitHub repos and the author's Twitter profile.

Packer, Terraform & Jenkins
IaC & CD for small stacks
Simon McCartney
Twitter: @simonmcc

Background:
Isn’t this a solved problem?

Background:
Don’t containers solve
everything?

Goal 1: Traceable Images
• What was this image built from?
• Project, branch, SHA, clean, dirty
• Store it inside the image and
externally via tags

Goal 2: Testable Images
• Validate that an image is good
• Chef’s InSpec for os/infra
• App/Service specific testing

Goal 3: Self Contained
• Monorepo FTW, single pipeline
• Minimal Jenkins Plugins
• Sanity wrappers for Terraform & Packer
• Terraform for the app
• Terraform for the build environment

Packer Wrapper – build.sh
•Modeled on base & app AMI – overkill for
this POC
•Expose git SHA & clean/dirty state to packer
for including in tags etc (‘cos CLI building
should still be possible!)
•Only build base/app AMI when necessary

tfw
•Terraform wrapper
•Map git branch to terraform workspace
•Map git branch to tfvars
•Expose git branch & sha to aid tagging &
building unique resources (RDS Instance
etc)

tfw
•Configure terraform remote state
render backend_config.tf
AWS: create s3 bucket & DynamoDB
Azure: create Resource Group & Storage Account

Tips
•Watch out for account or globally unique
resources (that’s why we expose branch &
SHA1 to packer & terraform)
•SHA1 for images in this POC is weak – it’s
of a git object that “mostly” represents the
image build source.
•Jenkins aws-credentials & docker agent is
broken :(

Links
•https://github.com/simonmcc/cfgmgmtcmp-2019-
iac-ci-cd
•https://github.com/jenkins201/jenkins-container
•Twitter: @simonmcc
•https://www.axon.com

"Apache Kafka is an amazing piece of technology, that has been furiously adopted by companies all around the world to implement event-driven architectures. While its adoption continues to increase; the reality is that most developers often complain about the complexity of managing the clusters by themselves, which seriously decreases their ability to be agile. This talk will introduce Confluent Cloud, a service that offers Apache Kafka and the Confluent Platform so developers can focus on what they do best: the coding part. Through interactive demos, it will be shown how to quickly reuse code written for standard Kafka APIs to connect to Confluent Cloud and doing some interesting stuff with it. This is a zero-experience-needed type of session, where the focus is on providing the first steps to beginners."

Python introduction why to study

Saravanakumar viswanathan

This document provides an introduction and overview of the Python programming language, including its history as a successor to ABC and Modula-3 dating back to the late 1980s, its key features, different versions, common applications, organizations that use Python, and potential career paths for those skilled in Python such as software engineer, Python developer, research analyst, data analyst, and data scientist.

Top 5 Things I've Messed Up in Live Streaming

Fastly

June 24, 2014 - SF Video Technology Meetup. Live streaming can be difficult and challenging, yet extremely rewarding. In the video gaming world, it has become the most powerful way of reaching your audience. It combines traditional web technologies with even more traditional broadcast video stacks in a "it's going live now" environment. In this presentation Lee Chen, head of product at Fastly, goes over the top 5 things that have lost him sleep, made his eyebrows go up, and sometimes even made him fail -- what he did to correct it.

App::RemoteCommand

Shoichi Kaji

App::RemoteCommand is a simple remote command launcher via SSH. It was created because existing IT automation tools can be obtrusive for ad hoc operations, while SSH alone lacks power. Key features include executing commands in parallel on multiple hosts, remembering sudo passwords, specifying local script files, and appending host/time to output. It is built on Net::OpenSSH for its SSH capabilities and uses its own event loop to monitor SSH connections and processes.

01.from java to kotlin

Muhammad Sadewa Wicaksana Wibowo

Git Ready! Workflows

Atlassian

How do you choose a git workflow that increases productivity and reduces the friction of your team? What are the battle tested practices of successful teams that moved to git? How is git used inside Atlassian? As you might have heard git has many compelling features and has gained incredible momentum in the industry at large. Yet understanding how an enterprise team can adopt it can be daunting. The presentation aims to help answer these questions and more: - Available collaboration models when using a distributed version control system like git - Branching models that foster and enhance parallel development - Emerging code best practices and choices that can be safely adopted when migrating to git - How Continuous Integration changes when your team embraces git

"GitHub com for lazy in cloud project management" by Sergejs Matancevs from S...

DevClub_lv

Experience sharing on how we in Scope Technologies have managed to make project management developer oriented or at least not painful for developer. Our project managers once allowed developers to organize project managements the way we developers want. What happened? How we’ve achieved minimization of administrative stuff? How we use GitHub.com as a tool for lazy project management. Sergejs is experienced .NET and WEB developer and team lead at Scope Technologies.

Maxime Petazzoni gave a presentation on Docker at SignalFx. He discussed how SignalFx uses Docker for infrastructure separation, development lifecycles, application packaging and delivery, and orchestration. He explained how SignalFx monitors Docker containers using CollectD and their Docker plugin to collect host and container metrics. SignalFx provides curated dashboards and anomaly detection for correlated system, container and application metrics.

Titanium 3.2 CLI - TiAppCamp2 - 11/2/2013

Chris Barber

It's the way of the present - Why you should use plone.app.contenttypes

Philip Bauer

Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...

Flink Forward

Moving from Lambda and Kappa Architectures to Kappa+ at Uber Kappa+ is a new approach developed at Uber to overcome the limitations of the Lambda and Kappa architectures. Whether your realtime infrastructure processes data at Uber scale (well over a trillion messages daily) or only a fraction of that, chances are you will need to reprocess old data at some point. There can be many reasons for this. Perhaps a bug fix in the realtime code needs to be retroactively applied (aka backfill), or there is a need to train realtime machine learning models on last few months of data before bringing the models online. Kafka's data retention is limited in practice and generally insufficient for such needs. So data must be processed from archives. Aside from addressing such situations, enabling efficient stream processing on archived as well as realtime data also broadens the applicability of stream processing. This talk introduces the Kappa+ architecture which enables the reuse of streaming realtime logic (stateful and stateless) to efficiently process any amounts of historic data without requiring it to be in Kafka. We shall discuss the complexities involved in such kind of processing and the specific techniques employed in Kappa+ to tackle them.

Does Your Stuff Scale?

stevenh0lmes

This document discusses scaling Django applications on Google App Engine. It provides information on different approaches for running Django on App Engine, including Django non-rel, Djappengine, and using CloudSQL. It also discusses scaling at the technical level through services like memcache, task queues, and versioning, as well as non-technical scaling through organizational practices like removing bottlenecks, centralized tools, and security. The overall message is to focus on making good things and removing overhead through a minimalist approach.

Tips For Maintaining OSS Projects

Taro L. Saito

Automate release processes, think of project maintenance as learning opportunities, and build what you actually need. Tips include using plugins like sbt-sonatype to enable one-command releases to Maven Central, learning new technologies through small contributions to open source projects, and focusing on libraries that solve daily tasks and application development needs. Examples include libraries for packaging, release, logging, configuration, and serialization created for Scala projects.

Terraform day 1

Kalkey

This document provides an overview and tutorial on using Terraform for DevOps. It introduces Terraform as a tool for defining and managing infrastructure as code. It then covers installing Terraform, deploying AWS infrastructure like EC2 instances using Terraform configurations, managing variables and outputs, using provisioners, organizing code with modules and workspaces, and managing Terraform state. The document aims to help users get started with Terraform for infrastructure as code.

React state management with Redux and MobX

Darko Kukovec

Component-first Applications

Miguelangel Fernandez

[FOSDEM 2020] Lazy distribution of container images

Akihiro Suda

https://fosdem.org/2020/schedule/event/containers_lazy_image_distribution/ The biggest problem of the OCI Image Spec is that a container cannot be started until all the tarball layers are downloaded, even though more than 90% of the tarball contents are often unneeded for the actual workload. This session will show state-of-the-art alternative image formats, which allow runtime implementations to start a container without waiting for all its image contents to be locally available. Especially, this session will put focus on CRFS/stargz and its implementation status in containerd (https://github.com/containerd/containerd/issues/3731). The plan for BuildKit integration will be shown as well.

systemd @ Facebook in 2019

Davide Cavalca

eZ Publish 5: from zero to automated deployment (and no regressions!) in one ...

Gaetano Giunta

1. The workshop will cover Docker, managing environments, database changes, and automated deployments for eZPublish websites. 2. A Docker stack is proposed that includes containers for Apache, MySQL, Solr, PHP, and other tools to replicate a production environment for development. Configuration and code are mounted as volumes. 3. Managing environments involves storing settings in the code repository and using symlinks to deploy different configurations. Database changes should be managed via migration scripts rather than connecting directly to a shared database. 4. Automating deployments is important and involves tasks like updating code, the database, caches and reindexing content. The same deployment script should be used for development and production. Testing websites is also recommended.

Steamlining your puppet development workflow

Tomas Doran

The document discusses ways to streamline a Puppet development workflow including using revision control, running Puppet in noop or automatic mode, moving changes slowly through testing and using branches, reporting on changes, and implementing testing strategies like unit testing with rspec-puppet and integration testing with serverspec. It also recommends tools like Foreman, Norman, Puppetfile, and Jenkins to improve testing and deployment.

Puppet Camp New York 2014: Streamlining Puppet Development Workflow

Puppet

Platform Clouds, Containers, Immutable Infrastructure Oh My!

Stuart Charlton

Terraform training 🎒 - Basic

StephaneBoghossian1

遅延レイヤ取得による高互換コンテナ起動高速化手法

Shinagawa Laboratory, The University of Tokyo

Containers in depth – Understanding how containers work to better work with c...

All Things Open

Presented by: Brent Laster Presented at the All Things Open 2021 Raleigh, NC, USA Raleigh Convention Center Abstract: Containers are all the rage these days – from Docker to Kubernetes and everywhere in-between. But to get the most out of them it can be helpful to understand how containers are constructed, how they depend and interact with the operating system, and what the differences and interactions are between layers, images, and containers. Join R&D Director, Brent Laster as he does a quick, visual overview of how containers work and how applications such as Docker work with them. Topics to be discussed include: • What containers are and the benefits they provide • How containers are constructed • The differences between layers, images, and containers • What does immutability really mean • The core Linux functionalities that containers are based on • How containers reuse code • The differences between containers and VMs • What Docker really does • The Open Container Initiative • A good analogy for understanding all of this

Terraform modules and best-practices - September 2018

Anton Babenko

UiPath Test Automation using UiPath Test Suite series, part 6

DianaGray10

Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI. UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities. Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes. What will you get from this session? 1. Insights into integrating generative AI. 2. Understanding how this integration enhances test automation within the UiPath platform 3. Practical demonstrations 4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath Topics covered: What is generative AI Test Automation with generative AI and Open AI. UiPath integration with generative AI Speaker: Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

Similar to Packer, Terraform & Jenkins

Docker {at,with} SignalFx

Maxime Petazzoni

Docker at and with SignalFx

SignalFx

Titanium 3.2 CLI - TiAppCamp2 - 11/2/2013

Chris Barber

It's the way of the present - Why you should use plone.app.contenttypes

Philip Bauer

Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...

Flink Forward

Does Your Stuff Scale?

stevenh0lmes

Tips For Maintaining OSS Projects

Taro L. Saito

Terraform day 1

Kalkey

React state management with Redux and MobX

Darko Kukovec

Component-first Applications

Miguelangel Fernandez

[FOSDEM 2020] Lazy distribution of container images

Akihiro Suda

systemd @ Facebook in 2019

Davide Cavalca

eZ Publish 5: from zero to automated deployment (and no regressions!) in one ...

Gaetano Giunta

Steamlining your puppet development workflow

Tomas Doran

Puppet Camp New York 2014: Streamlining Puppet Development Workflow

Puppet

Platform Clouds, Containers, Immutable Infrastructure Oh My!

Stuart Charlton

Terraform training 🎒 - Basic

StephaneBoghossian1

遅延レイヤ取得による高互換コンテナ起動高速化手法

Shinagawa Laboratory, The University of Tokyo

Containers in depth – Understanding how containers work to better work with c...

All Things Open

Terraform modules and best-practices - September 2018

Anton Babenko

Similar to Packer, Terraform & Jenkins (20)

Docker {at,with} SignalFx

Docker at and with SignalFx

Titanium 3.2 CLI - TiAppCamp2 - 11/2/2013

It's the way of the present - Why you should use plone.app.contenttypes

Flink Forward San Francisco 2019: Moving from Lambda and Kappa Architectures ...

Does Your Stuff Scale?

Tips For Maintaining OSS Projects

Terraform day 1

React state management with Redux and MobX

Component-first Applications

[FOSDEM 2020] Lazy distribution of container images

systemd @ Facebook in 2019

eZ Publish 5: from zero to automated deployment (and no regressions!) in one ...

Steamlining your puppet development workflow

Puppet Camp New York 2014: Streamlining Puppet Development Workflow

Platform Clouds, Containers, Immutable Infrastructure Oh My!

Terraform training 🎒 - Basic

遅延レイヤ取得による高互換コンテナ起動高速化手法

Containers in depth – Understanding how containers work to better work with c...

Terraform modules and best-practices - September 2018

Recently uploaded

UiPath Test Automation using UiPath Test Suite series, part 6

DianaGray10

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

“I’m still / I’m still / Chaining from the Block”

Claudio Di Ciccio

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Serial Arm Control in Real Time Presentation

tolgahangng

Mariano G Tinti - Decoding SpaceX

Mariano Tinti

HCL Notes and Domino License Cost Reduction in the World of DLAU

panagenda

Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/ The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this! We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model. Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward. These topics will be covered - Reducing license cost by finding and fixing misconfigurations and superfluous accounts - How do CCB and CCX licenses really work? - Understanding the DLAU tool and how to best utilize it - Tips for common problem areas, like team mailboxes, functional/test users, etc - Practical examples and best practices to implement right away

How to Get CNIC Information System with Paksim Ga.pptx

danishmna97

Driving Business Innovation: Latest Generative AI Advancements & Success Story

Safe Software

Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency. During the hour, we’ll take you through: Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board. Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes. Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI. We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI. This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!

Mind map of terminologies used in context of Generative AI

Kumud Singh

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

Neo4j

Dr. Sean Tan, Head of Data Science, Changi Airport Group Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.

AI 101: An Introduction to the Basics and Impact of Artificial Intelligence

IndexBug

How to use Firebase Data Connect For Flutter

Daiki Mogmet Ito

National Security Agency - NSA mobile device best practices

Quotidiano Piemontese

Building Production Ready Search Pipelines with Spark and Milvus

Zilliz

Artificial Intelligence for XMLDevelopment

Octavian Nadolu

In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject. We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup. Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved. The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring. The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise. By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.

GraphRAG for Life Science to increase LLM accuracy

Tomaz Bratanic

Microsoft - Power Platform_G.Aspiotis.pdf

Uni Systems S.M.S.A.

GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024

Neo4j

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 6

Removing Uninteresting Bytes in Software Fuzzing

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

“I’m still / I’m still / Chaining from the Block”

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Serial Arm Control in Real Time Presentation

Mariano G Tinti - Decoding SpaceX

HCL Notes and Domino License Cost Reduction in the World of DLAU

How to Get CNIC Information System with Paksim Ga.pptx

Driving Business Innovation: Latest Generative AI Advancements & Success Story

Mind map of terminologies used in context of Generative AI

GraphSummit Singapore | Enhancing Changi Airport Group's Passenger Experience...

AI 101: An Introduction to the Basics and Impact of Artificial Intelligence

How to use Firebase Data Connect For Flutter

National Security Agency - NSA mobile device best practices

Building Production Ready Search Pipelines with Spark and Milvus

Artificial Intelligence for XMLDevelopment

GraphRAG for Life Science to increase LLM accuracy

Microsoft - Power Platform_G.Aspiotis.pdf

GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024

Packer, Terraform & Jenkins

1. Packer, Terraform & Jenkins IaC & CD for small stacks Simon McCartney Twitter: @simonmcc

2. Background: Isn’t this a solved problem?

3. Background: Don’t containers solve everything?

4. Goal 1: Traceable Images • What was this image built from? • Project, branch, SHA, clean, dirty • Store it inside the image and externally via tags

5. Goal 2: Testable Images • Validate that an image is good • Chef’s InSpec for os/infra • App/Service specific testing

6. Goal 3: Self Contained • Monorepo FTW, single pipeline • Minimal Jenkins Plugins • Sanity wrappers for Terraform & Packer • Terraform for the app • Terraform for the build environment

7. Packer Wrapper – build.sh •Modeled on base & app AMI – overkill for this POC •Expose git SHA & clean/dirty state to packer for including in tags etc (‘cos CLI building should still be possible!) •Only build base/app AMI when necessary

8. tfw •Terraform wrapper •Map git branch to terraform workspace •Map git branch to tfvars •Expose git branch & sha to aid tagging & building unique resources (RDS Instance etc)

9. tfw •Configure terraform remote state render backend_config.tf AWS: create s3 bucket & DynamoDB Azure: create Resource Group & Storage Account

10. Jenkinsfile •Simple master/!master path

11. Tips •Watch out for account or globally unique resources (that’s why we expose branch & SHA1 to packer & terraform) •SHA1 for images in this POC is weak – it’s of a git object that “mostly” represents the image build source. •Jenkins aws-credentials & docker agent is broken :(

12. Links •https://github.com/simonmcc/cfgmgmtcmp-2019- iac-ci-cd •https://github.com/jenkins201/jenkins-container •Twitter: @simonmcc •https://www.axon.com

13. We’re hiring! SRE, Remote UK & EU

Editor's Notes

If you google enough, and have been coming to conferences like cfgmgmtcamp for a few years, it would be easy to think that VM image baking was a solved problem, we’ve been talking about it for ages, there are tools that do some of the essentials, but in my opinion, there was still a lot left as an exercise for the student. I started a new job recently, and we have a infrastructure that’s state of the art for the year 2010, terraform for cloud based infrastructure, puppet & salt for long lived VMs configuration management & orchestration, but this lead to pain & bad practices, like manually resizing VMs & editing the terraform to keep sync, adding VMs to cope with a demand spike was painful, removing them was even more painfuil (puppet certs, manual DNS, discovery via hierdata, again all solved problems in some way)
Nope. Many of our services are moving to k8s managed containers, but not everything will, for good & bad reasons. There are also some environments & companies where they’re just not interested in the k8s & PaaS overhead, they just want somewhere better to run their LAMP stack apps that fits with their appetite for change.

Packer, Terraform & Jenkins

Recommended

Recommended

More Related Content

Similar to Packer, Terraform & Jenkins

Similar to Packer, Terraform & Jenkins (20)

Recently uploaded

Recently uploaded (20)

Packer, Terraform & Jenkins

Editor's Notes