Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Best practices include collaborative approach to infrastructure provisioning, use of version control systems and prevent manual changes, as well as efficient management of boundaries between different teams, roles, applications and deployment tiers. In this session we will walk you through our journey from early serverless "never thought our dreams would ever come true" adopters to enterprise "why are we still dealing with this crap" developers. We will focus on lessons learned and best practices that goes above and beyond official documentation.
Pipenv, creado por Kenneth Reitz hace poco más de dos años, se ha convertido en el recurso oficial recomendado por Python para administrar dependencias de paquetes.
Pero todavía hay confusión sobre qué problemas resuelve y cómo es más útil que el flujo de trabajo estándar usando pip y un archivo requirements.txt. En esta charla se intentará explicar la solución a dichos problemas.
Project Flogo: Serverless Integration, Powered by Flogo and LambdaLeon Stigter
No matter the metric, serverless is definitely gaining interest. It’s the dream of every developer, supplying the ability to deploy services in the cloud in no time, automatically scale them, enjoy automagic management by a cloud provider—and, most important, keep it all cost effective! How does this dream become a reality?
This session covered what serverless is all about and the benefits of running your apps in the serverless environment. It covers the monoliths-microservices-functions progression and when, where, and why to use serverless architecture and how Project Flogo fits in to the overall picture
Kubernetes - 7 lessons learned from 7 data centers in 7 monthsMichael Tougeron
Learn from the rapid journey that Adobe Advertising Cloud took to reach a multi-cloud/multi-region deployment and the lessons learned along the way. Mike will cover 7 challenging scenarios with StatefulSets, GitOps, autoscaling for machine learning and auto-remediation. Go behind the scenes of the wins & failures to see how the engineering and platform teams prevented the derailing of their Kubernetes journey. From developing with spot instances in dev to multi-cloud in production, the cloud platform team has dealt with an interesting set of challenges. As Mike will show, even challenging requirements can lead to successful deployments.
INTERFACE, by apidays - The Evolution of Data Movement.pdfapidays
INTERFACE, by apidays - APIs: the next 10 years
June 8, 9 & 10 2022
The Evolution of Data Movement
Michel Tricot, Co-founder and CEO at Airbyte
------------
Check out our conferences at https://www.apidays.global/
Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8
Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io
Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/
Deep dive into the API industry with our reports:
https://www.apidays.global/industry-reports/
Subscribe to our global newsletter:
https://apidays.typeform.com/to/i1MPEW
We're connecting more and more systems together, and also responsible for upgrading existing integrations to unlock new value. In this conference presentation, I looked at why you modernize integration, things you need to consider, and what practices to employ.
Pipenv, creado por Kenneth Reitz hace poco más de dos años, se ha convertido en el recurso oficial recomendado por Python para administrar dependencias de paquetes.
Pero todavía hay confusión sobre qué problemas resuelve y cómo es más útil que el flujo de trabajo estándar usando pip y un archivo requirements.txt. En esta charla se intentará explicar la solución a dichos problemas.
Project Flogo: Serverless Integration, Powered by Flogo and LambdaLeon Stigter
No matter the metric, serverless is definitely gaining interest. It’s the dream of every developer, supplying the ability to deploy services in the cloud in no time, automatically scale them, enjoy automagic management by a cloud provider—and, most important, keep it all cost effective! How does this dream become a reality?
This session covered what serverless is all about and the benefits of running your apps in the serverless environment. It covers the monoliths-microservices-functions progression and when, where, and why to use serverless architecture and how Project Flogo fits in to the overall picture
Kubernetes - 7 lessons learned from 7 data centers in 7 monthsMichael Tougeron
Learn from the rapid journey that Adobe Advertising Cloud took to reach a multi-cloud/multi-region deployment and the lessons learned along the way. Mike will cover 7 challenging scenarios with StatefulSets, GitOps, autoscaling for machine learning and auto-remediation. Go behind the scenes of the wins & failures to see how the engineering and platform teams prevented the derailing of their Kubernetes journey. From developing with spot instances in dev to multi-cloud in production, the cloud platform team has dealt with an interesting set of challenges. As Mike will show, even challenging requirements can lead to successful deployments.
INTERFACE, by apidays - The Evolution of Data Movement.pdfapidays
INTERFACE, by apidays - APIs: the next 10 years
June 8, 9 & 10 2022
The Evolution of Data Movement
Michel Tricot, Co-founder and CEO at Airbyte
------------
Check out our conferences at https://www.apidays.global/
Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8
Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io
Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/
Deep dive into the API industry with our reports:
https://www.apidays.global/industry-reports/
Subscribe to our global newsletter:
https://apidays.typeform.com/to/i1MPEW
We're connecting more and more systems together, and also responsible for upgrading existing integrations to unlock new value. In this conference presentation, I looked at why you modernize integration, things you need to consider, and what practices to employ.
How open source is driving DevOps innovation: CloudOpen NA 2015Gordon Haff
It’s no coincidence that all the interest around DevOps today comes at a time when open source technologies and processes are so dominant in cloud computing, data storage and analysis, and--increasingly--in networking. Innovations in Linux and other projects, including containers, configuration management, and continuous integration, are what make DevOps workflows and portable application deployments possible. But it’s also the result of open source culture, practices, and the tools supporting those practices that have made iterative development and collaboration such a powerful model for creating great software in communities. And now, they’re also providing a template for how to develop and operate applications internally within enterprises. In this session, we will discuss how open source tools and practices can be applied to create effective DevOps workflows and practices.
Gerrit Analytics applied to Android source codeLuca Milanesio
GerritForge trialled the Gerrit Analytics plugin and ETL with the Android Open-Source Project code-base. The results of the trial have been presented at the Gerrit User Summit 2019 at Gothenburg and Sunnyvale CA. Find inside an overview of the problems involved, the solutions implemented and also the use of the pull-replication plugin to fetch the code from the official Android repository.
Nato nel 2008, il movimento DevOps promette di essere un approccio nuovo alle tematiche dello sviluppo, gestione e manutenzione di sistemi IT. Anche per cercare di capire che cosa vi fosse dietro questo movimento, se solo un fattore di moda o anche della sostanza, ho organizzato nel 2012 e nel 2013 un “Incontro DevOps Italia” (http://idi2013.devops.it e http://idi2014.devops.it) che ha raccolto un buon successo di pubblico e mi ha permesso di approfondire il tema con altri appassionati. Intendo parlare della mia visione del movimento, delle potenzialità e di alcuni aspetti critici.
Dipping Your Toes Into Cloud Native Application DevelopmentMatthew Farina
Presented at CloudDevelop 2016
Building cloud native applications in containers is a new hot topic. Netflix and Google are two prime examples that have been doing it successfully for some time. Some of the new exciting projects like Docker and Kubernetes are focused on cloud native applications in containers. There are supposed to be numerous benefits including the ability to scale applications out easily while doing development on small systems like laptops, the ability for the system to handle some operational problems, and the capability to safely deploy updates to production many times per day. But, what does this look like in practice and how do you start the move to cloud native and containerized applications? In this session we'll look at what makes up a cloud native application, how they work, and how you can start small. We'll look at applications from an architecture and process point of view along with how you can deploy them to AWS, Azure, or Google Cloud. You'll walk away ready to start development on a cloud native app.
Leading organizations around the globe are realizing that going cloud native has moved from a buzz-phrase to a business imperative. But many are wondering: How do I get there? What does it mean for IT? What even is this “cloud native” thing? Doesn’t using containers get me there?
Going cloud native means rethinking how we organize and deliver software. It means changing the structure of our organizations. It requires platform level thinking about how IT supports application teams. Most importantly, it requires focusing on time to business value.
In this session, you’ll learn what it means to be truly “Cloud Native”, why your organization needs to make the change, and how Cloud Foundry is the right platform for cloud native applications.
This was an introductory talk about Devops presented as a guest lecture for Melbourne University masters students on Apr 2019. The presentation goes through my personal journey of discovery of Devops at different stages and in organizations of different size and maturity levels. It also contains a step by step example on setting up a continuous delivery pipeline and keeping all the items as code in your git repository. It also presents devops practices based in these four dimensions:
Exchange of Crafts
A culture of shared ownership
Optimizing flow/delivery of value
Operating sustainable services
DevOps Transformation in Microsoft – Case Study by Ognjen Bajic and Ana Roje ...Bosnia Agile
Microsoft has embarked on its DevOps journey more than 8 years ago, when it started modeling a new set of engineering tools and practices called One Engineering System (1ES), fully based on their own platforms – Azure DevOps (ex Visual Studio Team Services - VSTS) and Azure. Since then, multiple product development teams have gone through a massive transition in terms of how they organize their work and deal with infrastructure, tools and processes.
This session tells the story of the transformation of 70+ teams with more than 800 people that develop Azure DevOps, both Service and Server (ex Team Foundation Server – TFS). These teams have been previously producing the monolithic 3-tier on premises TFS environment every couple of years. Nowadays, they moved to the cloud cadence and deliver daily updates to Azure DevOps Services, their global scale, always available, cloud SaaS service made up of dozens of micro-services. At the same time, they deliver the same capabilities on premises every couple of months.
Join us to learn how Microsoft’s teams transitioned from an inert waterfall to a highly performing agile organization, whereby drastically changing the way how they develop, test, deploy and monitor their products. We will explain how they manage large agile distributed product development, how they improved their development process by switching to GIT, how they changed their approach to testing and quality control and managed to take full control over their technical debt.
Lessons learned from 3 years inside cncf - WTF is Cloud Native, 4 September 2021Cheryl Hung
Cheryl Hung discusses her journey inside CNCF, home of Kubernetes and one of the top open source foundations, some hard truths about community, and thoughts about the future of cloud native.
oicheryl.com
If you need the support of the top python app development agency, you should pick one that uses the latest version of Python 3.11 released on 2nd March 2022.
Deep Learning State of the Art (2019) - MIT by Lex FridmanPeerasak C.
Deep Learning State of the Art (2019) - MIT by Lex Fridman
Watch video: https://youtu.be/53YvP6gdD7U
New lecture on recent developments in deep learning that are defining the state of the art in our field (algorithms, applications, and tools). This is not a complete list, but hopefully includes a good sampling of new exciting ideas. For more lecture videos visit our website or follow code tutorials on our GitHub repo.
INFO:
Website: https://deeplearning.mit.edu
CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
Containers, Serverless, Polyglot Development World, And Others…10 trends resh...PROIDEA
During this presentation, you will learn about the 10 changes that might reshape the developer tools market in the next 10 years. Jarek will discuss containers, serverless functions, and how it all supports an agile and CI/CD experience. The move to a polyglot development world means most applications will be written in a mix of languages, with developers favoring tools that help them navigate easily between languages. Jarek will also walk us through the evolution away from stand-alone developer workstations toward cloud-and-container based development environments offered as a service.
Hands-on with the Terraform Module for AWS Landing Zone Indy Cloud Conf 2020Mitoc Group
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Best practices include collaborative approach to infrastructure provisioning, use of version control systems and prevent manual changes, as well as efficient management of boundaries between different teams, roles, applications and deployment tiers. In this session we will walk you through our journey of helping customers set up AWS Landing Zone -- a secure, multi-account AWS environment based on AWS best practices. We will focus on lessons learned and best practices that goes above and beyond official documentation.
Hands-On Terraform Module for AWS Landing Zone at HashiTalks2020Mitoc Group
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Best practices include collaborative approach to infrastructure provisioning, use of version control systems and prevent manual changes, as well as efficient management of boundaries between different teams, roles, applications and deployment tiers. In this session we will walk you through our journey of helping customers set up AWS Landing Zone -- a secure, multi-account AWS environment based on AWS best practices. We will focus on lessons learned and best practices that goes above and beyond official documentation.
More Related Content
Similar to Terraform for Serverless - Best Practices
How open source is driving DevOps innovation: CloudOpen NA 2015Gordon Haff
It’s no coincidence that all the interest around DevOps today comes at a time when open source technologies and processes are so dominant in cloud computing, data storage and analysis, and--increasingly--in networking. Innovations in Linux and other projects, including containers, configuration management, and continuous integration, are what make DevOps workflows and portable application deployments possible. But it’s also the result of open source culture, practices, and the tools supporting those practices that have made iterative development and collaboration such a powerful model for creating great software in communities. And now, they’re also providing a template for how to develop and operate applications internally within enterprises. In this session, we will discuss how open source tools and practices can be applied to create effective DevOps workflows and practices.
Gerrit Analytics applied to Android source codeLuca Milanesio
GerritForge trialled the Gerrit Analytics plugin and ETL with the Android Open-Source Project code-base. The results of the trial have been presented at the Gerrit User Summit 2019 at Gothenburg and Sunnyvale CA. Find inside an overview of the problems involved, the solutions implemented and also the use of the pull-replication plugin to fetch the code from the official Android repository.
Nato nel 2008, il movimento DevOps promette di essere un approccio nuovo alle tematiche dello sviluppo, gestione e manutenzione di sistemi IT. Anche per cercare di capire che cosa vi fosse dietro questo movimento, se solo un fattore di moda o anche della sostanza, ho organizzato nel 2012 e nel 2013 un “Incontro DevOps Italia” (http://idi2013.devops.it e http://idi2014.devops.it) che ha raccolto un buon successo di pubblico e mi ha permesso di approfondire il tema con altri appassionati. Intendo parlare della mia visione del movimento, delle potenzialità e di alcuni aspetti critici.
Dipping Your Toes Into Cloud Native Application DevelopmentMatthew Farina
Presented at CloudDevelop 2016
Building cloud native applications in containers is a new hot topic. Netflix and Google are two prime examples that have been doing it successfully for some time. Some of the new exciting projects like Docker and Kubernetes are focused on cloud native applications in containers. There are supposed to be numerous benefits including the ability to scale applications out easily while doing development on small systems like laptops, the ability for the system to handle some operational problems, and the capability to safely deploy updates to production many times per day. But, what does this look like in practice and how do you start the move to cloud native and containerized applications? In this session we'll look at what makes up a cloud native application, how they work, and how you can start small. We'll look at applications from an architecture and process point of view along with how you can deploy them to AWS, Azure, or Google Cloud. You'll walk away ready to start development on a cloud native app.
Leading organizations around the globe are realizing that going cloud native has moved from a buzz-phrase to a business imperative. But many are wondering: How do I get there? What does it mean for IT? What even is this “cloud native” thing? Doesn’t using containers get me there?
Going cloud native means rethinking how we organize and deliver software. It means changing the structure of our organizations. It requires platform level thinking about how IT supports application teams. Most importantly, it requires focusing on time to business value.
In this session, you’ll learn what it means to be truly “Cloud Native”, why your organization needs to make the change, and how Cloud Foundry is the right platform for cloud native applications.
This was an introductory talk about Devops presented as a guest lecture for Melbourne University masters students on Apr 2019. The presentation goes through my personal journey of discovery of Devops at different stages and in organizations of different size and maturity levels. It also contains a step by step example on setting up a continuous delivery pipeline and keeping all the items as code in your git repository. It also presents devops practices based in these four dimensions:
Exchange of Crafts
A culture of shared ownership
Optimizing flow/delivery of value
Operating sustainable services
DevOps Transformation in Microsoft – Case Study by Ognjen Bajic and Ana Roje ...Bosnia Agile
Microsoft has embarked on its DevOps journey more than 8 years ago, when it started modeling a new set of engineering tools and practices called One Engineering System (1ES), fully based on their own platforms – Azure DevOps (ex Visual Studio Team Services - VSTS) and Azure. Since then, multiple product development teams have gone through a massive transition in terms of how they organize their work and deal with infrastructure, tools and processes.
This session tells the story of the transformation of 70+ teams with more than 800 people that develop Azure DevOps, both Service and Server (ex Team Foundation Server – TFS). These teams have been previously producing the monolithic 3-tier on premises TFS environment every couple of years. Nowadays, they moved to the cloud cadence and deliver daily updates to Azure DevOps Services, their global scale, always available, cloud SaaS service made up of dozens of micro-services. At the same time, they deliver the same capabilities on premises every couple of months.
Join us to learn how Microsoft’s teams transitioned from an inert waterfall to a highly performing agile organization, whereby drastically changing the way how they develop, test, deploy and monitor their products. We will explain how they manage large agile distributed product development, how they improved their development process by switching to GIT, how they changed their approach to testing and quality control and managed to take full control over their technical debt.
Lessons learned from 3 years inside cncf - WTF is Cloud Native, 4 September 2021Cheryl Hung
Cheryl Hung discusses her journey inside CNCF, home of Kubernetes and one of the top open source foundations, some hard truths about community, and thoughts about the future of cloud native.
oicheryl.com
If you need the support of the top python app development agency, you should pick one that uses the latest version of Python 3.11 released on 2nd March 2022.
Deep Learning State of the Art (2019) - MIT by Lex FridmanPeerasak C.
Deep Learning State of the Art (2019) - MIT by Lex Fridman
Watch video: https://youtu.be/53YvP6gdD7U
New lecture on recent developments in deep learning that are defining the state of the art in our field (algorithms, applications, and tools). This is not a complete list, but hopefully includes a good sampling of new exciting ideas. For more lecture videos visit our website or follow code tutorials on our GitHub repo.
INFO:
Website: https://deeplearning.mit.edu
CONNECT:
- If you enjoyed this video, please subscribe to this channel.
- Twitter: https://twitter.com/lexfridman
- LinkedIn: https://www.linkedin.com/in/lexfridman
- Facebook: https://www.facebook.com/lexfridman
- Instagram: https://www.instagram.com/lexfridman
Containers, Serverless, Polyglot Development World, And Others…10 trends resh...PROIDEA
During this presentation, you will learn about the 10 changes that might reshape the developer tools market in the next 10 years. Jarek will discuss containers, serverless functions, and how it all supports an agile and CI/CD experience. The move to a polyglot development world means most applications will be written in a mix of languages, with developers favoring tools that help them navigate easily between languages. Jarek will also walk us through the evolution away from stand-alone developer workstations toward cloud-and-container based development environments offered as a service.
Similar to Terraform for Serverless - Best Practices (20)
Hands-on with the Terraform Module for AWS Landing Zone Indy Cloud Conf 2020Mitoc Group
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Best practices include collaborative approach to infrastructure provisioning, use of version control systems and prevent manual changes, as well as efficient management of boundaries between different teams, roles, applications and deployment tiers. In this session we will walk you through our journey of helping customers set up AWS Landing Zone -- a secure, multi-account AWS environment based on AWS best practices. We will focus on lessons learned and best practices that goes above and beyond official documentation.
Hands-On Terraform Module for AWS Landing Zone at HashiTalks2020Mitoc Group
Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Best practices include collaborative approach to infrastructure provisioning, use of version control systems and prevent manual changes, as well as efficient management of boundaries between different teams, roles, applications and deployment tiers. In this session we will walk you through our journey of helping customers set up AWS Landing Zone -- a secure, multi-account AWS environment based on AWS best practices. We will focus on lessons learned and best practices that goes above and beyond official documentation.
The Ultimate Publisher's Need-to-Know Guide to Ad BlockersMitoc Group
For the past few years, publishers have been battling a trend that threatens to damage their traditional monetization / advertising model: adblocking. Check out this guide to get a better picture of how much adblockers have grown and how they’re currently affecting the online publishing industry.
How Serverless Computing Enables Microservices and Micropayment Mitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. AWS, through the serverless computing initiative, is changing this perception. In this session, Eugene Istrati from Mitoc Group will show hands-on how they are using serverless building blocks like S3, CloudFront, API Gateway, Lambda, DynamoDB, SQS, Elasticache – and this is only the foundation. As official AWS Technology and AWS Lambda partners, Mitoc Group focuses mainly on building new applications or transforming existing applications that achieve high scalability, high availability, high performance without breaking your bank. At the end, Eugene will demo AdTechMedia.io, an innovative Advertising Technology for Media companies, built on top of serverless components from AWS.
Eugene Istrati has spent over 15 years as a technologist in a variety of industries, including digital media, cloud computing, and reputation marketing. Before joining Mitoc Group as Technology Partner, Eugene worked as Director of Engineering at Hearst Corporation and Program Manager at Amazon Web Services. He holds a dual Bachelor of Science degree in Mathematics and Computer Science and a Master of Science degree in Computer Science and Engineering.
How Serverless Computing Enables Microservices and Micropayment Mitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. AWS, through the serverless computing initiative, is changing this perception. In this session, Eugene Istrati from Mitoc Group will show hands-on how they are using serverless building blocks like S3, CloudFront, API Gateway, Lambda, DynamoDB, SQS, Elasticache – and this is only the foundation. As official AWS Technology and AWS Lambda partners, Mitoc Group focuses mainly on building new applications or transforming existing applications that achieve high scalability, high availability, high performance without breaking your bank. At the end, Eugene will demo AdTechMedia.io, an innovative Advertising Technology for Media companies, built on top of serverless components from AWS.
Eugene Istrati has spent over 15 years as a technologist in a variety of industries, including digital media, cloud computing, and reputation marketing. Before joining Mitoc Group as Technology Partner, Eugene worked as Director of Engineering at Hearst Corporation and Program Manager at Amazon Web Services. He holds a dual Bachelor of Science degree in Mathematics and Computer Science and a Master of Science degree in Computer Science and Engineering.
Serverless Microservices - Real life story of a Web App that uses AngularJS, ...Mitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. What if we can change this perception and help developers design full-stack applications that are low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group using AngularJS, NodeJS and powered by abstracted services like AWS Lambda, Amazon CloudFront, Amazon DynamoDB, and so on.
Eugene Istrati, Technology Partner at Mitoc Group, will dive deep into their approach to microservices architecture using serverless platform from AWS and demonstrate how anyone can use serverless computing to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Building Scalable Web Applications using Microservices Architecture and Serve...Mitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. What if we can change this perception and help developers design full-stack applications that are low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group using AngularJS, NodeJS and powered by abstracted services like AWS Lambda, Amazon CloudFront, Amazon DynamoDB, and so on.
Eugene Istrati, Technology Partner at Mitoc Group, will dive deep into their approach to microservices architecture using serverless platform from AWS and demonstrate how anyone can use serverless computing to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Building Scalable Web Applications using Microservices Architecture and NodeJ...Mitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. What if we can change this perception and help developers design full-stack applications that are low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group using AngularJS, NodeJS and powered by abstracted services like AWS Lambda, Amazon CloudFront, Amazon DynamoDB, and so on.
Eugene Istrati, Technology Partner at Mitoc Group, will dive deep into their approach to microservices architecture using serverless platform from AWS and demonstrate how anyone can use serverless computing to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Serverless Microservices - Real life story of a Web App that uses AWS LambdaMitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. What if we can change this perception and help developers design full-stack applications that are low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group using AngularJS, NodeJS and powered by abstracted services like AWS Lambda, Amazon CloudFront, Amazon DynamoDB, and so on.
Eugene Istrati, Technology Partner at Mitoc Group, will dive deep into their approach to microservices architecture using serverless platform from AWS and demonstrate how anyone can use serverless computing to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Microservices Architecture for Content Management Systems using AWS Lambda an...Mitoc Group
Content Management Systems are by nature resource intensive, expensive to customize, and difficult to manage at scale. What if we can change this perception and help PHP / WordPress developers architect a content platform that is high performance and low cost, high security and low maintenance? This talk will focus on 3 key topics: 1) serverless environment, 2) microservices architecture and 3) hands-on demos. We will describe a serverless solution and propose a scalable architecture that will help WordPress community to adopt cloud-native approach without huge efforts or expensive resources allocation.
Eugene Istrati, Technology Partner at Mitoc Group, will dive deep into their approach to microservices architecture using serverless platform from AWS and demonstrate how anyone can use serverless computing to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Microservices Architecture for Web Applications using Serverless Computing wi...Mitoc Group
Scalable applications are by nature resource intensive, expensive to build and difficult to manage. What if we can change this perception and help developers design full-stack applications that are low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group using AngularJS, NodeJS and powered by abstracted services like AWS Lambda, Amazon CloudFront, Amazon DynamoDB, and so on.
Eugene Istrati, Technology Partner at Mitoc Group, will dive deep into their approach to microservices architecture using serverless platform from AWS and demonstrate how anyone can use serverless computing to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Microservices Architecture for Web Applications using AWS Lambda and moreMitoc Group
Digital platforms are by nature resource intensive, expensive to build, and difficult to manage at scale. What if we can change this perception and help developers architect a digital platform that is low cost and low maintenance, highly secure and highly performant? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group and powered by AWS abstracted services like AWS Lambda, Amazon CloudFront, and Amazon DynamoDB.
Eugene Istrati, Technology Partner at Mitoc Group will dive deep into their approach to microservices architecture on serverless environments and demonstrate how anyone can architect AWS abstracted services to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Microservices Architecture for Web Applications using Amazon AWS CloudMitoc Group
Large Web Applications are by nature resource intensive, expensive to customize, and difficult to manage at scale. What if we can change this perception and help developers architect a web application that is high performance and low cost, high security and low maintenance? This talk will focus on 3 key topics: 1) serverless infrastructure, 2) microservices architecture and 3) hands-on demos. We will describe a serverless solution and propose a scalable architecture that will help Generator Hub community to adopt cloud-native approach without huge efforts or expensive resources allocation.
Microservices Architecture for Content Management Systems using AWS Lambda an...Mitoc Group
Content Management Systems are by nature resource intensive, expensive to customize, and difficult to manage at scale. What if we can change this perception and help PHP / Drupal developers architect a content platform that is high performance and low cost, high security and low maintenance? This talk will focus on 3 key topics: 1) serverless environment, 2) microservices architecture and 3) hands-on demos. We will describe a serverless solution and propose a scalable architecture that will help Drupal community to adopt cloud-native approach without huge efforts or expensive resources allocation.
Microservices Architecture for Digital Platforms using Serverless AWSMitoc Group
Digital platforms are by nature resource intensive, expensive to build, and difficult to manage at scale. What if we can change this perception and help developers architect a digital platform that is low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group and powered by AWS abstracted services like AWS Lambda, Amazon CloudFront, and Amazon DynamoDB. Eugene Istrati, the CTO of Mitoc Group, will dive deep into their approach to microservices architecture on serverless environments and demonstrate how anyone can architect AWS abstracted services to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Microservices Architecture for MEAN Applications using Serverless AWSMitoc Group
Digital platforms are by nature resource intensive, expensive to build, and difficult to manage at scale. What if we can change this perception and help MEAN developers architect a digital platform that is low cost and low maintenance? This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group and powered by AWS abstracted services like AWS Lambda, Amazon CloudFront, and Amazon DynamoDB. Eugene Istrati, the CTO of Mitoc Group, will dive deep into their approach to microservices architecture on serverless environments and demonstrate how anyone can architect AWS abstracted services to achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Build Web Applications using Microservices on Node.js and Serverless AWSMitoc Group
This session describes the underlying architecture behind www.deep.mg, the microservices marketplace built by Mitoc Group and powered by abstracted services from AWS like Amazon S3, AWS Lambda, and Amazon DynamoDB. Eugene Istrati, the CTO of Mitoc Group, will dive deep into their approach to microservices architecture on serverless AWS and demonstrate how anyone can build web apps that achieve high scalability, high availability, and high performance without huge efforts or expensive resources allocation.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Zoom is a comprehensive platform designed to connect individuals and teams efficiently. With its user-friendly interface and powerful features, Zoom has become a go-to solution for virtual communication and collaboration. It offers a range of tools, including virtual meetings, team chat, VoIP phone systems, online whiteboards, and AI companions, to streamline workflows and enhance productivity.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
GraphSummit Paris - The art of the possible with Graph TechnologyNeo4j
Sudhir Hasbe, Chief Product Officer, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
Mobile App Development Company In Noida | Drona InfotechDrona Infotech
Looking for a reliable mobile app development company in Noida? Look no further than Drona Infotech. We specialize in creating customized apps for your business needs.
Visit Us For : https://www.dronainfotech.com/mobile-application-development/
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
Enterprise Resource Planning System includes various modules that reduce any business's workload. Additionally, it organizes the workflows, which drives towards enhancing productivity. Here are a detailed explanation of the ERP modules. Going through the points will help you understand how the software is changing the work dynamics.
To know more details here: https://blogs.nyggs.com/nyggs/enterprise-resource-planning-erp-system-modules/
Enhancing Research Orchestration Capabilities at ORNL.pdfGlobus
Cross-facility research orchestration comes with ever-changing constraints regarding the availability and suitability of various compute and data resources. In short, a flexible data and processing fabric is needed to enable the dynamic redirection of data and compute tasks throughout the lifecycle of an experiment. In this talk, we illustrate how we easily leveraged Globus services to instrument the ACE research testbed at the Oak Ridge Leadership Computing Facility with flexible data and task orchestration capabilities.
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
Atelier - Innover avec l’IA Générative et les graphes de connaissancesNeo4j
Atelier - Innover avec l’IA Générative et les graphes de connaissances
Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement.
Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
E-commerce Application Development Company.pdfHornet Dynamics
Your business can reach new heights with our assistance as we design solutions that are specifically appropriate for your goals and vision. Our eCommerce application solutions can digitally coordinate all retail operations processes to meet the demands of the marketplace while maintaining business continuity.
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
openEuler Case Study - The Journey to Supply Chain Security
Terraform for Serverless - Best Practices
1. DevOps Pro Europe, Vilnius
March 2019
Terraform for Serverless.
Best Practices. Lessons Learned.
2. DevOps Pro Europe, Vilnius
March 2019
DevOps Landscape. Daily Challenges.
3. DevOps Pro Europe, Vilnius
March 2019
If That Was Not Enough…
Introducing Serverless Landscape
4. DevOps Pro Europe, Vilnius
March 2019
Serverless Landscape. Daily Challenges.
5. DevOps Pro Europe, Vilnius
March 2019
Please excuse my honesty, but...
this feels like
TOO MUCH
to keep up with
6. DevOps Pro Europe, Vilnius
March 2019
Eugene
ISTRATI
@eistrati
About Presenter
• CTO, Tech Partner @ Mitoc Group
• Ex-AWS, ex-Hearst, ex-GrubHub
• Certified AWS Solutions Architect
• 20 Years in IT; 10 Years in Cloud
Computing; 5 Years in Enterprise IT
• Focusing on: Automation, DevOps,
Serverless
7. DevOps Pro Europe, Vilnius
March 2019
Agenda
Exponential increase
in services and tools
for cloud & serverless
Solution: we are striving
to make IT simpler,
reusable & cloud native
The Devil is in details
8. DevOps Pro Europe, Vilnius
March 2019
We Are Striving To…
Make IT Simpler
9. DevOps Pro Europe, Vilnius
March 2019
We Are Striving To…
Make IT Simpler
Yeah, Good Luck With That!
11. DevOps Pro Europe, Vilnius
March 2019
Prerequisites: Terraform For Serverless
1. Understand IT-as-a-Service Spectrum
1. Understand DevOps Spectrum
2. Understand Scope & Boundaries
12. DevOps Pro Europe, Vilnius
March 2019
1. Understand IT-as-a-Service Spectrum
On-Prem
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
Colocation
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
Hosting
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
IaaS
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
PaaS
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
SaaS
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
Managed by Customer Managed by Provider
14. DevOps Pro Europe, Vilnius
March 2019
Serverless in IT-as-a-Service Spectrum
On-Prem
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
Colocation
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
Hosting
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
IaaS
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
PaaS
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
SaaS
Data
Application
Databases
Operation System
Virtualization
Physical Servers
Network & Storage
Data Center
Managed by Customer Managed by Provider
Serverless Architecture
not in scope
16. DevOps Pro Europe, Vilnius
March 2019
3. Understand Scope & Boundaries
A
B C
17. DevOps Pro Europe, Vilnius
March 2019
Terraform For Serverless
A
B C
B == Terraform
A + B + C == Terraform
For Serverless
18. DevOps Pro Europe, Vilnius
March 2019
Terraform For Serverless
Best Practices.
Lessons Learned.
19. DevOps Pro Europe, Vilnius
March 2019
Best Practice #1 (of 8)
Adopt microservices architecture; aim for 1-to-1 relationship
between serverless resources and terraform configurations
20. DevOps Pro Europe, Vilnius
March 2019
Best Practice #1 (of 8)
Adopt microservices architecture; aim for 1-to-1 relationship
between serverless resources and terraform configurations
21. DevOps Pro Europe, Vilnius
March 2019
Best Practice #1 (of 8)
Adopt microservices architecture; aim for 1-to-1 relationship
between serverless resources and terraform configurations
22. DevOps Pro Europe, Vilnius
March 2019
Best Practice #2 (of 8)
Pass variables between resources using terraform remote state
23. DevOps Pro Europe, Vilnius
March 2019
Best Practice #2 (of 8)
Pass variables between resources using terraform remote state
24. DevOps Pro Europe, Vilnius
March 2019
Best Practice #2 (of 8)
Pass variables between resources using terraform remote state
25. DevOps Pro Europe, Vilnius
March 2019
Best Practice #3 (of 8)
Avoid code build using local provisioner or external data; instead
use hooks provided by terraform orchestration tools
26. DevOps Pro Europe, Vilnius
March 2019
Best Practice #3 (of 8)
Avoid code build using local provisioner or external data; instead
use hooks provided by terraform orchestration tools
27. DevOps Pro Europe, Vilnius
March 2019
Best Practice #3 (of 8)
Avoid code build using local provisioner or external data; instead
use hooks provided by terraform orchestration tools
28. DevOps Pro Europe, Vilnius
March 2019
Best Practice #4 (of 8)
Execute in parallel your automated terraform workflows; don’t
ignore terraform configurations dependencies
29. DevOps Pro Europe, Vilnius
March 2019
Best Practice #4 (of 8)
Execute in parallel your automated terraform workflows; don’t
ignore terraform configurations dependencies
30. DevOps Pro Europe, Vilnius
March 2019
Best Practice #4 (of 8)
Execute in parallel your automated terraform workflows; don’t
ignore terraform configurations dependencies
31. DevOps Pro Europe, Vilnius
March 2019
Best Practice #5 (of 8)
Optimize automated terraform workflows with git diff
32. DevOps Pro Europe, Vilnius
March 2019
Best Practice #5 (of 8)
Optimize automated terraform workflows with git diff
33. DevOps Pro Europe, Vilnius
March 2019
Best Practice #5 (of 8)
Optimize automated terraform workflows with git diff
34. DevOps Pro Europe, Vilnius
March 2019
Best Practice #6 (of 8)
Logically separate environments using terraform workspace
35. DevOps Pro Europe, Vilnius
March 2019
Best Practice #6 (of 8)
Logically separate environments using terraform workspace
dev == default
36. DevOps Pro Europe, Vilnius
March 2019
Best Practice #6 (of 8)
Logically separate environments using terraform workspace
dev == default
37. DevOps Pro Europe, Vilnius
March 2019
Best Practice #7 (of 8)
Overwrite environment specific values using variables precedence
38. DevOps Pro Europe, Vilnius
March 2019
Best Practice #7 (of 8)
Overwrite environment specific values using variables precedence
https://www.terraform.io/docs/configuration/variabl
es.html#variable-precedence
39. DevOps Pro Europe, Vilnius
March 2019
Best Practice #7 (of 8)
Overwrite environment specific values using variables precedence
https://www.terraform.io/docs/configuration/variabl
es.html#variable-precedence
40. DevOps Pro Europe, Vilnius
March 2019
Best Practice #8 (of 8)
Get comfortable with lots of terraform code; or use terrahub cli
41. DevOps Pro Europe, Vilnius
March 2019
Best Practice #8 (of 8)
Get comfortable with lots of terraform code; or use terrahub cli
$ find . -name ‘*.tf*’ | xargs wc -l | grep total
33998 total
$ find . -name ‘.terrahub*.yml’ | xargs wc -l | grep total
22118 total
42. DevOps Pro Europe, Vilnius
March 2019
Best Practice #8 (of 8)
Get comfortable with lots of terraform code; or use terrahub cli
$ find . -name ‘*.tf*’ | xargs wc -l | grep total
33998 total
$ find . -name ‘.terrahub*.yml’ | xargs wc -l | grep total
22118 total
43. DevOps Pro Europe, Vilnius
March 2019
Best Practices Summary
1. Adopt microservices architecture; aim for 1-to-1 relationship
1. Pass variables between resources using terraform remote state
1. Avoid code build using local provisioner or external data
1. Execute in parallel your automated terraform workflows; don’t ignore
terraform configurations dependencies
1. Optimize automated terraform workflows with git diff
1. Logically separate environments using terraform workspace
1. Overwrite environment specific values using variables precedence
1. Get comfortable with lots of terraform code; or use terrahub cli
44. DevOps Pro Europe, Vilnius
March 2019
Lessons Learned Summary
1. Adopt microservices architecture; aim for 1-to-1 relationship
1. Pass variables between resources using terraform remote state
1. Avoid code build using local provisioner or external data
1. Execute in parallel your automated terraform workflows; don’t ignore
terraform configurations dependencies
1. Optimize automated terraform workflows with git diff
1. Logically separate environments using terraform workspace
1. Overwrite environment specific values using variables precedence
1. Get comfortable with lots of terraform code; or use terrahub cli
https://github.com/TerraHubCorp
/terrahub
45. DevOps Pro Europe, Vilnius
March 2019
Terraform For Serverless
DEMO
https://github.com/TerraHubCorp/demo-
terraform-automation-aws
47. DevOps Pro Europe, Vilnius
March 2019
Terraform for Serverless.
Best Practices. Lessons Learned.
Eugene Istrati @eistrati
eugene@mitocgroup.com
Thank You!
Editor's Notes
Good afternoon!
This picture describes best what kind of challenges I’m dealing with on a daily basis. And I believe I’m not the only one. But, if you are lucky enough to use only 1 tool or service from each square, please reach out to me at the end, I’ll buy you a drink.
Like that was not enough, welcome to the landscape of serverless.
Invented by AWS, and joined by all other major providers like Microsoft Azure and Google Cloud, serverless is quickly becoming a parallel universe.
Please excuse my honesty but this feels like too much to keep up with, right?
My name is Eugene. I’m the chief technology officer of Mitoc Group. We are cloud native automation company, born in the cloud about the same time when AWS Lambda was launched in preview. These are my credentials.
And today I would like to discuss about how we felt overwhelmed by these constantly evolving services and tools, and what we did in order to keep up with latest emerging solutions that are compatible with legacy, as well as cutting edge technologies. The Devil is in details.
We strive to make IT simpler.
Yeah, right. I hope you’re smiling, otherwise my humor is outdated and I would be in a really awkward situation.
Nevertheless, a smart guy once said: When something is important enough, you do it even if the odds are not in your favor. History shows that he was right.
So, after a short description of the problem and a small explanation into the way we think about this problem, I’ll dive into Terraform for Serverless. Next, let’s make sure that we have the same understanding on the following 3 concepts: IT as a service, DevOps, and overall scope and boundaries.
First, IT as a service spectrum spreads across on-premises, collocation, hosting, infrastructure as a service, platform as a service and software as a service. As you can see, services are more or less the same in every category, but the difference is whether the customer is managing these services or not.
Now, speaking of serverless, I have seen several definitions out there, but my favorite one is this tweet by Netflix former chief architect, nowadays vice president at AWS, Adrian Cockroft. If your platform as a service can efficiently start instances in 20 milliseconds that run for half a second, then call it serverless.
Therefore, in serverless universe, we generally assume and agree among us that PaaS or SaaS are in scope, everything else is out of scope.
Second, DevOps spectrum includes all development operations that we as engineers must go through in order to manage our IT needs end to end. Therefore agile development, CI and CD are part of DevOps, and definitely not optional or out of scope.
And third, if we go back to devops landscape, mark deployment services as A, configuration management and provisioning as B, orchestration and scheduling as C.
Terraform is B, but Terraform for Serverless is a combination of A + B + C.
And there we go. This is the core of our presentation. So thank you for your patience so far. Now we are ready to talk about best practices and lessons learned from using Terraform for Serverless.
Best practice number 1 is to adopt microservices architecture. This might sound trivial to many of you, but I still come across engineers who don’t apply this in practice.
Heads up disclosure: I will provide examples for each best practice. To be clear, these examples are not wrong, but rather an indication that something can be done better. So, in this case of microservices architecture, instead of putting all your resources for a service in 1 terraform configuration file.
Break it down into smaller pieces or components, aim for 1-to-1 relationship between serverless resources and terraform configuration files.
After you break down into microservices, use terraform built in functionality to pass variables between resources. This built in feature is called terraform remote state.
So, if you have 2 services that need to reference to the same variable, instead of hardcoding it in 2 different configuration files...
Define it in service1, enable terraform backend like we do it in this example via S3, expose it in output and reference it in service2 using data terraform remote state.
Third best practice – avoid using local provisioner or external data to execute external scripts. In our experience, it was extremely difficult to identify issues when something goes wrong.
In this example above, data external will trigger the execution of build.js node script every time lambda function resource is created or updated. Fun fact, our team established as an internal rule to avoid this kind of functionality.
Instead, we use terraform orchestration tools like terragrunt, or atlantis, or terrahub – our own home grown orchestration and automation tool.
When we developed serverless applications, we ended up with a lot of terraform components. Unfortunately, automated terraform workflows are not built into this open source tool. Either you do it yourself, use one of previously mentioned orchestration tools or switch to using terraform enterprise.
To be factually correct, I must mention that terraform has a built in option called parallelism, but it does not work across different terraform components.
We ended up adding this feature into terrahub cli. This is one of the most used commands in this tool.
Very often, we don’t need to run automated terraform workflows for all terraform components. So instead of specifying a static list of components, we use git diff to dynamically see what changed between commits or in a specific pull request, and execute automated workflow on the map of components that changed.
Like in the example above. Instead of running automation for all 9 components in my repository...
We used git diff and reduced the list of components down to 2. If you ask me, this is a very nice optimization of almost 80%
One way to separate environments is to have separate code base for each environment. We don’t do that. Instead, we use terraform workspace to logically separate environments.
Like in the example above, instead of keeping 4 different versions of pretty much the same configs for our dev environment, and test, and stage and prod...
We use default terraform workspace as our de facto dev environment and values or variables that change for specific environment, we store them in workspace folder. When using terraform orchestration tools like terrahub cli, switching between environments is as easy as adding --env option in command line.
Another best practice is built around terraform variables precedence.
For example, if you have same variables used across different terraform workspaces, instead of storing the same values across different tfvar files...
We store everything in default tfvar file corresponding to default terraform workspace, and only environment specific values that are different from default terraform workspace, we add them into separate tfvar files. When running terraform commands like plan or apply, default tfvar file comes first and specific tfvar file comes second. Please follow this link to learn more about variable precedence.
And, finally, get used to a lot of terraform. Or don’t.
As you can see in the example above, we have almost 34 thousand lines of code for around 100 terraform components.
But when we switched to YAML based version using terrahub cli, we ended up with a little bit over 22 thousand lines of code. That’s over 30% reduction of the original terraform codebase. Another benefit of using terrahub cli is that we are seamlessly transforming YAML to HCL and the other way around.
To summarize, today we have discussed 8 best practices. All lessons that we have learned while working with terraform for serverless...
We have included them into this open source tool for terraform automation and orchestration called terrahub cli. If you like this project, please show some support and leave us a star on GitHub.
Now, if we have some time left, let’s try to do a demo.
At the end, I’d like to leave you with this quote from Andy Jassy, CEO of AWS. There is no compression algorithm for experience.