This document provides a summary of best practices for DevOps as outlined by Erik Osterman of Cloud Posse. It discusses practices across organizational structure, software development, infrastructure automation, monitoring and security. Some key best practices include: establishing a makers culture with uninterrupted focus time for developers; using containers for local development environments and tools; strict branch protection and pull requests for code changes; immutable infrastructure with infrastructure as code; actionable alerts and post-mortems for monitoring; and identity-aware access, temporary credentials, and multi-factor authentication for security. The document aims to share proven strategies that help achieve reliability, speed, ease of use and affordability of systems.
Secrets are any sensitive piece of information (like a password, API token, TLS private key) that must be kept safe. This presentation is a practical guide covering what we've done at Cloud Posse to lock down secrets in production. It includes our answer to avoid the same pitfalls that Shape Shift encountered when they were hacked. The techniques presented are compatible with automated cloud environments and even legacy systems.
Learn how Cloud Posse recently architected and implemented Wordpress for massive scale on Amazon EC2. We'll show you exactly the tools that we used and our recipe to both secure and power Wordpress setups on AWS using Elastic Beanstalk, EFS, CodePipeline, Memcached, Aurora and Varnish.
How to implement data encryption at rest in compliance with enterprise requir...Steffen Mazanek
This presentation has been given at the #AWS #Community day #2019 in #Hamburg by Steffen Mazanek and Louay Mresheh. Title has been "How to implement data encryption at rest in compliance with enterprise requirements"
In recent months, Deep Learning has become the hottest topic in the IT industry. However, its arcane jargon and its intimidating equations often discourage software developers, who wrongly think that they’re “not smart enough”. Through code-level demos based on Apache MXNet, we’ll demonstrate how to build, train and use models based on different types of networks: multi-layer perceptrons, convolutional neural networks and long short-term memory networks. Finally, we’ll share some optimization tips which will help improve the training speed and the performance of your models.
Secrets are any sensitive piece of information (like a password, API token, TLS private key) that must be kept safe. This presentation is a practical guide covering what we've done at Cloud Posse to lock down secrets in production. It includes our answer to avoid the same pitfalls that Shape Shift encountered when they were hacked. The techniques presented are compatible with automated cloud environments and even legacy systems.
Learn how Cloud Posse recently architected and implemented Wordpress for massive scale on Amazon EC2. We'll show you exactly the tools that we used and our recipe to both secure and power Wordpress setups on AWS using Elastic Beanstalk, EFS, CodePipeline, Memcached, Aurora and Varnish.
How to implement data encryption at rest in compliance with enterprise requir...Steffen Mazanek
This presentation has been given at the #AWS #Community day #2019 in #Hamburg by Steffen Mazanek and Louay Mresheh. Title has been "How to implement data encryption at rest in compliance with enterprise requirements"
In recent months, Deep Learning has become the hottest topic in the IT industry. However, its arcane jargon and its intimidating equations often discourage software developers, who wrongly think that they’re “not smart enough”. Through code-level demos based on Apache MXNet, we’ll demonstrate how to build, train and use models based on different types of networks: multi-layer perceptrons, convolutional neural networks and long short-term memory networks. Finally, we’ll share some optimization tips which will help improve the training speed and the performance of your models.
Whether you are building an e-commerce site or a business application, security is a key consideration when architecting your website or application. In this session, you will learn more about some of the things CloudFront does behind the scenes to protect the delivery of your content such as OCSP Stapling and Perfect Forward Secrecy. We will also share best practices on how you can use CloudFront to securely deliver content end-to-end, control who accesses your content, how to shield your origins from the Internet, and getting a A+ on SSL labs.
This is a great session for those new to this this tooll. PalominoDB's Ben Black will cover common tasks in RDS and gotchas for DBA's that are new to RDS.
DevOps (Continuous Integrations, Continuous Delivery & Continuous deployment using Jenkins and Visual studio team services, setting up VTST build Agents, Integrating VSTS with SonarQube, NDepend,) , Complete automation of pushing code into VSTS from Visual Studio, Building Code by a Jenkin Server hosted on Azure and pushing that successful build on to Azure Web App via Release Pipeline or directly from Jenkins,VSTS Default agents, Setting up local agent from scratch, Setting up agents for code build, VSTS, Visual Studio Online Agents, Agent Pools, Hosted Agents, Hosted VS2017. Hosted Linux Agents, Setting up agent on VS Dev Test Labs, Setting up Template Parameters for Continuos Pipeline, Build Agent Creation Dynamically, Random Machine Name, Random Passwords, Dynamic Agent creation in VS Dev Test labs, Sonarcube, Code quality, Code Analysis, MSBuild, Integrate VSTS Build with NDepend, Package manager, Monolithic Architecture, Nuget, Package management, Npm js.com, Semantic versioning, Creating a nuget package, nuspec file, GitVersion Plugin, FeedURL, Chocolatey for package management, Chocolatey, chocolatey workflow,
Aws ebs snapshot with iam cross account accessNaoya Hashimoto
* Create EBS Snapshot to retrieve other AWS account's EBS snapshot over IAM Cross Account Access.
* Prepare EC2 instance with Amazon Linux AM and install Ruby script with AWS SDK for Ruby to create EBS snapshot.
* Use IAM Role to assume the role to create EBS snapshot.
SSM combined with Simple AD are powerful tools that can help you and your organization get away from things like every user using the Administrator username and password to get into the instances.
These slides are from the AWS Atlanta Meetup group's February 2016 meeting -http://www.meetup.com/AWS-Atlanta/
Want to learn more about running containers on AWS? In this we cover best practices for running container orchestration systems such as Amazon EC2 Container Service (ECS) and Kubernetes (K8s) on AWS.
Running and managing large scale applications with microservices architectures is difficult and often requires operating complex container management infrastructure. Amazon EC2 Container Service (ECS) is a highly scalable, high performance service for running and managing Docker applications. In this webinar, we will walk through a number of patterns and tools used by our customers to run their applications on Amazon ECS. We will show you how to set up, manage and scale your Amazon ECS resources, keep them secure and deploy your applications to an Amazon ECS cluster. We will also provide best practices for monitoring, logging and service discovery.
Learning Objectives:
• Learn how to set up and manage Amazon ECS for production applications
• Learn how to schedule containers on production clusters using Amazon ECS
Who Should Attend:
•Developers, DevOps, Sys Admin
(SEC404) Incident Response in the Cloud | AWS re:Invent 2014Amazon Web Services
You've employed the practices outlined for incident detection, but what do you do when you detect an incident in the cloud? This session walks you through a hypothetical incident response on AWS. Learn to leverage the unique capabilities of the AWS environment when you respond to an incident, which in many ways is similar to how you respond to incidents in your own infrastructure. This session also covers specific environment recovery steps available on AWS.
A presentation on the microservice Lambda by AWS for creating Lambda packages in the Python language and examples of good and bad use cases for using lambda.
Presented by the AWS Atlanta Meetup group
Getting Started with Amazon ECS: Run Docker Containers on AWSTung Nguyen
Learn about AWS ECS, the native AWS container management service. ECS is the easiest way to run Docker containers on AWS. AWS runs and maintains the orchestration software, handles scaling and simplifies the process. In this DevTalk, you'll get an introduction to the ECS terms and concepts. We'll cover the ECS ecosystem and list some of the tools in the space. Then we'll jump into a demo using an ECS deploy tool called ufo. As a bonus, we'll talk about the strategies to optimize costs with ECS.
(WEB203) Building a Website That Costs Pennies to Operate | AWS re:Invent 2014Amazon Web Services
Amazon S3 gives you the ability to serve files from your Amazon S3 buckets. This session shows you how to set up a website with Amazon S3 to serve your static content. We show how you can use open source tools like Jekyll and Octopress to run a blog on your static site. Finally, you see how you can make that site more dynamic using other AWS products and the AWS SDK for JavaScript.
- TeamSQL AWS Architecture
- VPC Introduction (Public, private subnets) and Demo
- EC2 Introduction and Demo
- RDS Introduction and Demo
- Introduction to Cloudformation
- A simple Cloudformation Script and make it live (Creating EC2 with Cloudformation)
- Deleting Cloudformation Stack
- More advanced Cloudformation Script and make it live
(Cloudformation parameters, VPC, public, private subnets, RDS, ElasticBeanstalk, ElastiCache)
- Updating Cloudformation Stack
- Hands on - Advanced Cloudformation Script
Infrastructure as Code: Manage your Architecture with GitDanilo Poccia
With the AWS Cloud you have an on-demand, programmable infrastructure that you can manage using tools and practices from software development. You can create resources when you need and dispose of them when you don’t. Using Amazon CloudFormation you can describe your architecture in text files. To change your infrastructure, you edit those files. Having application and infrastructure code in a single, robust, versioned repository like Git gives a lot of advantages. Using AWS Elastic Beanstalk you can link your Git branches to different infrastructure environments (e.g. test, production) and automate deployments. You can create test environments on-demand, even for a short time. Instead of continuously update your resources, you can recreate them quickly from scratch, simplifying lifecycle management and making deployments immutable. As a result, you have more time to focus on the unique features of your application.
Whether you are building an e-commerce site or a business application, security is a key consideration when architecting your website or application. In this session, you will learn more about some of the things CloudFront does behind the scenes to protect the delivery of your content such as OCSP Stapling and Perfect Forward Secrecy. We will also share best practices on how you can use CloudFront to securely deliver content end-to-end, control who accesses your content, how to shield your origins from the Internet, and getting a A+ on SSL labs.
This is a great session for those new to this this tooll. PalominoDB's Ben Black will cover common tasks in RDS and gotchas for DBA's that are new to RDS.
DevOps (Continuous Integrations, Continuous Delivery & Continuous deployment using Jenkins and Visual studio team services, setting up VTST build Agents, Integrating VSTS with SonarQube, NDepend,) , Complete automation of pushing code into VSTS from Visual Studio, Building Code by a Jenkin Server hosted on Azure and pushing that successful build on to Azure Web App via Release Pipeline or directly from Jenkins,VSTS Default agents, Setting up local agent from scratch, Setting up agents for code build, VSTS, Visual Studio Online Agents, Agent Pools, Hosted Agents, Hosted VS2017. Hosted Linux Agents, Setting up agent on VS Dev Test Labs, Setting up Template Parameters for Continuos Pipeline, Build Agent Creation Dynamically, Random Machine Name, Random Passwords, Dynamic Agent creation in VS Dev Test labs, Sonarcube, Code quality, Code Analysis, MSBuild, Integrate VSTS Build with NDepend, Package manager, Monolithic Architecture, Nuget, Package management, Npm js.com, Semantic versioning, Creating a nuget package, nuspec file, GitVersion Plugin, FeedURL, Chocolatey for package management, Chocolatey, chocolatey workflow,
Aws ebs snapshot with iam cross account accessNaoya Hashimoto
* Create EBS Snapshot to retrieve other AWS account's EBS snapshot over IAM Cross Account Access.
* Prepare EC2 instance with Amazon Linux AM and install Ruby script with AWS SDK for Ruby to create EBS snapshot.
* Use IAM Role to assume the role to create EBS snapshot.
SSM combined with Simple AD are powerful tools that can help you and your organization get away from things like every user using the Administrator username and password to get into the instances.
These slides are from the AWS Atlanta Meetup group's February 2016 meeting -http://www.meetup.com/AWS-Atlanta/
Want to learn more about running containers on AWS? In this we cover best practices for running container orchestration systems such as Amazon EC2 Container Service (ECS) and Kubernetes (K8s) on AWS.
Running and managing large scale applications with microservices architectures is difficult and often requires operating complex container management infrastructure. Amazon EC2 Container Service (ECS) is a highly scalable, high performance service for running and managing Docker applications. In this webinar, we will walk through a number of patterns and tools used by our customers to run their applications on Amazon ECS. We will show you how to set up, manage and scale your Amazon ECS resources, keep them secure and deploy your applications to an Amazon ECS cluster. We will also provide best practices for monitoring, logging and service discovery.
Learning Objectives:
• Learn how to set up and manage Amazon ECS for production applications
• Learn how to schedule containers on production clusters using Amazon ECS
Who Should Attend:
•Developers, DevOps, Sys Admin
(SEC404) Incident Response in the Cloud | AWS re:Invent 2014Amazon Web Services
You've employed the practices outlined for incident detection, but what do you do when you detect an incident in the cloud? This session walks you through a hypothetical incident response on AWS. Learn to leverage the unique capabilities of the AWS environment when you respond to an incident, which in many ways is similar to how you respond to incidents in your own infrastructure. This session also covers specific environment recovery steps available on AWS.
A presentation on the microservice Lambda by AWS for creating Lambda packages in the Python language and examples of good and bad use cases for using lambda.
Presented by the AWS Atlanta Meetup group
Getting Started with Amazon ECS: Run Docker Containers on AWSTung Nguyen
Learn about AWS ECS, the native AWS container management service. ECS is the easiest way to run Docker containers on AWS. AWS runs and maintains the orchestration software, handles scaling and simplifies the process. In this DevTalk, you'll get an introduction to the ECS terms and concepts. We'll cover the ECS ecosystem and list some of the tools in the space. Then we'll jump into a demo using an ECS deploy tool called ufo. As a bonus, we'll talk about the strategies to optimize costs with ECS.
(WEB203) Building a Website That Costs Pennies to Operate | AWS re:Invent 2014Amazon Web Services
Amazon S3 gives you the ability to serve files from your Amazon S3 buckets. This session shows you how to set up a website with Amazon S3 to serve your static content. We show how you can use open source tools like Jekyll and Octopress to run a blog on your static site. Finally, you see how you can make that site more dynamic using other AWS products and the AWS SDK for JavaScript.
- TeamSQL AWS Architecture
- VPC Introduction (Public, private subnets) and Demo
- EC2 Introduction and Demo
- RDS Introduction and Demo
- Introduction to Cloudformation
- A simple Cloudformation Script and make it live (Creating EC2 with Cloudformation)
- Deleting Cloudformation Stack
- More advanced Cloudformation Script and make it live
(Cloudformation parameters, VPC, public, private subnets, RDS, ElasticBeanstalk, ElastiCache)
- Updating Cloudformation Stack
- Hands on - Advanced Cloudformation Script
Infrastructure as Code: Manage your Architecture with GitDanilo Poccia
With the AWS Cloud you have an on-demand, programmable infrastructure that you can manage using tools and practices from software development. You can create resources when you need and dispose of them when you don’t. Using Amazon CloudFormation you can describe your architecture in text files. To change your infrastructure, you edit those files. Having application and infrastructure code in a single, robust, versioned repository like Git gives a lot of advantages. Using AWS Elastic Beanstalk you can link your Git branches to different infrastructure environments (e.g. test, production) and automate deployments. You can create test environments on-demand, even for a short time. Instead of continuously update your resources, you can recreate them quickly from scratch, simplifying lifecycle management and making deployments immutable. As a result, you have more time to focus on the unique features of your application.
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...Amazon Web Services
Organizations around the globe are leveraging the cloud to accomplish world-changing missions. This session will address how AWS can help organizations put more money toward their mission and scale outreach and operations to achieve more with less. Hear some of AWS’s most advanced customers on how their organizations handle DevOps, continuous integration and deployment. Learn how these practices allow them to rapidly develop, iterate, test and deploy highly-scalable web applications and core operational systems on AWS. The discussion will focus on best practices, lessons learned, and the specific technologies and services they use.
Integrating-Cloud-Development-Security-And-Operations.pdfAmazon Web Services
Managing infrastructure as code has become an important process in scaling software organizations. This brings many software development processes and ideas to operations, including version control, automated testing, configuration management and reliable duplication. Programmable infrastructure becomes invaluable as application services grows, in quantity and granularity, in a growing company.
Automating the provisioning, configuration and deployment of complex applications requires some design choices on top of AWS services. This presentation discusses how to implement modularity, reliability and security into continuous delivery pipelines ("DevSecOps"). Learn how to automate application delivery using AWS CloudFormation and other tools from Amazon Web Services.
Talk given at ISC2 Secure SDLC event in Austin, TX
The release velocity for our applications is increasing, often leaving security testing behind. In some cases, the security team ends up being the bottleneck. That's bad. In an idyllic world, security testing would happen earlier in the development lifecycle, but lets do one better. Lets do security testing on every code change. Using automation tooling and DevOps practices, this talk will help you tune security testing to your release cadence and more importantly help you deliver more rugged software.
Experiences using CouchDB inside Microsoft's Azure teamBrian Benz
Co-presented with Will Perry (@willpe). Real-world experiences using CouchDB inside Microsoft, and also how to get started with CouchDB on Microsoft Azure.
DEV326_DevOps Essentials An Introductory Workshop on CICD PracticesAmazon Web Services
In a few hours, quickly learn how to effectively leverage various AWS services to improve developer productivity and reduce the overall time to market new product capabilities. In this workshop, we demonstrate a prescriptive approach to incrementally adopt and embrace some of the best practices around continuous integration/continuous delivery (CI/CD) using AWS developer tools and third-party solutions. The tools include source control systems including GitHub and AWS CodeCommit, AWS CodeBuild, AWS CodePipeline, and AWS CodeDeploy, to name a few.
You've heard about Continuous Integration and Continuous Deilvery but how do you get code from your machine to production in a rapid, repeatable manner? Let a build pipeline do the work for you! Sam Brown will walk through the how, the when and the why of the various aspects of a Contiuous Delivery build pipeline and how you can get started tomorrow implementing changes to realize build automation. This talk will start with an example pipeline and go into depth with each section detailing the pros and cons of different steps and why you should include them in your build process.
DevOps Fest 2020. immutable infrastructure as code. True story.Vlad Fedosov
In this talk I’ll explain how we went from classic Pet servers to immutable infrastructure, fully described as code, with Cattle instances. I’ll also share which tools we use and how we evolved our experience with them.
Continues Integration and Continuous Delivery with Azure DevOps - Deploy Anyt...Janusz Nowak
Continues Integration and Continuous Delivery with Azure DevOps - Deploy Anything to Anywhere with Azure DevOps
Janusz Nowak
@jnowwwak
https://www.linkedin.com/in/janono
https://github.com/janusznowak
https://blog.janono.pl
What does it take to get an application into production? Many processes, tools and automation surround that application to deliver it to the customer. As it becomes more common for development teams to autonomously deliver and run their software, the focus of the traditional operational teams shifts towards an as-a-service mindset. But how is such a team positioned within the company? And is Platform Engineering any different from Software Engineering?
In this talk I’ll share my experiences as a platform engineer and explain why I believe that every company should be conscious about why and how to setup this responsibility. I’ll also discuss the biggest challenges surrounding it - and how to tackle them.
Unlimited Staging Environments on KubernetesErik Osterman
How to run complete, disposable apps on Kubernetes for Staging and Development
What if you could rapidly spin up new environments in a matter of minutes entirely from scratch, triggered simply by the push of a button or automatically for every Pull Request or Branch. Would that be cool?
That’s what we thought too! Companies running complex microservices architectures need a better way to do QA, prototype new features & discuss changes. We want to show that there’s a simpler way to collaborate and it’s available today if you’re running Kubernetes.
Tune in to learn how you can assemble 100% Open Source components with a CodeFresh CI/CD Pipeline to deploy your full stack for any branch and expose it on a unique URL that you can share. Not only that, we ensure that it’s fully integrated with CI/CD so console expertise is not required to push updates. Empower designers and front-end developers to push code freely. Hand it over to your sales team so they can demo upcoming features for customers! The possibilities are truly unlimited. =)
A new movement is taking cloud by storm; Docker is evolving the way services are deployed by organizations so that they can operate more efficiently at scale — both in the cloud and on bare metal. In the same way shipping containers revolutionized the cargo industry, cheap, zero-penalty Linux Containers (LXC) are like shrink-wrapped VMs but without the fat. What’s not obvious, however, is how to roll your own Docker deployments and all tools you’ll need to leverage along the way.
This discussion will cover:
• Principles of Immutable Infrastructure
• Docker Basics
• Docker for Dev & QA
• Docker in Production
• Business Drivers
• Answering the Question: Is Docker Ready for Prime Time?
An Ensemble Core with Docker - Solving a Real Pain in the PaaS Erik Osterman
Docker by itself is only an engine powering containers. You need a containership to run it in production. CoreOS is a purpose-built containership that powers Docker conatiners, however, without higher-level orchestration managing hundreds or thousands of containers is not manageable. Ensemble is the answer for running containers at scale on top of CoreOS.
Docker Demystified - Virtual VMs without the FatErik Osterman
DevOps guru Erik Osterman has been at the forefront of large-scale cloud architectures as the Director of Cloud Architecture for CBS Interactive and advisor for numerous successful startups. Now he’s ready to show you why Docker is all the rage.
A new movement is taking cloud by storm; Docker is evolving the way services are deployed by organizations so that they can operate more efficiently at scale — both in the cloud and on bare metal. In the same way shipping containers revolutionized the cargo industry, cheap, zero-penalty Linux Containers (LXC) are like shrink-wrapped VMs but without the fat. What’s not obvious, however, is how to roll your own Docker deployments and all tools you’ll need to leverage along the way.
Tune in to learn how you too can run a micro services architecture that supports thousands of containers controlled effortlessly from your laptop’s command line.
This webinar is free to attend and will cover:
• Principles of Immutable Infrastructure
• Docker Basics
• Docker for Dev & QA
• Docker in Production
• Business Drivers
• Answering the Question: Is Docker Ready for Prime Time?
Webcast at http://webcast.cloudposse.com/
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
Overview of the fundamental roles in Hydropower generation and the components involved in wider Electrical Engineering.
This paper presents the design and construction of hydroelectric dams from the hydrologist’s survey of the valley before construction, all aspects and involved disciplines, fluid dynamics, structural engineering, generation and mains frequency regulation to the very transmission of power through the network in the United Kingdom.
Author: Robbie Edward Sayers
Collaborators and co editors: Charlie Sims and Connor Healey.
(C) 2024 Robbie E. Sayers
TECHNICAL TRAINING MANUAL GENERAL FAMILIARIZATION COURSEDuvanRamosGarzon1
AIRCRAFT GENERAL
The Single Aisle is the most advanced family aircraft in service today, with fly-by-wire flight controls.
The A318, A319, A320 and A321 are twin-engine subsonic medium range aircraft.
The family offers a choice of engines
Student information management system project report ii.pdfKamal Acharya
Our project explains about the student management. This project mainly explains the various actions related to student details. This project shows some ease in adding, editing and deleting the student details. It also provides a less time consuming process for viewing, adding, editing and deleting the marks of the students.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
CFD Simulation of By-pass Flow in a HRSG module by R&R Consult.pptxR&R Consult
CFD analysis is incredibly effective at solving mysteries and improving the performance of complex systems!
Here's a great example: At a large natural gas-fired power plant, where they use waste heat to generate steam and energy, they were puzzled that their boiler wasn't producing as much steam as expected.
R&R and Tetra Engineering Group Inc. were asked to solve the issue with reduced steam production.
An inspection had shown that a significant amount of hot flue gas was bypassing the boiler tubes, where the heat was supposed to be transferred.
R&R Consult conducted a CFD analysis, which revealed that 6.3% of the flue gas was bypassing the boiler tubes without transferring heat. The analysis also showed that the flue gas was instead being directed along the sides of the boiler and between the modules that were supposed to capture the heat. This was the cause of the reduced performance.
Based on our results, Tetra Engineering installed covering plates to reduce the bypass flow. This improved the boiler's performance and increased electricity production.
It is always satisfying when we can help solve complex challenges like this. Do your systems also need a check-up or optimization? Give us a call!
Work done in cooperation with James Malloy and David Moelling from Tetra Engineering.
More examples of our work https://www.r-r-consult.dk/en/cases-en/
COLLEGE BUS MANAGEMENT SYSTEM PROJECT REPORT.pdfKamal Acharya
The College Bus Management system is completely developed by Visual Basic .NET Version. The application is connect with most secured database language MS SQL Server. The application is develop by using best combination of front-end and back-end languages. The application is totally design like flat user interface. This flat user interface is more attractive user interface in 2017. The application is gives more important to the system functionality. The application is to manage the student’s details, driver’s details, bus details, bus route details, bus fees details and more. The application has only one unit for admin. The admin can manage the entire application. The admin can login into the application by using username and password of the admin. The application is develop for big and small colleges. It is more user friendly for non-computer person. Even they can easily learn how to manage the application within hours. The application is more secure by the admin. The system will give an effective output for the VB.Net and SQL Server given as input to the system. The compiled java program given as input to the system, after scanning the program will generate different reports. The application generates the report for users. The admin can view and download the report of the data. The application deliver the excel format reports. Because, excel formatted reports is very easy to understand the income and expense of the college bus. This application is mainly develop for windows operating system users. In 2017, 73% of people enterprises are using windows operating system. So the application will easily install for all the windows operating system users. The application-developed size is very low. The application consumes very low space in disk. Therefore, the user can allocate very minimum local disk space for this application.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
4. About Me
● Former Director of Cloud Architecture, CBS Interactive in San Francisco
● Ran Operations for TV.com, Metacritic.com, and Clicker.com
● Worked with AWS since 2006 / Private Invite-only Beta
● Advise numerous successful venture backed startups
● Backend Software Developer, Open Source Advocate / Contributor
● Took ~2 years off to travel; visted ~30 countries
5. This Talk
● ~90 Minutes
● Q&A at the end
● Write question in the chat
● Actionable, practical advice
● Collection of our “Best Practices”
6. Best Practices
(my) definition: An opinionated & proven strategy with specific tactics to help
achieve the objectives for some overarching goal.
10. Realize we’re different.
Managers vs Makers - We’re work differently
(Paul Graham - YCombinator Founder)
Makers plan in half-day blocks of time
Managers plan to minimize empty 15 minute slots in their calendar
Interrupts are costly for developers and therefore the business
11. HumanOps (i.e. not cyborgs)
Humans get tired and stressed, they feel happy and sad.
Human issues are system issues.
Human health impacts business health.
Humans need to switch off and on again (aka sleep).
Humans build and fix systems.
Humans > systems
http://www.humanops.com/
12. Right Tools for the Job
Email == external communication
(not tasks, threaded conversations, cat pics)
Slack == all internal communications; channels for topics #dogs
Quip == all documentation for transparency
(engineering & business)
Zoom == reliable cross-platform conferencing
Asana == issue tracking
13. Technical Debt is Real
Tradeoffs are inevitable. Pay the tax now or later.
Later usually means bankruptcy & software rewrites
Includes upgrades, refactoring, optimizations, etc
It’s anything that doesn’t move the product forward
But it will hold the product back
This is not just a software problem.
It’s a business problem too.
...and unavoidable
15. Software Development
Cloud Native Design - the “12 Factor” Pattern
Stable Code Requires Feature Branching / Pull Requests / Code Reviews
Versioning / Version Pinning
Logging
Local Development Environments
16. Some Bad Practices
Cowboy Coding, committing to master
Hardcoding secrets, hostnames, paths, etc
“Clever” code is often “complicated” code
Writing un-greppable code, terse variable names,
Inconsistent naming conventions, long functions, and………… you get the point.
Using tabs :P
17. Some Good Ones….
Strict Linting (e.g. eslint, go lint)
Semantic Versioning (semver)
.editorconfig (tabs or spaces? http://editorconfig.org/)
Seed project repositories
CHANGELOG.md
18. Best Practice: Open Source Pattern*
Leads to much cleaner code with fewer proprietary dependencies
Fewer proprietary dependencies makes it more reusable across projects
If decide to release, it demonstrates the kind of engineering you do
It works because developer’s ego is on the line to write stuff that doesn’t suck
Pro tip: follow the conventions of your favorite framework or package system
* Does not require that organization releases code as open source
19. Best Practice:
README.md & CHANGELOG.md
Use well-formed Markdown syntax (.md)
Write “README” files on all your projects. Explain the purpose of the project
Show how to get started and where to look for more information
Document breaking changes & upgrade path in CHANGELOG.md
Pro tip: Use a markdown editor if you’re not familiar with the syntax
20. Best Practice: Use Makefiles
Provide targets for common usage
E.g. deps, build, run, clean
Include them with all repos
Document targets purpose (##)
21. Makefile Example
-include .secrets
DB_HOST ?= localhost
## build a docker image
build:
docker build -t cloudposse/test .
## run container
run:
docker run -v $$(pwd):/app
-e DB_HOST=$(DB_HOST)
-e DB_PASS=$(DB_PASS)
-p 8080:80
cloudposse/test
## test
test:
curl http://localhost:8080/
22. Best Practice: Local Dev Environments
Onboarding new hires should take minutes not hours
Use fully automated local dev environments
Use same Docker images that will run in staging/production
Bind-mount local volumes to speed up iterations for “live editing”
Pro Tip: Use docker-compose rather than vagrant which is too heavy
23. Best Practice: Developers write Dockerfiles
Always use alpine:3.5 Base images (be wary of unofficial images)
Declare all ENV in Dockerfile (like function arguments to an OS)
Write as few layers as possible (chain with && )
Version Pin Everything
Use 2-stage build process for thin images (C/C++, Golang)
24. Best Practice: Branch Protection
Essential for security and stability of your codebase
Require PR approval to merge to master
Force branches to be up-to-date
Disallow commits to master
Restrict to squash+merge
26. Best Practice: Pull Requests
Smaller the better; implement exactly 1 feature
Milestones
Use Labels:
Define PULL_REQUEST_TEMPLATE (## what, ## why, ## dependencies)
Use checkboxes for TODOs
….for clean commit histories in master
29. Best Practice: Application Logging
Use JSON structured log events
Libraries will efficiently generate/parse
Human readable, highly consistent
Pro tip: use Sentry to aggregate errors+warnings and log them in issue tracker
31. Best Practice: Pair Programming
Lose: speed (arguably)
Gain: fewer bugs, business continuity, education, team building/camaraderie
When: implementing complicated features, onboarding, and triaging
Pro tip: Use tmate for instant terminal sharing (https://tmate.io/)
33. Best Practice: Bug Blowouts
Set aside 1 day per week to dog food your own app
Prepare test scripts (aka flows) for everyone to follow
Get everyone on board, not just QA.
That means developers, graphic artists, customer support, etc
Monitor logs, submit bugs immediately to issue tracker
34. Best Practice: Synthetic Testing
Continuous Testing of Critical User Paths
Uses Browser to Automate Tests of Production
Ensure User Registrations, Password Resets, Shopping Carts, and Checkout
work 100% of the time
Pro Tip: Checkout Selenium or PhantomJS
36. “12 Factor” in a Nutshell
Use Environment Variables for all configuration
(credentials, ports, tuning parameters, etc)
Use Backing Services for everything durable
Write all services as stateless & disposable
Automate all admin tasks
(the rest is meh)
37. Best Practice: X509 Client Certificates
Use CA to Sign SSL Certificates that perform certain functions
Automatic transport & endpoint security for APIs
Highly scalable - no API requests to validate tokens
Don’t Rely on API tokens which are costly to authenticate and don’t secure the
transport layer
Examples: Kubernetes APIs, etcd
38. CI/CD
Frequency reduces Difficulty. The more you deploy, the easier it gets.
Latency between check-in and production is risky. It’s like HFT.
Faster delivery improves software development practices
Consistency improves confidence
39. Ensure applications support same backend schema for adjacent releases
Use feature flags to enable new features of backend schemas
Best Practice: Safe Schema Migrations
40. Write terse .travis.yaml, circle.yaml, Jenkinsfile
Use the same targets in all projects
Use Makefile to automate build, test
Clone harness repo after git checkout
Example: https://github.com/cloudposse/build-harness
Best Practice: Use a Build Harness
41. Best Practice: Liberal Tagging
Tag all docker images with multiple tags, in addition to release tags
Let $ref = {branch|tag}
Then, tag
$ref
$ref-$build
$git_hash
46. What it actually is...
A cross-disciplinary engineering culture
Infrastructure is Code
Automation over toil
A path towards “Serverless” (but we’re still far away!)
Site Reliability Engineering (“SRE”)
47. Infrastructure as Code
Infrastructure is now 100% API driven
“Best Practices” of Development → Infrastructure
Versioned Infrastructure
Automated Remediations
48. Use Terraform to fully orchestrate environments
(e.g. DNS, instances, volumes, AutoScaling Groups, Load Balancers, Databases)
S3 remote backends to store state for collaboration and backups
Use modules to encapsulate business logic for consistency / manageability
Version pin modules and dependencies to ensure stability
Best Practice: Automated Orchestration
49. Best Practice: Tools as Containers
Only local dependency should be docker and maybe make =)
Distribute all other local development tools or dependencies as containers
(e.g. terraform, aws, kops, helm, etc...)
Easier to standardize on one OS
Example: https://github.com/cloudposse/geodesic/
50. Best Practice: 100% Isolation
Use (1) AWS Account per Stage (E.g. production, staging, dev)
Use (1) VPC per Cluster
Use (1) Dedicated TLD per AWS Account
(e.g. foobar.com, foobar.qa, foobar.org)
Use (1) Single Process Containers for all Apps
51. Best Practice: Identical Environments
Environments should only differ in size, not shape
“Production”, “Staging”, “Dev” are only labels
Run as many parallel environments as we need
Only manual action is initiating build
E.g. other labels: pentest, loadtest, erik
Pro tip: each environment gets it’s own DNS zone (e.g. erik.cloudposse.org)
52. What We Want
Reliable - we want things to be online 100% of the time and when things go
wrong, we want them to auto-heal.
Fast - we want to run a site that can scale horizontally as traffic increases
Easy - we shouldn't need rocket scientists to operate it on a day-to-day basis
Affordable - we want it to be easy and cost effective to maintain in the long run
Maintainable - we want to have a development or staging environment that is
identical to production, so we can efficiently work on new versions of the site
without it affecting production
Secure - we don't want to get hacked
53. Technically, we need this… “Everything”
Horizontal Auto Scaling, Auto Healing, Auto DNS, Auto SSL
Automated deployments and rollbacks, Versioned History
Service Discovery & Load Balancing
Batch Job, Scheduled Job Execution
Storage/Volume Orchestration
...out of the box
54. Best Practice: Use Kubernetes (sometimes)
Ideally suited for microservices architectures, larger engineering teams
“Infrastructure as Code” - write documents that describe you microservices
(Pods ~ VMs, ReplicaSets ~ clusters, Services ~ Load Balancers)
Comes with Everything out-of-the-box
Cons: more complex to get started, difficult to triage issues, requires SME
Pro tip: Use kops to spin up clusters automatically in AWS and GCE
56. Best Practice: Use Elastic Beanstalk
Ideally suited for monolithic architectures
Comes with almost Everything out-of-the-box
Supports instances inside private VPC with root SSH access
Formal process for promoting code to production / automatic rollbacks
Pro tip: Use terraform to spin up beanstalk clusters automatically in AWS
59. Best Practice: Immutable Containers/AMIs
Like “Burning” a copy of your code in an image
Easy to know exactly what is running
Fast to deploy and rollback
Use Docker containers for applications
Use something like CoreOS for underlying host (~dom0)
60. Best Practice: Imperative Infrastructure
“Give me a load balancer, 2 filesystems, 2 GB ram, 4 CPUs, 4 instances”
There’s no guess work about what is output
Compatible with legacy architectures
There’s less magic
61. Monitoring
Application - Synthetic Testing
Infrastructure
Real-User Monitoring (RUM)
SLI
Systems don't have feelings. They only have SLAs.
62. Best Practice: Team Dashboards
Display Service Level Indicators (~ KPIs) relevant for specific teams
Create dashboards for specific services like Kafka and Zookeeper
First place to look when triaging issues
Pro tip: Use Datadog dashboards with namespace filtering on clusters
64. Alerting
Alert Fatigue == Human Fatigue
Dashboards > Alerts > Email
Human health impacts business health.
Budgets
Metrics driven; not log events
Alerts need to be actionable - with links to documentation
67. Escalation & Remediation
Automate as much as possible, escalate to a human as a last resort.
KPI~SLI / SLO / SLA
On-call Engineers
PagerDuty - Manage Calendars and Phone/SMS Escalations
68. Best Practice: #OCE Slack Channel
One channel to reach engineers
Searchable history of events and conversations
Use topic to announce who is on-call
Linked Google Calendar with Relevant Events (E.g. Customer Demo Calendar)
69. Best Practice: Post-Mortems
Kill the shame game. Human issues are system issues.
5 Whys - Root Cause Analysis (“RCA”)
Use Consistent Template (KISS)
Weekly Retrospectives with past OCEs and Stakeholders
Documented in Quip → Instantly Searchable
Pro Tip: Check out how Google does it:
https://landing.google.com/sre/book/chapters/postmortem-culture.html
71. What not to do...
1. Store secrets in git repository
2. Hardcode secrets in configurations
3. Write them in plain-text
4. Manually distributed them
5. Reuse/share keys across users and apps
6. Build homegrown systems to protect secrets
(* unless you’re Netflix, Hashicorp or Google)
...but you already knew that!
72. Best Practice: Beyond Corp Model
Enterprise zero-trust security model used by Google
Shift access controls from the network perimeter to individual devices/users
Allow employees to work more securely from any location
Do not rely on traditional VPNs
73. Best Practice: Identity-Aware Proxy (IAP).
Protect internal services using an IAP
Integrates cleanly with your SSO provide
MFA
Pro tip: Use the Bitly OAuth2 Proxy to add auth layer to any service
74. Best Practice: Bastion Host
Centralized point for accessing systems
Session logs, Slack Login Notifications
Require MFA to authenticate
Disable proxy mode and TCP socket forwarding
Use bastion only for triage, not administration (because that’s scripted!)
Pro Tip: Use Duo Push Notifications + Geofencing
76. Best Practice: SSH Key Management
2 options - Github Public Key API or Signed Certificates
● You can’t protect the private key
● You can add multiple factors (a.k.a. MFA)
● Our Solution
○ Use Github Public Key API to distribute public keys
https://github.com/cloudposse/github-authorized-keys
○ Use Duo for MFA Push Notifications + Geofencing
https://github.com/cloudposse/bastion
Pro tip: Checkout Bless by Netflix
78. Best Practice: SSM Scripted Remediations
Use SSM to execute commands in parallel across machines
(don’t use parallel ssh since that is harder to audit)
Full audit logs of command and output
Use IAM roles to restrict execution
Pro tip: use the aws cli to trigger remediations on the command line
79.
80. Best Practice: Federated Accounts
Reduce the blast radius when things explode
Use one account per environment: dev, staging, production
Use a one account for billing aggregation, IAM federation
Assumed Roles (e.g. read-only, admin, dba)
MFA required to assume roles - to devalue credentials
Pro Tip: Use STS API with MFA to generate short lived AWS credentials
Example: https://github.com/cloudposse/aws-assumed-role
AWS
81. Best Practice: AWS Secrets (Client-side)
Client Side (e.g. Terraform, AWS Cli)
● IAM User Account Access Keys (never shared!)
● Access Keys only permit Assume Role+MFA
● Assumed Roles (limit scope)
● Temporary Sessions Tokens with STS (expire after 1 hour)
● MFA (devalue credentials)
Solution: https://github.com/cloudposse/aws-assumed-role
82. Best Practice: AWS Secrets (Server-side)
Dynamic, Auto Rotating Credentials for Server Applications
Never ever hardcode AWS credentials on EC2 instances
Server Side (e.g. EC2 Instance, Docker Container)
● IAM Instance Profiles with Assumed Roles
● Use Kube2IAM with Kubernetes (kops)
https://github.com/cloudposse/charts/tree/master/incubator/kube2iam-kops
○ Temporary AWS credentials
○ Drop-in Compatiblity with all official AWS client library
83. Best Practice: Bootstrap Secrets
Secrets you need to provision new clusters on AWS...
● Run terraform inside of Container
● Private S3 Configuration Bucket
● Encrypted Bucket Objects
● Mount S3 Bucket inside container (S3FS)
● Use /dev/shm for caching
Geodesic: https://github.com/cloudposse/geodesic
84. Best Practice: Password Managers
Store Organizational Secrets in Password Manager
(webhook urls, master account credentials, shared MFA)
Use Vaults specific to some shared objective (e.g. team)
Require MFA for decryption
Avoid Shared Credentials as much as possible (this is a last resort)
SSO > Shared Passwords
Pro tip: Use 1Password for Teams. Abandon all other password managers.
85. Best Practice: Avoid Password Rules
They don't work
They frustrate average users
Penalize people that use real random password generators
They are often computationally weaker → vulnerable to brute force attacks
https://blog.codinghorror.com/password-rules-are-bullshit/