Sydney based cloud consultancy Cloudten's Richard Tomkinson shows how masterless Puppet can be used in concert with AWS's services including Lambda to automate server builds and manage code deployments
Introduction to Packer and Suitcase: A Packer-based OS Image Build SystemHubSpot Product Team
Introduction to Packer, a tool for building OS images and Suitcase, our framework for building Packer images. Presentation by Tom McLaughlin (@tmclaughbos) from HubSpot engineering.
Amazon Aurora: The New Relational Database Engine from AmazonAmazon Web Services
Amazon Aurora is a MySQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. The service is now in preview. Come to our session for an overview of the service and learn how Aurora delivers up to five times the performance of MySQL yet is priced at a fraction of what you'd pay for a commercial database with similar performance and availability.
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivAmazon Web Services
Today’s modern infrastructure allows product teams to take full advantage of “infrastructure-as-code” and deliver value to their customers faster through a seamless & smart delivery pipeline.This delivery pipeline is built using AWS and 3rd party tools such as CloudFormation, Lambda, Terraform, Jenkins, Beanstalk, CodeDeploy, Ansible, and Docker. In the presentation we will walk you through the best practices of combining all the above into a “smart-delivery-pipeline” for your team. By Oron Adam, Emind CTO
Amazon Elastic MapReduce (Amazon EMR) is a web service that allows you to easily and securely provision and manage your Hadoop clusters. In this talk, we will introduce you to Amazon EMR design patterns, such as using various data stores like Amazon S3, how to take advantage of both transient and active clusters, and how to work with other Amazon EMR architectural patterns. We will dive deep on how to dynamically scale your cluster and address the ways you can fine-tune your cluster. We will discuss bootstrapping Hadoop applications from our partner ecosystem that you can use natively with Amazon EMR. Lastly, we will share best practices on how to keep your Amazon EMR cluster cost-effective.
Using cobbler in a not so small environment 1.77chhorn
- cobbler basics
- why cobbler was chosen at a company
- how enterprise-requirements were met
- surrounding infrastructure (monitoring etc.)
- on community interaction
An overview of our experiments at Industrial Light and Magic to create a fully cloud based pipeline, based on Mesos, Docker and automated with Ansible.
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...Holden Karau
Beyond Shuffling - Tips & Tricks for scaling your Apache Spark programs. This talk walks through a number of common mistakes which can keep our Spark programs from scaling and examines the solutions, as well as general techniques useful for moving from beyond a prof of concept to production. It covers topics like effective RDD re-use, considerations for working with key/value data, and finishes up with a preview of some of the work being done to add code generation to Spark ML.
Leveraging AWS for your business provides a catalyst for security programs as customers inherit a faster pace of security innovation simply by using AWS. This session highlights design and architecture patterns customers can employ to measurably improve the security of their organization. In this session, customers explore design patterns for data security using encryption, strong access controls, and least privilege; for implementing detective security controls, such as logging and monitoring, at scale; and for implementing a defense-in-depth network security architecture.
Using Puppet and Cobbler to Automate Your InfrastructurePhil Windley
Using tools to automate your infrastructure will let you sleep through the night. In addition, you'll have o hire fewer people, scale to meet demand more quickly, and make fewer mistakes.
(CMP407) Lambda as Cron: Scheduling Invocations in AWS LambdaAmazon Web Services
Do you need to run an AWS Lambda function on a schedule, without an event to trigger the invocation? This session shows how to use an Amazon CloudWatch metric and CloudWatch alarms, Amazon SNS, and Lambda so that Lambda triggers itself every minute—no external services required! From here, other Lambda jobs can be scheduled in crontab-like format, giving minute-level resolution to your Lambda scheduled tasks. During the session, we build this functionality up from scratch with a Lambda function, CloudWatch metric and alarms, sample triggers, and tasks.
Amazon Elastic MapReduce is one of the largest Hadoop operators in the world. Since its launch five years ago, AWS customers have launched more than 5.5 million Hadoop clusters.
In this talk, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters and other Amazon EMR architectural patterns. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We also share best practices to keep your Amazon EMR cluster cost efficient.
Speakers:
Ian Meyers, AWS Solutions Architect
Ian McDonald, IT Director, SwiftKey
This presentation elaborates on design decisions and design options when it comes to designing the master data architecture.
The presentation was given at the 16th Americas Conference on Information Systems (AMCIS 2010) in Lima, Peru.
Introduction to Packer and Suitcase: A Packer-based OS Image Build SystemHubSpot Product Team
Introduction to Packer, a tool for building OS images and Suitcase, our framework for building Packer images. Presentation by Tom McLaughlin (@tmclaughbos) from HubSpot engineering.
Amazon Aurora: The New Relational Database Engine from AmazonAmazon Web Services
Amazon Aurora is a MySQL-compatible database engine that combines the speed and availability of high-end commercial databases with the simplicity and cost-effectiveness of open source databases. The service is now in preview. Come to our session for an overview of the service and learn how Aurora delivers up to five times the performance of MySQL yet is priced at a fraction of what you'd pay for a commercial database with similar performance and availability.
Self Service Agile Infrastructure for Product Teams - Pop-up Loft Tel AvivAmazon Web Services
Today’s modern infrastructure allows product teams to take full advantage of “infrastructure-as-code” and deliver value to their customers faster through a seamless & smart delivery pipeline.This delivery pipeline is built using AWS and 3rd party tools such as CloudFormation, Lambda, Terraform, Jenkins, Beanstalk, CodeDeploy, Ansible, and Docker. In the presentation we will walk you through the best practices of combining all the above into a “smart-delivery-pipeline” for your team. By Oron Adam, Emind CTO
Amazon Elastic MapReduce (Amazon EMR) is a web service that allows you to easily and securely provision and manage your Hadoop clusters. In this talk, we will introduce you to Amazon EMR design patterns, such as using various data stores like Amazon S3, how to take advantage of both transient and active clusters, and how to work with other Amazon EMR architectural patterns. We will dive deep on how to dynamically scale your cluster and address the ways you can fine-tune your cluster. We will discuss bootstrapping Hadoop applications from our partner ecosystem that you can use natively with Amazon EMR. Lastly, we will share best practices on how to keep your Amazon EMR cluster cost-effective.
Using cobbler in a not so small environment 1.77chhorn
- cobbler basics
- why cobbler was chosen at a company
- how enterprise-requirements were met
- surrounding infrastructure (monitoring etc.)
- on community interaction
An overview of our experiments at Industrial Light and Magic to create a fully cloud based pipeline, based on Mesos, Docker and automated with Ansible.
Beyond Shuffling - Effective Tips and Tricks for Scaling Spark (Vancouver Sp...Holden Karau
Beyond Shuffling - Tips & Tricks for scaling your Apache Spark programs. This talk walks through a number of common mistakes which can keep our Spark programs from scaling and examines the solutions, as well as general techniques useful for moving from beyond a prof of concept to production. It covers topics like effective RDD re-use, considerations for working with key/value data, and finishes up with a preview of some of the work being done to add code generation to Spark ML.
Leveraging AWS for your business provides a catalyst for security programs as customers inherit a faster pace of security innovation simply by using AWS. This session highlights design and architecture patterns customers can employ to measurably improve the security of their organization. In this session, customers explore design patterns for data security using encryption, strong access controls, and least privilege; for implementing detective security controls, such as logging and monitoring, at scale; and for implementing a defense-in-depth network security architecture.
Using Puppet and Cobbler to Automate Your InfrastructurePhil Windley
Using tools to automate your infrastructure will let you sleep through the night. In addition, you'll have o hire fewer people, scale to meet demand more quickly, and make fewer mistakes.
(CMP407) Lambda as Cron: Scheduling Invocations in AWS LambdaAmazon Web Services
Do you need to run an AWS Lambda function on a schedule, without an event to trigger the invocation? This session shows how to use an Amazon CloudWatch metric and CloudWatch alarms, Amazon SNS, and Lambda so that Lambda triggers itself every minute—no external services required! From here, other Lambda jobs can be scheduled in crontab-like format, giving minute-level resolution to your Lambda scheduled tasks. During the session, we build this functionality up from scratch with a Lambda function, CloudWatch metric and alarms, sample triggers, and tasks.
Amazon Elastic MapReduce is one of the largest Hadoop operators in the world. Since its launch five years ago, AWS customers have launched more than 5.5 million Hadoop clusters.
In this talk, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters and other Amazon EMR architectural patterns. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We also share best practices to keep your Amazon EMR cluster cost efficient.
Speakers:
Ian Meyers, AWS Solutions Architect
Ian McDonald, IT Director, SwiftKey
This presentation elaborates on design decisions and design options when it comes to designing the master data architecture.
The presentation was given at the 16th Americas Conference on Information Systems (AMCIS 2010) in Lima, Peru.
Cloud-Größen wie Google, Twitter und Netflix haben die Kernbausteine ihrer Infrastruktur quelloffen verfügbar gemacht. Das Resultat aus vielen Jahren Cloud-Erfahrung ist nun frei zugänglich, und jeder kann seine eigenen Cloud-nativen Anwendungen entwickeln – Anwendungen, die in der Cloud zuverlässig laufen und fast beliebig skalieren. Die einzelnen Bausteine wachsen zu einem großen Ganzen zusammen, dem Cloud-Native-Stack. In dieser Session stellen wir die wichtigsten Konzepte und aktuellen Schlüsseltechnologien kurz vor. Anschließend implementieren wir einen einfachen Microservice mit .NET Core und Steeltoe OSS und bringen ihn zusammen mit ausgewählten Bausteinen für Service-Discovery und Konfiguration schrittweise auf einem Kubernetes-Cluster zum Laufen. @BASTAcon #BASTA17 @qaware #CloudNativeNerd
https://basta.net/microservices-services/cloud-native-net-microservices-mit-kubernetes/
As the complexity of your AWS environment grows, automating security is a crucial step in protecting your data from malicious attacks and unintentional vulnerabilities. Automation and security practices must work hand-in-hand in order to effectively protect your environment. In this session, you will learn how to leverage AWS tools and best-in-class 3rd party services to automate access control, security configuration, and monitoring in order to improve your overall security posture. Using real-life examples, you will come away with an understanding of how to secure your deployments while minimizing the work it takes to keep them secure – all while simplifying your compliance audit process. Topics include:
· Controlling access to your environment with automation tools
· Maintaining security during high velocity deployment cycles
· Protecting data from malicious attacks with automated security controls
· Monitoring and measuring configuration, access, and policy changes
DevOps, Continuous Integration and Deployment on AWS: Putting Money Back into...Amazon Web Services
Organizations around the globe are leveraging the cloud to accomplish world-changing missions. This session will address how AWS can help organizations put more money toward their mission and scale outreach and operations to achieve more with less. Hear some of AWS’s most advanced customers on how their organizations handle DevOps, continuous integration and deployment. Learn how these practices allow them to rapidly develop, iterate, test and deploy highly-scalable web applications and core operational systems on AWS. The discussion will focus on best practices, lessons learned, and the specific technologies and services they use.
Continuous Integration and Deployment Best Practices on AWS (ARC307) | AWS re...Amazon Web Services
With AWS, companies now have the ability to develop and run their applications with speed and flexibility like never before. Working with an infrastructure that can be 100 percent API driven enables businesses to use lean methodologies and realize these benefits. This in turn leads to greater success for those who make use of these practices. In this session, we talk about some key concepts and design patterns for continuous deployment and continuous integration, two elements of lean development of applications and infrastructures.
Mythical Mysfits: Monolith to Microservices with Docker and Fargate - MAD305 ...Amazon Web Services
Traditional monolithic architectures are hard to scale. As an application's code base grows, it becomes too complex to update and maintain. Introducing new features, languages, frameworks, and technologies becomes hard, limiting innovation and new ideas. Within a microservices architecture, each application component runs as its own service and communicates with other services through a well-defined API. Microservices are built around business capabilities, and each service performs a single function. In this workshop, we show you how to run a simple monolithic application in a Docker container, deploy the same application as microservices, and switch traffic to the microservices without downtime.
How (and why) to roll your own Docker SaaSRyan Crawford
SkyDock is an open source solution for building & hosting Docker images at scale in the cloud. Designed to solve real-world problems faced in traditional CI systems that limit throughput and hinder your ability to deliver software quickly and reliably.
From a Skyscanner Engineering perspective this is a great opportunity to showcase some of the cutting edge work that we are doing with Docker, Ansible and AWS. It also highlights that we are solving problems at scale from a both a technical and organisational perspective.
Oscon 2017: Build your own container-based system with the Moby projectPatrick Chanezon
Build your own container-based system
with the Moby project
Docker Community Edition—an open source product that lets you build, ship, and run containers—is an assembly of modular components built from an upstream open source project called Moby. Moby provides a “Lego set” of dozens of components, the framework for assembling them into specialized container-based systems, and a place for all container enthusiasts to experiment and exchange ideas.
Patrick Chanezon and Mindy Preston explain how you can leverage the Moby project to assemble your own specialized container-based system, whether for IoT, cloud, or bare-metal scenarios. Patrick and Mindy explore Moby’s framework, components, and tooling, focusing on two components: LinuxKit, a toolkit to build container-based Linux subsystems that are secure, lean, and portable, and InfraKit, a toolkit for creating and managing declarative, self-healing infrastructure. Along the way, they demo how to use Moby, LinuxKit, InfraKit, and other components to quickly assemble full-blown container-based systems for several use cases and deploy them on various infrastructures.
Top 3 reasons why you should run your Enterprise workloads on GKESreenivas Makam
This deck covers top 3 reasons why Google Kubernetes engine is best suited to run containerized workloads. The reasons covered are Security, Observability and Maturity.
Docker containers have been making inroads into Windows and Azure world. Docker has now replaced the traditional Azure IaaS & PaaS services, offering superior container versions which are more responsive, cost effective, and agile. In this session for Charlotte Azure User Group, we will take an in-depth look at the intersection of Docker and Azure, and how Docker is empowering next gen Azure services.
Here's the link to CAG meetup for the event - https://www.meetup.com/Charlotte-Microsoft-Azure/events/fpftgmyxjbjb/
Tear It Down, Build It Back Up: Empowering Developers with Amazon CloudFormationJames Andrew Vaughn
As a product grows, and the infrastructure becomes more complex, the Operations team traditionally shoulders the burden of maintaining this infrastructure while deploying code from Software Engineers. Code is sometimes given to Operations with little to no information regarding how it should run or what the criteria for successful deployment is. This is not due to lack of caring, Software Engineers often lack the context themselves to provide production deployment instructions. To Software Engineers, production can be like a walled off city, filled with pathways and rooms not to be explored, guarded by Operations.
This presentation aims to provide a solution to this problem. We will address how the traditional separation of Operations and Software Engineers slows innovation, and redefine their relationship -- blending responsibilities. We will examine the transition of two real teams, an Operations team and Engineering team, from complete isolation, to closer environments through virtual machines, to one cloud environment shared by all and managed with CloudFormation.
Similar to Masterless Puppet Using AWS S3 Buckets and IAM Roles (20)
Opendatabay - Open Data Marketplace.pptxOpendatabay
Opendatabay.com unlocks the power of data for everyone. Open Data Marketplace fosters a collaborative hub for data enthusiasts to explore, share, and contribute to a vast collection of datasets.
First ever open hub for data enthusiasts to collaborate and innovate. A platform to explore, share, and contribute to a vast collection of datasets. Through robust quality control and innovative technologies like blockchain verification, opendatabay ensures the authenticity and reliability of datasets, empowering users to make data-driven decisions with confidence. Leverage cutting-edge AI technologies to enhance the data exploration, analysis, and discovery experience.
From intelligent search and recommendations to automated data productisation and quotation, Opendatabay AI-driven features streamline the data workflow. Finding the data you need shouldn't be a complex. Opendatabay simplifies the data acquisition process with an intuitive interface and robust search tools. Effortlessly explore, discover, and access the data you need, allowing you to focus on extracting valuable insights. Opendatabay breaks new ground with a dedicated, AI-generated, synthetic datasets.
Leverage these privacy-preserving datasets for training and testing AI models without compromising sensitive information. Opendatabay prioritizes transparency by providing detailed metadata, provenance information, and usage guidelines for each dataset, ensuring users have a comprehensive understanding of the data they're working with. By leveraging a powerful combination of distributed ledger technology and rigorous third-party audits Opendatabay ensures the authenticity and reliability of every dataset. Security is at the core of Opendatabay. Marketplace implements stringent security measures, including encryption, access controls, and regular vulnerability assessments, to safeguard your data and protect your privacy.
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Data Centers - Striving Within A Narrow Range - Research Report - MCG - May 2...pchutichetpong
M Capital Group (“MCG”) expects to see demand and the changing evolution of supply, facilitated through institutional investment rotation out of offices and into work from home (“WFH”), while the ever-expanding need for data storage as global internet usage expands, with experts predicting 5.3 billion users by 2023. These market factors will be underpinned by technological changes, such as progressing cloud services and edge sites, allowing the industry to see strong expected annual growth of 13% over the next 4 years.
Whilst competitive headwinds remain, represented through the recent second bankruptcy filing of Sungard, which blames “COVID-19 and other macroeconomic trends including delayed customer spending decisions, insourcing and reductions in IT spending, energy inflation and reduction in demand for certain services”, the industry has seen key adjustments, where MCG believes that engineering cost management and technological innovation will be paramount to success.
MCG reports that the more favorable market conditions expected over the next few years, helped by the winding down of pandemic restrictions and a hybrid working environment will be driving market momentum forward. The continuous injection of capital by alternative investment firms, as well as the growing infrastructural investment from cloud service providers and social media companies, whose revenues are expected to grow over 3.6x larger by value in 2026, will likely help propel center provision and innovation. These factors paint a promising picture for the industry players that offset rising input costs and adapt to new technologies.
According to M Capital Group: “Specifically, the long-term cost-saving opportunities available from the rise of remote managing will likely aid value growth for the industry. Through margin optimization and further availability of capital for reinvestment, strong players will maintain their competitive foothold, while weaker players exit the market to balance supply and demand.”
4. Copyright 2015
Cloudten
Industries
Project
Overview
• Client was a startup with VC backing.
• They wanted to build a mobile social media app:
• Dynamically scalable up to 100,000 hits per second
• Cross site active/active with real time DR
• Multiple test environments to be stood up/down at will
• End to end security with encryption at rest
• Engaged separate mobile and web app developers
5. Copyright 2015
Cloudten
Industries
Multiple
Development
Teams
Company
A: Mobile
development
team
in
Melbourne
Company
B: Web
App
development
team
in
Perth/India
6. Copyright 2015
Cloudten
Industries
Multiple
Development
Teams
• Using
Xamarin
Studio
to
create
Apple
and
Android
front
ends
• Require
access
to
code
base
to
publish
WSDLs
under
web
app
• Require
access
to
app
logs
• Using
a
JDK
to
create
a
J2EE
app
running
in
Tomcat
• Require
access
to
entire
web
app
code
base
• Require
access
to
more
logs
7. Copyright 2015
Cloudten
Industries
What
Did
We
Need
to
Solve
?
• Client didn’t have (or want) any full time IT staff.
• Client did want:
– Hands off builds and app deployments
– Self managed consistent server fleet
– No outage deployments
– No direct access to infrastructure from developers
8. Copyright 2015
Cloudten
Industries
Hosting
in
AWS
• AWS provide on-demand scalable resources
• Facility to implement “Infrastructure as Code”
• Secure and durable object storage for code drops
• Fine grained security controls to create server roles
and limit developer access.
• Additional services to co-ordinate deployments
(Lambda, SNS, SQS)
9. Copyright 2015
Cloudten
Industries
AWS
Autoscaling
AWS can automatically add (and remove) servers to a load
balancer pool based on a given metric ( eg. CPU or number of
connections)
Scaling
trigger
hit Scale
out
to
share
the
load
10. Copyright 2015
Cloudten
Industries
How
does
it
work
?
• Launches and builds identical virtual machines
• Destroys them with reckless abandon.
• Essential to get all logs into a central store
• Any generated content must be shared (i.e. can’t
be stored on a local machine)
• The instance launch can trigger a build process
(e.g. Puppet )
11. Copyright 2015
Cloudten
Industries
Automated
Server
Builds
Start
with
base
Amazon
Linux
Image
Invoke
bootstrap
script
Install
Puppet
RPM
S3
sync
config and
Puppet
manifests
Puppet
Apply
12. Copyright 2015
Cloudten
Industries
Puppet
Build
Tasks
• Linux security patching and kernel hardening
• Define custom package repositories
• AWS tagging using facter
– Querying metadata to set instance specific tags
– Set tags for environment variables to be used later
• Package installation, config and version enforcement
– NGINX, WAF, Tomcat, monit, sumo agents etc
• Configuration management
– functional users, public keys, cron jobs, log rotations, system health checks
16. Copyright 2015
Cloudten
Industries
Event
Based
Triggers
cron
1
2
334 4
5 5
cron
control
script
6
7
Complete
deployment
and
set
a
marker
file
as
a
trigger
Check
for
marker.
If
present
put
message
on
SQS
queue
CS
checks
for
message
on
queue
Get
message
from
queue
S3
sync
of
changes
to
staging
area
Puppet
applies
changes
and
restarts
services
17. Copyright 2015
Cloudten
Industries
How
Does
that
Queue
Work
?
• Lambda creates the queue once it detects trigger file (if it doesn’t exist already)
• Lambda queries the auto-scaling group and creates a message on the SQS
queue for each member. It then deletes trigger file
• Each message has a 30 minute expiry ( deployments usually take <5 minutes)
• SQS queue has multiple consumers ( app servers)
• App server checks for its own message, retrieves then deletes it once deployment
is successful.
• Lambda periodically checks for messages about to expire and the dead letter
queue. Triggers an email alert if there is an issue.
18. Copyright 2015
Cloudten
Industries
Puppet
Deployment
Tasks
Check for changes in
staging content area
Sync content
Check for configuration
file changes in staging
areas
Apply changes
Restart/reload
relevant services
Random sleep
20. Copyright 2015
Cloudten
Industries
AWS
IAM
Users/Groups/Roles
• IAM (Identity & Access Management) allows fine grained
user, group and role definitions
• S3 Bucket policies add a further level of security to restrict
access to resources stored in S3
• Web developer bucket policy allows full access for web dev
group
• Mobile developer bucket policy allows full access for mobile
dev group and read only access for web dev group
24. Copyright 2015
Cloudten
Industries
IAM
Users/Groups/Roles
IAM roles allow dynamically launched virtual servers to securely
access credentials by querying locally accessible metadata at
the special use IP address 169.254.169.254
$ curl http://169.254.169.254/latest/meta-data/iam/security-credentials/s3read-only
{
"Code" : "Success",
"LastUpdated" : "2015-04-26T16:39:16Z",
"Type" : "AWS-HMAC",
"AccessKeyId" : "AKIAIOSFODNN7EXAMPLE",
"SecretAccessKey" : "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY",
"Token" : "token",
"Expiration" : "2015-04-27T22:39:16Z"
}
25. Copyright 2015
Cloudten
Industries
Why
Masterless Puppet
• Less instances to manage ( no IT staff to manage it )
• No access for developers to internal infrastructure
• Not Enterprise Puppet
• No need to manage Puppet certificates ( AWS API calls are all
encrypted and IAM enforces authentication)
26. Copyright 2015
Cloudten
Industries
What
Issues
Did
We
Have
?
• Enforced versions being removed from repos
• Auto-scaling and healthcheck tuning
• S3 sync not handling zero byte files properly
• Event trigger mechanism needed tweeking
• Developers storing files on local instances
• S3FS. Don’t use it ! EVER !
27. Copyright 2015
Cloudten
Industries
Who
Are
Cloudten ?
• Advanced AWS Consulting Partner
• Specialise in the design, delivery and support of
cloud based infrastructure projects
• Focus on cloud security and hybrid integration
• We are a Puppet shop !
