Scaling on Amazon AWS : From the perspective of AWS, and the application stack. Talks about the available options on AWS, and also the architecture of the scalable application.
We are now witnessing a new wave of IT revolution and its effect is very similar to the Cloud and Virtualization revolutions that started in the last decade. This new wave, called Containerization, is related to technologies such as Docker and Kubernetes, which now fuel large scale solutions including Big Data and IoT.
Learn about:
- Typical DevOps challenges and modern solutions
- Using Docker as Amazon EC2 Container Service Evolution of Enterprise Architecture (Containers, IoT, Machine Learning and technologies of tomorrow)
- Business value of using advances DevOps technologies with real-life case study
Blue Chip Tek AWS Connect and Protect Presentation #2Kimberly Macias
The document discusses account, VPC and subnet design for AWS infrastructure. It recommends starting with 1 AWS account, 1 VPC, and between 9-16 subnets, with 1 subnet per availability zone and 1 subnet group per connectivity requirement. This simple design focuses on security, flexibility and ease of administration while allowing for complexity to be added later if needed.
This document discusses deploying microservices on AWS. It begins by explaining what microservices are and then discusses hosting options on AWS including EC2, ECS, and Lambda. ECS is identified as the preferred option since it allows hosting containers with Docker. The document then covers deployment aspects like using source control with Git for multiple environments, building and testing code, deploying single services or entire clusters, live testing, and monitoring with alerts.
Serverless architectures rely on third-party services and remote procedure calls rather than maintaining servers. Azure Functions is a serverless computing service that allows developers to write code without managing infrastructure. Functions can be triggered by events and connected to other Azure services through bindings. Functions scale automatically based on demand and only charge for execution time and resources used.
Sascha Möllering gave a presentation on deploying applications to the AWS cloud. He began with an overview of AWS services like EC2, S3, RDS and explained how to initially create a simple cloud service with one instance each for a web application and database. He then described how to improve the architecture by separating components, adding redundancy and elasticity using services like ELB, autoscaling and read replicas. Sascha demonstrated deploying a sample application built with JHipster and Docker to AWS Elastic Beanstalk, which handles running the containers and mapping environment variables for the database connection.
Securing Containers From Day One | null Ahmedabad MeetupKumar Ashwin
Kumar Ashwin gives a presentation on securing containers from day one. The presentation covers what containers are, why we need them, the difference between virtual machines and containers, cgroups and namespaces, Docker basics, building optimized Docker images, and best practices for Dockerfile security. Some key points discussed include using minimal base images, ignoring unnecessary files, creating "golden images" as hardened base templates, not running as root, avoiding secrets in Dockerfiles, and using tools like Hadolint and Dockle to scan for issues.
Infrastructure Automation on AWS using a Real-World Customer ExampleAPI Talent
This technical session focuses on a customer use case and how using the AWS Cloud together with automation has enabled them to standardise and automate their systems.
This talk will describe how this is achieved with two tools, Cloud formation and Puppet. Cloud formation is a declarative templating language that enables the deployment of environments in a standardised way. Combined with a configuration management tool like Puppet allows for the automation of ongoing software deployments and maintenance in a low overhead manner. Puppet is a Configuration Management tool that installs and configures software on instances. Taken together a complete system can be built from the ground up.
Scaling on Amazon AWS : From the perspective of AWS, and the application stack. Talks about the available options on AWS, and also the architecture of the scalable application.
We are now witnessing a new wave of IT revolution and its effect is very similar to the Cloud and Virtualization revolutions that started in the last decade. This new wave, called Containerization, is related to technologies such as Docker and Kubernetes, which now fuel large scale solutions including Big Data and IoT.
Learn about:
- Typical DevOps challenges and modern solutions
- Using Docker as Amazon EC2 Container Service Evolution of Enterprise Architecture (Containers, IoT, Machine Learning and technologies of tomorrow)
- Business value of using advances DevOps technologies with real-life case study
Blue Chip Tek AWS Connect and Protect Presentation #2Kimberly Macias
The document discusses account, VPC and subnet design for AWS infrastructure. It recommends starting with 1 AWS account, 1 VPC, and between 9-16 subnets, with 1 subnet per availability zone and 1 subnet group per connectivity requirement. This simple design focuses on security, flexibility and ease of administration while allowing for complexity to be added later if needed.
This document discusses deploying microservices on AWS. It begins by explaining what microservices are and then discusses hosting options on AWS including EC2, ECS, and Lambda. ECS is identified as the preferred option since it allows hosting containers with Docker. The document then covers deployment aspects like using source control with Git for multiple environments, building and testing code, deploying single services or entire clusters, live testing, and monitoring with alerts.
Serverless architectures rely on third-party services and remote procedure calls rather than maintaining servers. Azure Functions is a serverless computing service that allows developers to write code without managing infrastructure. Functions can be triggered by events and connected to other Azure services through bindings. Functions scale automatically based on demand and only charge for execution time and resources used.
Sascha Möllering gave a presentation on deploying applications to the AWS cloud. He began with an overview of AWS services like EC2, S3, RDS and explained how to initially create a simple cloud service with one instance each for a web application and database. He then described how to improve the architecture by separating components, adding redundancy and elasticity using services like ELB, autoscaling and read replicas. Sascha demonstrated deploying a sample application built with JHipster and Docker to AWS Elastic Beanstalk, which handles running the containers and mapping environment variables for the database connection.
Securing Containers From Day One | null Ahmedabad MeetupKumar Ashwin
Kumar Ashwin gives a presentation on securing containers from day one. The presentation covers what containers are, why we need them, the difference between virtual machines and containers, cgroups and namespaces, Docker basics, building optimized Docker images, and best practices for Dockerfile security. Some key points discussed include using minimal base images, ignoring unnecessary files, creating "golden images" as hardened base templates, not running as root, avoiding secrets in Dockerfiles, and using tools like Hadolint and Dockle to scan for issues.
Infrastructure Automation on AWS using a Real-World Customer ExampleAPI Talent
This technical session focuses on a customer use case and how using the AWS Cloud together with automation has enabled them to standardise and automate their systems.
This talk will describe how this is achieved with two tools, Cloud formation and Puppet. Cloud formation is a declarative templating language that enables the deployment of environments in a standardised way. Combined with a configuration management tool like Puppet allows for the automation of ongoing software deployments and maintenance in a low overhead manner. Puppet is a Configuration Management tool that installs and configures software on instances. Taken together a complete system can be built from the ground up.
This document provides an overview of serverless computing using Azure Functions. It discusses the benefits of serverless such as increased server utilization, instant scaling, and reduced time to market. Serverless allows developers to focus on business logic rather than managing servers. Azure Functions is introduced as a way to develop serverless applications using triggers and bindings in languages like C#, Node.js, Python and more. Common serverless patterns are also presented.
This document summarizes an event-driven architecture presentation using Java. It discusses using Apache Kafka/Amazon Kinesis for messaging, Docker for containerization, Vert.x for reactive applications, Apache Camel/AWS Lambda for integration, and Google Protocol Buffers for data serialization. It covers infrastructure components, software frameworks, local and AWS deployment, and integration testing between Kinesis and Kafka. The presentation provides resources for code samples and Docker images discussed.
The document discusses serverless computing and introduces Microsoft Azure Functions as a serverless platform, highlighting how Functions allows developers to write code that runs in response to events using triggers and bindings to integrate with other Azure services, and provides examples of common serverless patterns that can be implemented using Functions.
These slides are from my Ph.D. defense at the University of California, Santa Barbara, discussing how we contribute research tools to forward how science is performed with cloud systems.
What does Serverless mean for DevOps, in practical terms? While Serverless does reduce the need for server-centric DevOps, it poses new challenges in many areas including security, app deployment and cloud resource provisioning, partly due to an explosion of "nanoservices". Based on a current project using AWS, we cover relevant tools, techniques and tips to deliver a smooth serverless experience for development through to production.
Delivered at Bristol DevOps meetup, 27 Jun 2018. To see detailed notes covering extra points not on slides, click the Notes link just below (or download the Powerpoint).
Update: here's the correct link for Gojko Adzic talk on the Backendless slide - https://www.youtube.com/watch?v=w7X4gAQTk2E
Have you heard the news? SQL Server is coming to Linux! What? That can't be right... or can it?
With the release of SQL Server 2017, Microsoft really is bringing SQL Server to the Linux platform. They've also added support for running SQL Server in Docker containers. How does it work? And how well does it work? In this session, we'll answer these questions (and more!), and we'll talk about how this enables some interesting new possibilities for DevOps.
This document discusses Kubernetes and Elixir. It provides an overview of Kubernetes, including what it is, basic concepts like pods and services, and how it can manage containerized workloads. It also compares Kubernetes to Elixir's BEAM capabilities, noting they are orthogonal and can coexist. The document argues Kubernetes can provide benefits to Elixir applications like easier deployments and multi-tenancy, while acknowledging there may be some drawbacks around hot deployments and BEAM clustering within Kubernetes.
AWS Lambda allows running code without provisioning or managing servers. It is a compute service that runs code in response to events and scales automatically. AWS CloudFormation provides templates to deploy and manage AWS resources in a declarative and repeatable way. Templates define resources and parameters and CloudFormation provisions and deploys resources based on the template. Custom resources can also be defined to provision custom infrastructure not supported natively by CloudFormation templates.
This morning I presented the “Managing VMware vSphere 4 with The Virtualization EcoShell” session for an audience of 200+ people at the Dutch VMUG event in Nieuwegein. The total number of attendees is over 600!!! Here’s a copy of my slide deck.
Serverless Application Model - Executing Lambdas LocallyAlex
This document discusses using the Serverless Application Model (SAM) to develop serverless applications locally. It describes how SAM templates allow defining Lambda functions and APIs that can then be run locally using the SAM CLI. This avoids needing an internet connection to test functions and allows using local tools like debuggers. Examples are provided for local development workflows involving APIs, Lambda functions, DynamoDB, and S3 event processing. Key links are also provided to learn more about SAM templates, the SAM CLI, and running DynamoDB locally.
Building and deploying an analytic service on Cloud is a challenge. A bigger challenge is to maintain the service. In a world where users are gravitating towards a model where cluster instances are to provisioned on the fly, in order for these to be used for analytics or other purposes, and then to have these cluster instances shut down when the jobs get done, the relevance of containers and container orchestration is more important than ever. In short Customers are looking for Serverless Spark Clusters. The Intent of this presentation is to share what is Serverless Spark and what are the benefits of running Spark in serverless manner.
This document summarizes serverless design patterns and tools. It begins with a brief history of cloud computing and an introduction to serverless computing. Common serverless use cases like event-driven applications and stream processing are described. Several serverless patterns are then outlined, such as hosting a static website or REST API using AWS Lambda and API Gateway. Finally, the document demonstrates a serverless application and discusses future directions for serverless technologies.
These are the slides from my talk about the AppScale project at the SBonRails meetup. It covers AppScale as well as Google App Engine and the research projects have come out of it, including Neptune, a Ruby DSL focused on computation-heavy workloads.
Kotlin is a language from the tool gurus at JetBrains. In 2016, after about six years of development, Kotlin reached version 1.0. In 2017 it won the hearts of developers and became an officially supported language for Android.
Kotlin, like Java, is for more than creating Android applications. It can replace or enhance Java most places it is used today including on AWS. AWS Lambda functions sometimes called Serverless Computing, is a service which lets us developers build web services without worrying about configuring servers.
In this session, we will create a lambda service on AWS using Kotlin. Along the way, we will learn what a makes Kotlin an excellent replacement for Java and how simple it is to construct an AWS Lambda function.
MJ Berends talk from AWS Chicago SummitAWS Chicago
This document discusses using LocalStack and Moto for local AWS development. LocalStack runs AWS services locally to allow developing and testing code against real services. Moto allows mocking AWS services in Python tests. The document recommends using these tools to explore services, create prototypes and tests, then gradually transitioning to real AWS for production. Local development tools help bridge differences between local and AWS environments.
This document summarizes Lucidchart's migration from EC2 Classic to VPC. Some key points:
- Lucidchart chose to migrate to VPC for improved security, interoperability with other AWS services, and new features like internal ELBs and dynamic security groups.
- Benefits of VPC included enhanced security from private IPs and network ACLs, easier ELB and reserved instance management. Drawbacks included extra cost and maintenance of NAT instances and more complex networking configuration.
- Lucidchart's migration plan moved servers layer by layer starting with web servers, then applications, then databases, with monitoring servers moved last. Managing security groups carefully during the phased migration was important.
- The top
This document provides an overview and introduction to Amazon Web Services (AWS). It discusses AWS' mission to enable businesses and developers to build scalable applications using web services. Statistics are presented showing AWS' dominance in the cloud computing market, with over 1 million active customers. The core AWS services across compute, storage, databases, networking, administration and more are outlined. The document also reviews AWS regions and availability zones, common concerns around cloud adoption, and provides a case study of how one company migrated their infrastructure to AWS to gain performance improvements, automation benefits and reduce costs.
This document provides an overview of serverless computing using Azure Functions. It discusses the benefits of serverless such as increased server utilization, instant scaling, and reduced time to market. Serverless allows developers to focus on business logic rather than managing servers. Azure Functions is introduced as a way to develop serverless applications using triggers and bindings in languages like C#, Node.js, Python and more. Common serverless patterns are also presented.
This document summarizes an event-driven architecture presentation using Java. It discusses using Apache Kafka/Amazon Kinesis for messaging, Docker for containerization, Vert.x for reactive applications, Apache Camel/AWS Lambda for integration, and Google Protocol Buffers for data serialization. It covers infrastructure components, software frameworks, local and AWS deployment, and integration testing between Kinesis and Kafka. The presentation provides resources for code samples and Docker images discussed.
The document discusses serverless computing and introduces Microsoft Azure Functions as a serverless platform, highlighting how Functions allows developers to write code that runs in response to events using triggers and bindings to integrate with other Azure services, and provides examples of common serverless patterns that can be implemented using Functions.
These slides are from my Ph.D. defense at the University of California, Santa Barbara, discussing how we contribute research tools to forward how science is performed with cloud systems.
What does Serverless mean for DevOps, in practical terms? While Serverless does reduce the need for server-centric DevOps, it poses new challenges in many areas including security, app deployment and cloud resource provisioning, partly due to an explosion of "nanoservices". Based on a current project using AWS, we cover relevant tools, techniques and tips to deliver a smooth serverless experience for development through to production.
Delivered at Bristol DevOps meetup, 27 Jun 2018. To see detailed notes covering extra points not on slides, click the Notes link just below (or download the Powerpoint).
Update: here's the correct link for Gojko Adzic talk on the Backendless slide - https://www.youtube.com/watch?v=w7X4gAQTk2E
Have you heard the news? SQL Server is coming to Linux! What? That can't be right... or can it?
With the release of SQL Server 2017, Microsoft really is bringing SQL Server to the Linux platform. They've also added support for running SQL Server in Docker containers. How does it work? And how well does it work? In this session, we'll answer these questions (and more!), and we'll talk about how this enables some interesting new possibilities for DevOps.
This document discusses Kubernetes and Elixir. It provides an overview of Kubernetes, including what it is, basic concepts like pods and services, and how it can manage containerized workloads. It also compares Kubernetes to Elixir's BEAM capabilities, noting they are orthogonal and can coexist. The document argues Kubernetes can provide benefits to Elixir applications like easier deployments and multi-tenancy, while acknowledging there may be some drawbacks around hot deployments and BEAM clustering within Kubernetes.
AWS Lambda allows running code without provisioning or managing servers. It is a compute service that runs code in response to events and scales automatically. AWS CloudFormation provides templates to deploy and manage AWS resources in a declarative and repeatable way. Templates define resources and parameters and CloudFormation provisions and deploys resources based on the template. Custom resources can also be defined to provision custom infrastructure not supported natively by CloudFormation templates.
This morning I presented the “Managing VMware vSphere 4 with The Virtualization EcoShell” session for an audience of 200+ people at the Dutch VMUG event in Nieuwegein. The total number of attendees is over 600!!! Here’s a copy of my slide deck.
Serverless Application Model - Executing Lambdas LocallyAlex
This document discusses using the Serverless Application Model (SAM) to develop serverless applications locally. It describes how SAM templates allow defining Lambda functions and APIs that can then be run locally using the SAM CLI. This avoids needing an internet connection to test functions and allows using local tools like debuggers. Examples are provided for local development workflows involving APIs, Lambda functions, DynamoDB, and S3 event processing. Key links are also provided to learn more about SAM templates, the SAM CLI, and running DynamoDB locally.
Building and deploying an analytic service on Cloud is a challenge. A bigger challenge is to maintain the service. In a world where users are gravitating towards a model where cluster instances are to provisioned on the fly, in order for these to be used for analytics or other purposes, and then to have these cluster instances shut down when the jobs get done, the relevance of containers and container orchestration is more important than ever. In short Customers are looking for Serverless Spark Clusters. The Intent of this presentation is to share what is Serverless Spark and what are the benefits of running Spark in serverless manner.
This document summarizes serverless design patterns and tools. It begins with a brief history of cloud computing and an introduction to serverless computing. Common serverless use cases like event-driven applications and stream processing are described. Several serverless patterns are then outlined, such as hosting a static website or REST API using AWS Lambda and API Gateway. Finally, the document demonstrates a serverless application and discusses future directions for serverless technologies.
These are the slides from my talk about the AppScale project at the SBonRails meetup. It covers AppScale as well as Google App Engine and the research projects have come out of it, including Neptune, a Ruby DSL focused on computation-heavy workloads.
Kotlin is a language from the tool gurus at JetBrains. In 2016, after about six years of development, Kotlin reached version 1.0. In 2017 it won the hearts of developers and became an officially supported language for Android.
Kotlin, like Java, is for more than creating Android applications. It can replace or enhance Java most places it is used today including on AWS. AWS Lambda functions sometimes called Serverless Computing, is a service which lets us developers build web services without worrying about configuring servers.
In this session, we will create a lambda service on AWS using Kotlin. Along the way, we will learn what a makes Kotlin an excellent replacement for Java and how simple it is to construct an AWS Lambda function.
MJ Berends talk from AWS Chicago SummitAWS Chicago
This document discusses using LocalStack and Moto for local AWS development. LocalStack runs AWS services locally to allow developing and testing code against real services. Moto allows mocking AWS services in Python tests. The document recommends using these tools to explore services, create prototypes and tests, then gradually transitioning to real AWS for production. Local development tools help bridge differences between local and AWS environments.
This document summarizes Lucidchart's migration from EC2 Classic to VPC. Some key points:
- Lucidchart chose to migrate to VPC for improved security, interoperability with other AWS services, and new features like internal ELBs and dynamic security groups.
- Benefits of VPC included enhanced security from private IPs and network ACLs, easier ELB and reserved instance management. Drawbacks included extra cost and maintenance of NAT instances and more complex networking configuration.
- Lucidchart's migration plan moved servers layer by layer starting with web servers, then applications, then databases, with monitoring servers moved last. Managing security groups carefully during the phased migration was important.
- The top
This document provides an overview and introduction to Amazon Web Services (AWS). It discusses AWS' mission to enable businesses and developers to build scalable applications using web services. Statistics are presented showing AWS' dominance in the cloud computing market, with over 1 million active customers. The core AWS services across compute, storage, databases, networking, administration and more are outlined. The document also reviews AWS regions and availability zones, common concerns around cloud adoption, and provides a case study of how one company migrated their infrastructure to AWS to gain performance improvements, automation benefits and reduce costs.
Meetup #3: Migrating an Oracle Application from on-premise to AWSAWS Vietnam Community
The document summarizes a case study of migrating an Oracle application from on-premise to AWS. It describes the existing on-premise architecture including hardware, software, network/security configuration, and disaster recovery requirements. It then discusses challenges of meeting the recovery time and point objectives on AWS. The first proposed solution is outlined along with its problems. Finally, the document proposes an improved solution on AWS and estimates it can save 70% on infrastructure costs.
One Hour Translation provides High quality, Fast, 24x7 Professional Human Translation in more than 57 languages. We are able to provide high-quality translations around the clock, any day, thanks to a community of over 10000 professional translators from over 100 different countries. One Hour Translation serve tens of thousands of projects a month thanks to AWS cloud technologies which enables fast and transparent scaling of our service. In this session I will present a case study for IT architecture and deployment on AWS with security monitoring using Newvem's services.
Oren Yagev Co-Founder, CTO at OneHourTranslation.com
AWS re:Invent 2016: Case Study: How Atlassian Uses Amazon EFS with JIRA to Cu...Amazon Web Services
Atlassian used Amazon EFS with JIRA to cut costs and accelerate performance. Testing showed Amazon EFS provided consistent throughput and capacity as usage increased. Atlassian now uses EFS for their production JIRA Data Center deployment, which benefits from EFS's automatic scaling, high availability, and lower costs compared to on-premises shared storage.
AWS Customer Presentation: Coca Cola Turkey migrates SAP ERP to AWS-SAPPHIRE ...Amazon Web Services
In this presentation, learn about Coca Cola Turkey’s business challenges, why they decided to move their SAP ERP to AWS, a project overview, and benefits realized.
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...Amazon Web Services
At Spokeo, we are running a fast, big data, and high-traffic website providing people search services. But at our scale, we started to reach limitations to how fast our conventional web stack could do things and concluded that a Ruby on Rails–only solution simply couldn't keep up. In this session, we cover some of the options we had to solve this problem and why we chose Amazon Elastic File System (Amazon EFS) as a central part of our solution with metrics and benchmarking. Using EFS, we were able to take response times down from 250 ms to below 70 ms. We look into the architecture of the solution and lessons we learned along the way. In the end, we find that faster response times are just the beginning of the benefits that we see.
(BIZ305) Case Study: Migrating Oracle E-Business Suite to AWS | AWS re:Invent...Amazon Web Services
With the maturity and breadth of cloud solutions, more enterprises are moving mission-critical workloads to the cloud. American Commercial Lines (ACL) recently migrated their Oracle ERP to AWS. ERP solutions such as Oracle E-Business Suite require specific knowledge in mapping AWS infrastructure to the specific configurations and needs of running these workloads. In this session, Apps Associates and ACL walk through the considerations for running Oracle E-Business Suite on AWS, including deployment architectures, concurrent processing, load balanced forms and web services, varying database transactional workloads, and performance requirements, as well as security and monitoring aspects. ACL shares their experiences and business drivers in making this transition to AWS.
Migrating Traditional Apps from On-Premises to the Hybrid CloudRackspace
Re-architecting legacy apps for the public cloud is very resource intensive. However, migrating apps to a hosted hybrid cloud that’s composed of bare-metal servers, VMware® virtualization, EMC® storage and public cloud offers cloud-bursting benefits, but with less risk and cost. Check out our presentation and learn the five-step path to hybrid cloud.
Real-Time Bidding (RTB) is a service offered by advertising networks to agencies. The agencies decide on the value of advertising opportunities in real-time and bid accordingly on behalf of their advertising clients. Typically the window of opportunity for bids to be calculated from provided consumer details (e.g. cookies) and then submitted is 100ms.
Introduction to AWS VPC, Guidelines, and Best PracticesGary Silverman
I crafted this presentation for the AWS Chicago Meetup. This deck covers the rationale, building blocks, guidelines, and several best practices for Amazon Web Services Virtual Private Cloud. I classify it as a somewhere between a 101 and 201 level presentation.
If you like the presentation, I would appreciate you clicking the Like button.
The pathway to the cloud has many different options and levers that customers can pull. This webinar walks customers through actual steps from creating a cloud adoption vision to actually building a migration roadmap with actionable guidance. We’ll go through proven migration patterns, methods and tooling that AWS has leveraged successfully with hundreds of Enterprise customers around the globe. Learn what challenges customers face when planning the migrations to cloud, and how they overcome them to minimize risk and accelerate the adoption.
Amazon EC2 to Amazon VPC: A case study (CPN301) | AWS re:Invent 2013Amazon Web Services
In this session, you learn about Amazon Virtual Private Cloud and why you should consider using it for your applications. You also hear from the makers of Lucidchart, an online diagramming tool, which was originally launched in 2008 on the Amazon EC2 Classic platform. As the user base grew, so did their need for a more robust, secure infrastructure. After much debate about other vendors and colocation, Lucidchart chose Amazon VPC. To find out why, check out this session for a comparison of Amazon EC2 Classic against Amazon VPC. Matthew Barlocker, Chief Architect at Lucidchart, discusses their migration plan, pain points, and unexpected issues.
Attendees will learn the best web application security practices used by major US government entities. The presentation will cover network configuration, caching, replication, common web application vulnerabilities, and how making these changes will result in better web site performance and user satisfaction. The five most common types of web application attacks will be explained, along with simple ways to prevent them.
The document introduces cloud computing and cloud services. It defines cloud computing as using computing resources delivered over a network as a service. It then provides information about Radu Vunvulea and his background. The document outlines an agenda to cover cloud computing topics but does not fill in the agenda items. It also includes sections on cloud characteristics, cloud providers, market forecasts, Azure components, execution models, machine sizes, data management, queues, caching, and other Azure services.
Storage Requirements and Options for Running Spark on KubernetesDataWorks Summit
In a world of serverless computing users tend to be frugal when it comes to expenditure on compute, storage and other resources. Paying for the same when they aren’t in use becomes a significant factor. Offering Spark as service on cloud presents very unique challenges. Running Spark on Kubernetes presents a lot of challenges especially around storage and persistence. Spark workloads have very unique requirements of Storage for intermediate data, long time persistence, Share file system and requirements become very tight when it same need to be offered as a service for enterprise to mange GDPR and other compliance like ISO 27001 and HIPAA certifications.
This talk covers challenges involved in providing Serverless Spark Clusters share the specific issues one can encounter when running large Kubernetes clusters in production especially covering the scenarios related to persistence.
This talk will help people using Kubernetes or docker runtime in production and help them understand various storage options available and which is more suitable for running Spark workloads on Kubernetes and what more can be done
This document compares IaaS cloud services from Azure and Amazon. It provides an overview of key virtual machine components, pricing models, networking, load balancing, and cross-premises connectivity options from each provider. While both offer compute, storage, and networking services, the document notes Azure has better developer tools while Amazon provides more configurable performance options like IOPS. It concludes that which cloud is better depends on the specific needs and priorities of each application.
Architecting for AWS Cloud - let's do it right!Misha Hanin
The power of AWS cloud needs to be understood to be harnessed in the most effective manner. This first Winnipeg AWS User Group meetup provides a forum to explore the technology approach delivering successful solutions on AWS.
Azure provides several options for security and identity management:
- Azure Active Directory allows centralized management of user access and single sign-on across Azure, Office 365, and other cloud apps. It can extend on-premises directories to the cloud.
- Multi-factor authentication protects access using additional verification beyond a password. It can leverage on-premises Active Directory when used with Azure Active Directory.
- Encryption options in Azure help secure data at rest and in transit. These include BitLocker, storage encryption, and application-level encryption.
Introducing to serverless computing and AWS lambda - Israel Clouds MeetupBoaz Ziniman
Serverless computing allows you to build and run applications without the need for provisioning or managing servers. With serverless computing, you can build web, mobile, and IoT backends; run stream processing or big data workloads; run chatbots, and more.
This document discusses storage requirements for running Spark workloads on Kubernetes. It recommends using a distributed file system like HDFS or DBFS for distributed storage and emptyDir or NFS for local temp scratch space. Logs can be stored in emptyDir or pushed to object storage. Features that would improve Spark on Kubernetes include image volumes, flexible PV to PVC mappings, encrypted volumes, and clean deletion for compliance. The document provides an overview of Spark, Kubernetes benefits, and typical Spark deployments.
(ENT308) Best Practices for Implementing Hybrid Architecture Solutions | AWS ...Amazon Web Services
This document discusses moving workloads from multiple physical data centers to AWS to improve scalability and the user experience. It recommends a highly available multi-region hybrid solution using AWS services like Direct Connect, VPC, EC2, S3, Glacier, Route 53, ELB, and SNS. The solution would have disaster recovery in a separate region and integrate an on-premises Oracle RAC. It also emphasizes establishing a rigorous security model and centralized monitoring, backup, and incident tracking to meet governance and compliance requirements.
Building a Platform-as-a-Service with Docker and Node.jsKevin Swiber
Docker describes itself as "an open source project to pack, ship and run any application as a lightweight container." Learn how to use Docker to create a simple Platform-as-a-Service for packaging and deploying your Node.js applications! Introducing Borealis.
Microservices and Serverless for Mega Startups - DevOps IL MeetupBoaz Ziniman
1) The document discusses best practices for running microservices at scale, including breaking monolithic architectures into loosely coupled microservices, using the right tools for each job, securing services, focusing on organizational transformation, and automating everything.
2) Five principles for running microservices are outlined: microservices only rely on each other's public APIs, using the right tool for the job, securing services with defense-in-depth, focusing on cross-functional teams for alignment, and automating everything.
3) Examples of event-driven serverless architectures using AWS Lambda and other AWS services are provided.
AWS re:Invent 2016: Advanced Tips for Amazon EC2 Networking and High Availabi...Amazon Web Services
In this session, we focus on designing for high availability, with evaluation criteria for using services and features such as Amazon Route 53, Elastic Load Balancing, Auto Scaling, route tables, network interfaces, device clustering, and the Transit VPC architecture. We also explore how to create highly available networking between regions as well as on-premises.
The document introduces Microsoft's Windows Azure cloud platform. It summarizes that Windows Azure provides an operating system for the cloud that abstracts away hardware and provides services for automated management, scalable computing and storage. It allows developers to build applications and services that can easily scale across large, connected data centers. The talk demonstrates how Windows Azure allows building complex service architectures from simple components like web and worker roles that interact through a durable storage system. It emphasizes that the platform aims to provide a familiar development experience while handling all the complexities of highly scalable cloud services.
My @TriangleDevops talk from 2013-10-17. I covered the work that led us to @NetflixOSS (Acme Air), the work we did on the cloud prize (NetflixOSS on IBM SoftLayer/RightScale) and the @NetflixOSS platform (Karyon, Archaius, Eureka, Ribbon, Asgard, Hystrix, Turbine, Zuul, Servo, Edda, Ice, Denominator, Aminator, Janitor/Conformity/Chaos Monkeys of the Simian Army).
Wakanda is an open source platform that provides benefits of an open environment including freedom, adaptability, interoperability, portability, reusability, and community. It uses open source libraries and has open source and dual licensing. The Wakanda Studio includes tools like a model designer, GUI designer, and debugger. It supports add-ons, external widgets, and web components. The Wakanda backend integrates technologies like HTTP APIs, modules, and supports accessing external databases and technologies.
Write Once and REALLY Run Anywhere | OpenStack Summit HK 2013dotCloud
The document outlines the agenda for the OpenStack Summit in November 2013. The agenda includes sessions on Docker and its ecosystem, using Docker with OpenStack and Rackspace, and a cross-cloud deployment demo. Docker is presented as a solution for developing and deploying applications across multiple environments by encapsulating code and dependencies in portable containers. It can help eliminate inconsistencies between development, testing, and production environments.
This document discusses different cloud platforms for hosting Grails applications. It provides an overview of infrastructure as a service (IaaS) models like Amazon EC2 and shared/dedicated virtual private servers, as well as platform as a service (PaaS) options including Amazon Beanstalk, Google App Engine, Heroku, Cloud Foundry, and Jelastic. A comparison chart evaluates these platforms based on factors such as pricing, control, reliability, and scalability. The document emphasizes that competition and changes in the cloud space are rapid and recommends keeping applications loosely coupled and testing platforms using free trials.
Journey Towards Scaling Your Application to Million UsersAdrian Hornsby
The document discusses the steps and services needed to scale an application from initial development to supporting 10 million users on AWS. It recommends starting with high quality code, version control, CI/CD pipelines, and infrastructure as code from day one. As the application scales to support more users, it recommends adding services like S3 for static assets, Route53 for DNS, CloudFront for caching, auto scaling groups, database read replicas and sharding, asynchronous processing with SQS, event-driven architecture using Lambda, and microservices. The goal is to build operational excellence and architect the application for availability, redundancy, and scale from the start.
Similar to Amazon EC2 to Amazon VPC: A case study (20)
Initially presented at Software Architecture Conference in Boston, MA on 3/18/15.
Distributed systems are complex beasts. Breaking your application into multiple services introduces new types of errors, cascading failures, and CAP theorem limitations. Unfortunately, your uptime and sanity both suffer. This session will focus on various tactics and learnings from Lucid Software's migration to a service oriented architecture.
Initiallly presented at Scala User Group in SLC, UT on 3 Sep 2014.
Anorm is part of the Typesafe Play! framework stack. It has nice features, but is lacking performance. Relate is a new library, inspired by Anorm, whose performance is closer to the underlying JDBC library. This makes it a better candidate for enterprise and time sensitive workloads.
Initially presented at OpenWest 2014 conference.
Graphite and StatsD gather line series data and offer a robust set of APIs to access that data. While the tools are robust, the dashboards are straight from 1992 and alerting off the data is nonexistent. Nark, an opensource project, solves both of these problems. It provides easy to use dashboards and readily available alerts and notifications to users. It has been used in production at Lucid Software for almost a year. Related to Nark are the tools required to make Graphite highly available.
Graphite has an amazing ability to scale horizontally and achieve high availability. What the community is missing is a single source for alerting and dashboards on top of this already amazing product. That's where Nark comes in.
Nark is an open source project developed to provide alerting and dashboards for graphite. Developed by engineers at Lucid Software, Nark provides value to ops and product teams alike. Come learn about the project, how you can use it, and how to contribute.
The problem with frameworks and ORMs is the lackadaisical manner with which they write queries. Their focus is to straddle multiple database engines, not to write queries in the optimal way. The database layer is always the hardest layer to scale. Why would you put additional strain on your database to avoid writing SQL? This session focuses on understanding indices, queries, and optimization in MySQL.
This document provides an overview of building a Scala web application using the Play! framework. It begins with introductions to Scala and the Play! framework. It then covers topics like request handling, views, forms, database integration, the build system, internationalization, and testing. Exercises are provided to demonstrate creating routes, controllers, actions, views, forms, database models and more. The presenter is available to answer questions after the session and provides information on open positions at their company Lucid Software.
Originally presented at the Northeast PHP 2013 conference.
Git is hard to learn, but that's not a good excuse to ignore it. This presentation aims to teach the bare essentials for operating a git repository.
This document summarizes magic methods in PHP that allow for custom object behavior. It discusses methods for stringification (__toString()), object lifecycle (__construct(), __destruct()), property overloading (__get(), __set(), etc.), method overloading (__call(), __callStatic()), serialization (__sleep(), __wakeup()), cloning (__clone(), __set_state()), and object invocation (__invoke()). Examples are provided for each method. Performance benchmarks at the end show that magic methods have negligible overhead compared to normal method calls.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
2. About Me
•
•
•
Chief Architect at Lucid
Software Inc
Bachelors degree from BYU in
Computer Science
I love to
•
•
•
•
•
play board games
go 4-wheeling
wrestle my sons
fly airplanes
Follow me on
nineofclouds.blogspot.com
3. Lucid Software
•
•
•
•
•
•
•
Online Diagram Software
Online Print & Digital Publishing
Large Documents
Real-time Collaboration
All Changes Tracked
Vector Graphics
High Quality Images
4. Tech at Lucid
•
•
•
•
•
•
Google Closure
Javascript
PHP
Sharded MongoDB
Sharded MySQL
NodeJS
•
•
•
•
•
•
SOA
Scala
Play!
Chef
Zabbix, Graphite
AWS
5. Lucid on AWS
•
•
•
•
•
•
•
•
Elastic Compute Cloud
Virtual Private Cloud
Elastic Block Store
Auto Scaling
Elastic Load Balancing
Simple Storage Service
CloudFront
Export/Import
• Relational Database
Service
• Route53
• Simple Notification
Service
• Simple Email Service
• Availability Zones
• Regions
11. Migration Constraints
• EC2 & VPC
communication
• NAT traffic
• Not Shared:
–
–
–
–
–
Security groups
Load balancers
Auto Scale groups
Elastic IPs
EIP Limits
• Shared:
–
–
–
–
–
–
Instance Limit
EBS volumes
Snapshots
Instance Sizes
Zones
Regions
12. Migration Plan
•
•
•
•
Move top layer first
Move one layer at a time
Meticulously manage security groups
Move monitoring/utility servers last
• http://nineofclouds.blogspot.com/search/label/VPC
21. 2. MongoDB Migration
• Election Algorithm
• Intermediate Move to
Public Subnet
• 15 min Downtime
22. 1. NAT Bandwidth
•
•
•
•
NAT was t1.micro
Databases in EC2
Applications in VPC
Not enough
bandwidth through
NAT
• Avoidable
23. Please give us your feedback on this
presentation
CPN301
As a thank you, we will select prize
winners daily for completed surveys!
24. Join the Team!
•
•
•
•
•
•
Building the next generation of
collaborative web applications
VC funded
High growth rate
Profitable
Graduates from Harvard, MIT,
Stanford
Former Google, Amazon,
Microsoft employees
https://www.lucidchart.com/jobs