NASA's EVA group needs to extract text (OCR) from 100,000 pages per month to improve Astronaut safety. This presentation shows how we used AWS Lambda serverless tech to solve the problem quickly and cost-effectively.
AWS Summit 2014 Perth - Breakout 5
Increased agility, elasticity, focus on core business, optimized costs, and better security are all good outcomes when it comes to working with AWS. But, for an enterprise with many existing complex applications, integrated operations, and sophisticated teams, this integration and transition requires thought and planning. Within this session, we will start with a typical enterprise customer and work backwards step by step to show how an extreme IT makeover it possible with reusable examples; demonstrating that an application doesn’t have to be written for cloud to realize cloud’s benefits. Many large-scale shops are already leveraging AWS without sacrificing what they are good at, and we will explain the framework for kicking off this initiative within your own organization.
Presenter: Richard Busby, Solutions Architect, Amazon Web Services
"Is serverless another passing technology fad or the new standard for application deployment in cloud computing?” It’s a good question and the topic of this presentation. We will discuss the current state of serverless computing and the many considerations before investing time and resources in serverless infrastructure.
For many, data center priorities have shifted from absolute uptime and performance to ”move fast and break things” as espoused by Silicon Valley, a great mantra for those with limited legacy systems and a greenfield of new products. Though the question for many enterprises though is "How does serverless integrate into their existing data center strategy?"
The discussion will not only explain the state of today’s growing serverless landscape but how you can integrate your existing data center with a cloud-native serverless architecture.
Reducing Latency and Increasing Performance while Cutting Infrastructure CostsAmazon Web Services
Discussion on Datadog’s experiences, both successes and challenges, as they built our monitoring solutions on top AWS Lambda and Amazon API gateway with the goal of reducing latency and increasing performance while cutting infrastructure costs.
Accenture Cloud Platform helps customers manage public and private enterprise cloud resources effectively and securely. In this session, learn how we designed and built new core platform capabilities using a serverless, microservices-based architecture that is based on AWS services such as AWS Lambda and Amazon API Gateway. During our journey, we discovered a number of key benefits, including a dramatic increase in developer velocity, a reduction (to almost zero) of reliance on other teams, reduced costs, greater resilience, and scalability. We describe the (wild) successes we’ve had and the challenges we’ve overcome to create an AWS serverless architecture at scale. Session sponsored by Accenture.
AWS Competency Partner
(CMP407) Lambda as Cron: Scheduling Invocations in AWS LambdaAmazon Web Services
Do you need to run an AWS Lambda function on a schedule, without an event to trigger the invocation? This session shows how to use an Amazon CloudWatch metric and CloudWatch alarms, Amazon SNS, and Lambda so that Lambda triggers itself every minute—no external services required! From here, other Lambda jobs can be scheduled in crontab-like format, giving minute-level resolution to your Lambda scheduled tasks. During the session, we build this functionality up from scratch with a Lambda function, CloudWatch metric and alarms, sample triggers, and tasks.
4 years ago, mid 2013, we have identified a gap in the cloud echo-system. The landscape of IaaS, PaaS and SaaS provides solutions for VMs, Container and Networking, platforms of different types for backend developers, Backends for mobile developers and ready made software for individuals and enterprises. What is missing in the middle is the platform for web-sites and web-apps.
4 years down the line, with the emergence of Serverless, there are still no players in this gap. We will talk about what makes a platform for web-sites and web-apps. Things frontend optimized javascript, SEO, visual builder, web methods & backend javascript as well as request time container boot.
We have built Wix Code over the last 4 years targeting this exact gap – a serverless platform for website and web applications, and so …
Wix is taking the risk of predicting the future of serverless computing and where it should be 4 years from now.
AWS Summit 2014 Perth - Breakout 5
Increased agility, elasticity, focus on core business, optimized costs, and better security are all good outcomes when it comes to working with AWS. But, for an enterprise with many existing complex applications, integrated operations, and sophisticated teams, this integration and transition requires thought and planning. Within this session, we will start with a typical enterprise customer and work backwards step by step to show how an extreme IT makeover it possible with reusable examples; demonstrating that an application doesn’t have to be written for cloud to realize cloud’s benefits. Many large-scale shops are already leveraging AWS without sacrificing what they are good at, and we will explain the framework for kicking off this initiative within your own organization.
Presenter: Richard Busby, Solutions Architect, Amazon Web Services
"Is serverless another passing technology fad or the new standard for application deployment in cloud computing?” It’s a good question and the topic of this presentation. We will discuss the current state of serverless computing and the many considerations before investing time and resources in serverless infrastructure.
For many, data center priorities have shifted from absolute uptime and performance to ”move fast and break things” as espoused by Silicon Valley, a great mantra for those with limited legacy systems and a greenfield of new products. Though the question for many enterprises though is "How does serverless integrate into their existing data center strategy?"
The discussion will not only explain the state of today’s growing serverless landscape but how you can integrate your existing data center with a cloud-native serverless architecture.
Reducing Latency and Increasing Performance while Cutting Infrastructure CostsAmazon Web Services
Discussion on Datadog’s experiences, both successes and challenges, as they built our monitoring solutions on top AWS Lambda and Amazon API gateway with the goal of reducing latency and increasing performance while cutting infrastructure costs.
Accenture Cloud Platform helps customers manage public and private enterprise cloud resources effectively and securely. In this session, learn how we designed and built new core platform capabilities using a serverless, microservices-based architecture that is based on AWS services such as AWS Lambda and Amazon API Gateway. During our journey, we discovered a number of key benefits, including a dramatic increase in developer velocity, a reduction (to almost zero) of reliance on other teams, reduced costs, greater resilience, and scalability. We describe the (wild) successes we’ve had and the challenges we’ve overcome to create an AWS serverless architecture at scale. Session sponsored by Accenture.
AWS Competency Partner
(CMP407) Lambda as Cron: Scheduling Invocations in AWS LambdaAmazon Web Services
Do you need to run an AWS Lambda function on a schedule, without an event to trigger the invocation? This session shows how to use an Amazon CloudWatch metric and CloudWatch alarms, Amazon SNS, and Lambda so that Lambda triggers itself every minute—no external services required! From here, other Lambda jobs can be scheduled in crontab-like format, giving minute-level resolution to your Lambda scheduled tasks. During the session, we build this functionality up from scratch with a Lambda function, CloudWatch metric and alarms, sample triggers, and tasks.
4 years ago, mid 2013, we have identified a gap in the cloud echo-system. The landscape of IaaS, PaaS and SaaS provides solutions for VMs, Container and Networking, platforms of different types for backend developers, Backends for mobile developers and ready made software for individuals and enterprises. What is missing in the middle is the platform for web-sites and web-apps.
4 years down the line, with the emergence of Serverless, there are still no players in this gap. We will talk about what makes a platform for web-sites and web-apps. Things frontend optimized javascript, SEO, visual builder, web methods & backend javascript as well as request time container boot.
We have built Wix Code over the last 4 years targeting this exact gap – a serverless platform for website and web applications, and so …
Wix is taking the risk of predicting the future of serverless computing and where it should be 4 years from now.
https://datalake.aws.job4u.io/en/
You are a Data Analytics Team at Unicorn-Taxi Startup. You’ve been asked to help with the following tasks:
* Create a Dataset for Reporting and Visualization: Cleanse, Transform, Optimize for Reporting Queries
* Help solve a Machine Learning problem: Unicorn-Taxi’s Data Scientists need to understand Passenger tipping behavior
Elastically scalable architectures with microservices. The end of the monolith?Javier Arias Losada
In the last years the microservices architecture style has been gaining traction with some companies such as Netflix, Yelp, Gilt, PayPal. Many of that companies abandoned their previous monolithic architecture and moved to a microservices approach.
Does that mean that monolithic architectures are a thing of the past?
In this talk we will review some key microservices concepts (and misconceptions), search for the essence of microservices architectures and discuss about different approaches to implement them from the industry.
Getting Started with Serverless Architectures | AWS Public Sector Summit 2016Amazon Web Services
By building your application with AWS Lambda, Amazon API Gateway, and Amazon DynamoDB, you can free yourself from the burden of managing servers while gaining agility and simple scaling. After introducing the basics of building microservices with AWS Lambda and Amazon API Gateway, the session highlights how the Democratic National Committee (DNC) Technology Team uses AWS Lambda and Amazon DynamoDB microservices to provide campaigns and state parties customized applications on top of a core data platform. This serverless architecture has helped the DNC Technology Team improve their microservice functionality and development process, ensuring their applications are performant through the extremely erratic usage levels of a campaign cycle.
AWS re:Invent 2016: Building Complex Serverless Applications (GPST404)Amazon Web Services
Provisioning, scaling, and managing physical or virtual servers—and the applications that run on them—has long been a core activity for developers and system administrators. The expanding array of managed AWS cloud services, including AWS Lambda, Amazon DynamoDB, Amazon API Gateway and more, increasingly allows organizations to focus on delivering business value without worrying about managing the underlying infrastructure or paying for idle servers and other fixed costs of cloud services. In this session, we discuss the design, development, and operation of these next-generation solutions on AWS. Whether you're developing end-user web applications or back-end data processing systems, join us in this session to learn more about building your applications without servers.
AWS re:Invent 2016: Automating and Scaling Infrastructure Administration with...Amazon Web Services
In this session, we’ll show how customers can use management tools to standardize the creation of AWS resources and then govern these resources through the lifecycle. By using AWS CloudFormation and AWS Service Catalog to provision resources at scale, AWS Config to audit any changes to the configuration of these resources, Amazon CloudWatch to monitor the health of these resources, and AWS CloudTrail to audit who or what made API calls to these resources, customers can automate and scale the administration of their infrastructure on AWS. They can even go one step further and automate compliance checking and remediation by using AWS Config rules and Amazon CloudWatch Events. We will demo how this is possible by looking at some common use cases.
AWS re:Invent 2016: Running Batch Jobs on Amazon ECS (CON310)Amazon Web Services
Batch computing is a common way for developers, scientists and engineers to run a series of jobs on a large pool of shared compute resources, such as servers, virtual machines, and containers. Amazon ECS makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. In this session will show you how to run batch jobs using Amazon ECS and together with other AWS services, such as AWS Lambda and Amazon SQS. We will see how you can leverage Amazon EC2 Spot Instances to power your ECS cluster and easily scale your batch workloads. You'll hear from Mapbox on how they use ECS to power their entire batch processing architecture to collect and process over 100 million miles of sensor data per day that they use for powering their maps. Mapbox will also discuss how they optimize their batch processing framework on ECS using Spot Instances and demo their open source framework that will help you get up and running with ECS in minutes.
AWS re:Invent 2016: How Thermo Fisher Is Reducing Mass Spectrometry Experimen...Amazon Web Services
Mass spectrometry is the gold standard for determining chemical compositions, with spectrometers often measuring the mass of a compound down to a single electron. This level of granularity produces an enormous amount of hierarchical data that doesn't fit well into rows and columns. In this talk, learn how Thermo Fisher is using MongoDB Atlas on AWS to allow their users to get near real-time insights from mass spectrometry experiments—a process that used to take days. We also share how the underlying database service used by Thermo Fisher was built on AWS.
AWS vs. Azure vs. Google vs. SoftLayer: Network, Storage and DBaaSRightScale
Most enterprises have a multi-cloud strategy, but choosing the right cloud for a workload can be challenging. In a previous deck we covered differences in block/object storage, pricing, and container services. In this deck we’ll drill down on archival storage, database-as-a-service (DBaaS), and networking options for the leading public clouds.
Moving Enterprise Windows Workloads to AWS – Peter StanskiAmazon Web Services
In this session you will learn how to create an entire Microsoft Enterprise environment in AWS that includes AWS Active Directory Service, Simple System Management (SSM) service, MS Exchange and SharePoint. These will further integrate with new end user productivity services such as AWS WorkSpaces, AWS WorkDocs, and AWS WorkMail.
AWS re:Invent 2016: Application Lifecycle Management in a Serverless World (S...Amazon Web Services
Amazon API Gateway and AWS Lambda provide a new way of building applications by removing servers from the picture. But what does the removal of servers mean to tasks like deployment, monitoring, and debugging? How should you set up blue-green deployments or set alarms? Come learn all this and more, including how to use AWS services and tools like AWS CodePipeline, AWS CloudFormation, and Amazon CloudWatch to manage your serverless applications at high quality.
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)Amazon Web Services
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as the ECS CLI and Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; Amazon EC2 Container Registry to store your container images; and Amazon EC2 Container Service to manage and scale containers. In this session, you will learn how to build containers into your development workflow and orchestrate container deployments using Amazon ECS. You will hear how Okta runs 30,000 tests per developer commit and releases 10,000 new lines of code each week to production with a CI system based on 100% AWS services. We'll also discuss how Okta uses ECS for parallelized testing in CI and for production microservices in a multi-region, always on cloud service.
Deploying Serverless Cloud Optical Character Recognition in Support of NASA A...Chris Shenton
Presentation to the NASA Cloud Community of Interest on how we leveraged AWS Lambda in the GovCloud to do high volume OCR of design documents to improve astronaut safety.
https://datalake.aws.job4u.io/en/
You are a Data Analytics Team at Unicorn-Taxi Startup. You’ve been asked to help with the following tasks:
* Create a Dataset for Reporting and Visualization: Cleanse, Transform, Optimize for Reporting Queries
* Help solve a Machine Learning problem: Unicorn-Taxi’s Data Scientists need to understand Passenger tipping behavior
Elastically scalable architectures with microservices. The end of the monolith?Javier Arias Losada
In the last years the microservices architecture style has been gaining traction with some companies such as Netflix, Yelp, Gilt, PayPal. Many of that companies abandoned their previous monolithic architecture and moved to a microservices approach.
Does that mean that monolithic architectures are a thing of the past?
In this talk we will review some key microservices concepts (and misconceptions), search for the essence of microservices architectures and discuss about different approaches to implement them from the industry.
Getting Started with Serverless Architectures | AWS Public Sector Summit 2016Amazon Web Services
By building your application with AWS Lambda, Amazon API Gateway, and Amazon DynamoDB, you can free yourself from the burden of managing servers while gaining agility and simple scaling. After introducing the basics of building microservices with AWS Lambda and Amazon API Gateway, the session highlights how the Democratic National Committee (DNC) Technology Team uses AWS Lambda and Amazon DynamoDB microservices to provide campaigns and state parties customized applications on top of a core data platform. This serverless architecture has helped the DNC Technology Team improve their microservice functionality and development process, ensuring their applications are performant through the extremely erratic usage levels of a campaign cycle.
AWS re:Invent 2016: Building Complex Serverless Applications (GPST404)Amazon Web Services
Provisioning, scaling, and managing physical or virtual servers—and the applications that run on them—has long been a core activity for developers and system administrators. The expanding array of managed AWS cloud services, including AWS Lambda, Amazon DynamoDB, Amazon API Gateway and more, increasingly allows organizations to focus on delivering business value without worrying about managing the underlying infrastructure or paying for idle servers and other fixed costs of cloud services. In this session, we discuss the design, development, and operation of these next-generation solutions on AWS. Whether you're developing end-user web applications or back-end data processing systems, join us in this session to learn more about building your applications without servers.
AWS re:Invent 2016: Automating and Scaling Infrastructure Administration with...Amazon Web Services
In this session, we’ll show how customers can use management tools to standardize the creation of AWS resources and then govern these resources through the lifecycle. By using AWS CloudFormation and AWS Service Catalog to provision resources at scale, AWS Config to audit any changes to the configuration of these resources, Amazon CloudWatch to monitor the health of these resources, and AWS CloudTrail to audit who or what made API calls to these resources, customers can automate and scale the administration of their infrastructure on AWS. They can even go one step further and automate compliance checking and remediation by using AWS Config rules and Amazon CloudWatch Events. We will demo how this is possible by looking at some common use cases.
AWS re:Invent 2016: Running Batch Jobs on Amazon ECS (CON310)Amazon Web Services
Batch computing is a common way for developers, scientists and engineers to run a series of jobs on a large pool of shared compute resources, such as servers, virtual machines, and containers. Amazon ECS makes it easy to run and manage Docker-enabled applications across a cluster of Amazon EC2 instances. In this session will show you how to run batch jobs using Amazon ECS and together with other AWS services, such as AWS Lambda and Amazon SQS. We will see how you can leverage Amazon EC2 Spot Instances to power your ECS cluster and easily scale your batch workloads. You'll hear from Mapbox on how they use ECS to power their entire batch processing architecture to collect and process over 100 million miles of sensor data per day that they use for powering their maps. Mapbox will also discuss how they optimize their batch processing framework on ECS using Spot Instances and demo their open source framework that will help you get up and running with ECS in minutes.
AWS re:Invent 2016: How Thermo Fisher Is Reducing Mass Spectrometry Experimen...Amazon Web Services
Mass spectrometry is the gold standard for determining chemical compositions, with spectrometers often measuring the mass of a compound down to a single electron. This level of granularity produces an enormous amount of hierarchical data that doesn't fit well into rows and columns. In this talk, learn how Thermo Fisher is using MongoDB Atlas on AWS to allow their users to get near real-time insights from mass spectrometry experiments—a process that used to take days. We also share how the underlying database service used by Thermo Fisher was built on AWS.
AWS vs. Azure vs. Google vs. SoftLayer: Network, Storage and DBaaSRightScale
Most enterprises have a multi-cloud strategy, but choosing the right cloud for a workload can be challenging. In a previous deck we covered differences in block/object storage, pricing, and container services. In this deck we’ll drill down on archival storage, database-as-a-service (DBaaS), and networking options for the leading public clouds.
Moving Enterprise Windows Workloads to AWS – Peter StanskiAmazon Web Services
In this session you will learn how to create an entire Microsoft Enterprise environment in AWS that includes AWS Active Directory Service, Simple System Management (SSM) service, MS Exchange and SharePoint. These will further integrate with new end user productivity services such as AWS WorkSpaces, AWS WorkDocs, and AWS WorkMail.
AWS re:Invent 2016: Application Lifecycle Management in a Serverless World (S...Amazon Web Services
Amazon API Gateway and AWS Lambda provide a new way of building applications by removing servers from the picture. But what does the removal of servers mean to tasks like deployment, monitoring, and debugging? How should you set up blue-green deployments or set alarms? Come learn all this and more, including how to use AWS services and tools like AWS CodePipeline, AWS CloudFormation, and Amazon CloudWatch to manage your serverless applications at high quality.
AWS re:Invent 2016: Development Workflow with Docker and Amazon ECS (CON302)Amazon Web Services
Keeping consistent environments across your development, test, and production systems can be a complex task. Docker containers offer a way to develop and test your application in the same environment in which it runs in production. You can use tools such as the ECS CLI and Docker Compose for local testing of applications; Jenkins and AWS CodePipeline for building and workflow orchestration; Amazon EC2 Container Registry to store your container images; and Amazon EC2 Container Service to manage and scale containers. In this session, you will learn how to build containers into your development workflow and orchestrate container deployments using Amazon ECS. You will hear how Okta runs 30,000 tests per developer commit and releases 10,000 new lines of code each week to production with a CI system based on 100% AWS services. We'll also discuss how Okta uses ECS for parallelized testing in CI and for production microservices in a multi-region, always on cloud service.
Deploying Serverless Cloud Optical Character Recognition in Support of NASA A...Chris Shenton
Presentation to the NASA Cloud Community of Interest on how we leveraged AWS Lambda in the GovCloud to do high volume OCR of design documents to improve astronaut safety.
Ever wished you had a list of cheat codes to unleash the full power of AWS Lambda for your production workload? Come learn how to build a robust, scalable, and highly available serverless application using AWS Lambda. In this session, we discuss hacks and tricks for maximizing your AWS Lambda performance, such as leveraging customer reuse, using the 500 MB scratch space and local cache, creating custom metrics for managing operations, aligning upstream and downstream services to scale along with Lambda, and many other workarounds and optimizations across your entire function lifecycle.
You also learn how Hearst converted its real-time clickstream analytics data pipeline from a server-based model to a serverless one. The infrastructure of the data pipeline relied on Amazon EC2 instances and cron jobs to shepherd data through the process. In 2016, Hearst converted its data pipeline architecture to a serverless process that relies on event triggers and the power of AWS Lambda. By moving from a time-based process to a trigger-based process, Hearst improved its pipeline latency times by 50%.
Skillenza Build with Serverless Challenge - Advanced Serverless ConceptsDhaval Nagar
Skillenza is back with another game-changing virtual hackathon for you. Seize this amazing opportunity to create projects on serverless architecture. For those of you who are not acquainted with it, serverless architectures are system designs that use third-party services to build and run applications.
As developers, this helps you to gain better scalability and flexibility without needing any administration to manage infrastructure. So you can build quicker and at a reduced cost as well.
https://skillenza.com/challenge/build-with-serverless-online-hackathon-aws
Compute Without Servers – Building Applications with AWS Lambda - Technical 301Amazon Web Services
AWS Lambda enables developers to build scalable applications without managing servers. Come learn how Lambda's event driven approach helps build backend ingestion systems, real time stream processing, and scalable API backends. We will deep dive into the different approaches that customers have taken to building applications with Lambda, typical architectures that customers use Lambda for, and best practices for authoring, deploying, and managing Lambda functions.
Speaker: Ajay Nair, Sr Product Manager Lambda, Amazon Web Services
Lambda and serverless - DevOps North East Jan 2017Mike Shutlar
Introduction to AWS Lambda, serverless architectures, & the new AWS Serverless Application Model.
Source code for demo serverless application available here:
https://github.com/infectedsoundsystem/lambda-refarch-webapp
With AWS Lambda, you can easily build scalable microservices for mobile, web, and IoT applications or respond to events from other AWS services without managing infrastructure. In this session, you’ll see demonstrations and hear more about newly launched features. We’ll show you how to use Lambda to build web, mobile, or IoT backends and voice-enabled apps, and we'll show you how to extend both AWS and third party services by triggering Lambda functions. We’ll also provide productivity and performance tips for getting the most out of your Lambda functions and show how cloud native architectures use Lambda to eliminate “cold servers” and excess capacity without sacrificing scalability or responsiveness.
🌥️ “Cloud 101” is an event organized by our club's ☁️ Cloud lead to introduce students to the world of cloud computing. The event aims to equip students with the 🔧 skills and 💡 knowledge needed to get started with cloud computing.
👨💼 Host: The event will be hosted by the ☁️ Cloud lead of our club, who has an amazing experience in cloud computing.
🎯 Aim: The event aims to provide an introduction to cloud computing for students who are new to the field.
📚 Topics: The event will cover a range of topics related to cloud computing, such as ☁️ cloud architecture, 🔒 cloud security, ☁️ cloud services, ☁️ cloud deployment, and more.
👥 Activities: In addition to talks and workshops, the event will also feature hands-on activities and interactive sessions, designed to help students get a first-hand experience of working with cloud computing tools and technologies.
🤝 Networking: The event will provide ample opportunities for networking and connecting with like-minded individuals who share a passion for cloud computing.
📖 Prerequisites: No prior knowledge or experience in cloud computing is required to attend the event. The event is open to all students who are curious about the field and willing to learn.
📝 Registration: The event is free of cost and open to all students. However, pre-registration is mandatory to attend the event, as seats are limited.
So, if you want to get started with cloud computing and learn from an experienced ☁️ Cloud lead, join us at Cloud 101 – Your Introduction to Cloud Computing! 🚀
With AWS Lambda, you can easily build scalable microservices for mobile, web, and IoT applications or respond to events from other AWS services without managing infrastructure. In this session, you’ll see demonstrations and hear more about newly launched features. We’ll show you how to use Lambda to build web, mobile, or IoT backends and voice-enabled apps, and we'll show you how to extend both AWS and third party services by triggering Lambda functions. We’ll also provide productivity and performance tips for getting the most out of your Lambda functions and show how cloud native architectures use Lambda to eliminate “cold servers” and excess capacity without sacrificing scalability or responsiveness.
AWS Lambda is a compute service that runs your code in response to events and automatically manages the compute resources for you, making it easy to build applications that respond quickly to new information. AWS Lambda starts running your code within milliseconds of an event such as an image upload, in-app activity, website click, or output from a connected device.
The “Twelve-Factor” application model has come to represent twelve best practices for building modern, cloud-native applications. With guidance on things like configuration, deployment, runtime, and multiple service communication, the Twelve-Factor model prescribes best practices that apply to everything from web applications to APIs to data processing applications.
Although serverless computing and AWS Lambda have changed how application development is done, the “Twelve-Factor” best practices remain relevant and applicable in a serverless world. In this talk, Chris will share with you how to apply the “Twelve-Factor” model to serverless application development with AWS Lambda and Amazon API Gateway and show you how these services enable you to build scalable, low cost, and low administration applications.
Write less (code) and build more with serverlessDhaval Nagar
Presented the Serverless Use case at the Charotar University of Science and Technology.
CHARUSAT has a very active AWS Student Community.
https://aws.amazon.com/blogs/training-and-certification/aws-academys-big-bet-on-higher-education/
Similar to Serverless OCR for NASA EVA: AWS Meetup DC 2017-12-12 (20)
Orchestrating complex workflows with aws step functionsChris Shenton
We've been doing a lot of work with Lambda, and StepFunctions has given us a way to externalize workflows, allowing decoupling and much faster evolution of workflow and implementation logic. I'll talk about a couple applications we're using it for, and cover Dynamic Parallelism (introduced late last year), which has allowed us to rip out a lot of hairy tracking code. We love it, and think it'll help you in your work too.
Automating EVA Workflows with AWS Step FunctionsChris Shenton
NASA EVA Office (space walks, suits) has a lot of complex workflow. AWS Step Functions could free them from having to develop their own logic and engine. Demonstrate how to convert existing workflow diagram into Step Functions YAML, run it in AWS, and add one Lambda to implement a step.
Creating Serverless apps for NASA in GovCloudChris Shenton
Presentation given at AWS Public Sector Summit 2019 as the first "DevChat" session: fast, scaling with Lambda, architectural evolution, security to get ATO, benefits
Squeezing Machine Learning into Serverless for Image Recognition - AWS Meetup...Chris Shenton
Lightning talk describing how we use a Makefile and Docker to easily build small versions of machine learning tools that will fit in a Lambda, then how we use the Serverless Framework to painlessly deploy the function with its binaries/libraries, along with an S3 bucket, permissions and trigger.
Scaffolding for Serverless: lightning talk for AWS Arlington MeetupChris Shenton
Tools around a sample serverless app to automate the drudgery associated with a project. Testing with pytest and coverage; code smells with flake8; documentation with Sphinx for HTML/ePub output; different deployment environments based on code repo branch (dev, qa, prod, and per-developer); CI/CD pipeline to run all this, including deployment to separate AWS environments. And a simple app with API Gateway, Lambda, S3, DynamoDB.
Second Skin: Real-Time Retheming a Legacy Web Application with Diazo in the C...Chris Shenton
Presentation for PloneConf2017 in Barcelona. Backend tools used to develop and deploy the Diazo theme engine, for front- and back-end developers. Toolchain including build, tests, continuous integration, and deployment to a high-availability AWS cloud cluster. AWS cloudformation creation of the cluster.
[AWS DC Meetup] Not Your Father’s WebApp: The Cloud-Native Architecture of im...Chris Shenton
A revised version of the presentation I gave to NASA, edited slightly for time, and presented to the AWS DC Meetup 2017-05-31. It starts with simple web apps, their problems, then how to scale them for the cloud, and how images.nasa.gov is built on AWS services: EC2, ELB, DynamoDB, SQS, ElasticTranscoder, CloudSearch and more.
Not Your Father’s Web App: The Cloud-Native Architecture of images.nasa.govChris Shenton
Presentation to the NASA Cloud Community of Interest. Shows the evolution of simple webapps lacking resilience and scalability up to cloud-native apps that tolerate server faults and availability zone outages. Describes how images.nasa.gov leverages S3, EC2, ELB, DynamoDB, CloudSearch, ElasticTranscoder and more to provide a modern, scalable, mobile-friendly site showcasing the best of NASA's images, video and audio.
Understanding User Behavior with Google Analytics.pdfSEO Article Boost
Unlocking the full potential of Google Analytics is crucial for understanding and optimizing your website’s performance. This guide dives deep into the essential aspects of Google Analytics, from analyzing traffic sources to understanding user demographics and tracking user engagement.
Traffic Sources Analysis:
Discover where your website traffic originates. By examining the Acquisition section, you can identify whether visitors come from organic search, paid campaigns, direct visits, social media, or referral links. This knowledge helps in refining marketing strategies and optimizing resource allocation.
User Demographics Insights:
Gain a comprehensive view of your audience by exploring demographic data in the Audience section. Understand age, gender, and interests to tailor your marketing strategies effectively. Leverage this information to create personalized content and improve user engagement and conversion rates.
Tracking User Engagement:
Learn how to measure user interaction with your site through key metrics like bounce rate, average session duration, and pages per session. Enhance user experience by analyzing engagement metrics and implementing strategies to keep visitors engaged.
Conversion Rate Optimization:
Understand the importance of conversion rates and how to track them using Google Analytics. Set up Goals, analyze conversion funnels, segment your audience, and employ A/B testing to optimize your website for higher conversions. Utilize ecommerce tracking and multi-channel funnels for a detailed view of your sales performance and marketing channel contributions.
Custom Reports and Dashboards:
Create custom reports and dashboards to visualize and interpret data relevant to your business goals. Use advanced filters, segments, and visualization options to gain deeper insights. Incorporate custom dimensions and metrics for tailored data analysis. Integrate external data sources to enrich your analytics and make well-informed decisions.
This guide is designed to help you harness the power of Google Analytics for making data-driven decisions that enhance website performance and achieve your digital marketing objectives. Whether you are looking to improve SEO, refine your social media strategy, or boost conversion rates, understanding and utilizing Google Analytics is essential for your success.
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC
Ellisha Heppner, Grant Management Lead, presented an update on APNIC Foundation to the PNG DNS Forum held from 6 to 10 May, 2024 in Port Moresby, Papua New Guinea.
Meet up Milano 14 _ Axpo Italia_ Migration from Mule3 (On-prem) to.pdfFlorence Consulting
Quattordicesimo Meetup di Milano, tenutosi a Milano il 23 Maggio 2024 dalle ore 17:00 alle ore 18:30 in presenza e da remoto.
Abbiamo parlato di come Axpo Italia S.p.A. ha ridotto il technical debt migrando le proprie APIs da Mule 3.9 a Mule 4.4 passando anche da on-premises a CloudHub 1.0.
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBrad Spiegel Macon GA
Brad Spiegel Macon GA’s journey exemplifies the profound impact that one individual can have on their community. Through his unwavering dedication to digital inclusion, he’s not only bridging the gap in Macon but also setting an example for others to follow.
Italy Agriculture Equipment Market Outlook to 2027harveenkaur52
Agriculture and Animal Care
Ken Research has an expertise in Agriculture and Animal Care sector and offer vast collection of information related to all major aspects such as Agriculture equipment, Crop Protection, Seed, Agriculture Chemical, Fertilizers, Protected Cultivators, Palm Oil, Hybrid Seed, Animal Feed additives and many more.
Our continuous study and findings in agriculture sector provide better insights to companies dealing with related product and services, government and agriculture associations, researchers and students to well understand the present and expected scenario.
Our Animal care category provides solutions on Animal Healthcare and related products and services, including, animal feed additives, vaccination
Instagram has become one of the most popular social media platforms, allowing people to share photos, videos, and stories with their followers. Sometimes, though, you might want to view someone's story without them knowing.
2.Cellular Networks_The final stage of connectivity is achieved by segmenting...JeyaPerumal1
A cellular network, frequently referred to as a mobile network, is a type of communication system that enables wireless communication between mobile devices. The final stage of connectivity is achieved by segmenting the comprehensive service area into several compact zones, each called a cell.
2.Cellular Networks_The final stage of connectivity is achieved by segmenting...
Serverless OCR for NASA EVA: AWS Meetup DC 2017-12-12
1. Serverless Optical Character Recognition
in Support of NASA Astronaut Safety
Chris Shenton
CTO V! Studios
AWS DC Meetup
2017-12-12
2. Talk Overview
● The Problem
● The Challenge
● Architectures: server, cloud, serverless
● Lambda: FaaS, Events, Benefits, Limitations
● NASA EVA OCR Architecture
● Security, FedRAMP, ATO
● Serverless Framework
● Gotchas!
● Happy Customer
● Future Challenges and Opportunities
3. Problem: Life-Threatening Spacesuit Failure
On July 16, 2013, water
filled the helmet of
Italian astronaut Luca
Parmitano, creating a
life threatening
scenario which forced
NASA to abort his
spacewalk.
4. The Challenge
● Designs on paper or scanned without OCR ability
● Current reporting processes and procedures
cannot be changed
● About 60 Discrepancy Reports (20 pages) and 190
Task Performance Sheet reports (500 pages) per
month
● Started OCR in 2015, stopped due to server load
● Overwhelmed the EVA Data Integration pipeline
100,000
pages/month
5. Architecture evolution: server to cloud to serverless
● Datacenter: no scaling
● Cloud servers: scaling
● Cloud Containers: scaling
● Serverless: fast, painless scaling
7. Architecture [1b]: Datacenter, no scaling
PDF
doc
Server
OCR
process
TXT
doc
PDF
doc
PDF
doc
PDF
doc
PDF
doc
Load overwhelms OCR server
8. Architecture [2]: Cloud with scaling
PDF
doc
PDF
doc
PDF
doc
PDF
doc
PDF
doc
Pro:
● Scaling handles load spikes
Con:
● Complicated to set up
● Scale out takes a few minutes per server
● Still have to manage OS, security
Autoscaling group
SQS
Queue
Server
OCR
Server
OCR
Server
OCR
Server
OCR
Server
OCR
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
9. Architecture [3]: Cloud Containers with scaling
PDF
doc
PDF
doc
PDF
doc
PDF
doc
PDF
doc
Pro:
● Scaling handles load spikes
● Can deploy immutable instances
Con:
● Have to manage scaling
● Have to manage placement, orchestration
SQS
Queue
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
Container Serve
Container
OCR
Container
OCR
Container
OCR
Container
OCR
Container
OCR
Container Server
Container
OCR
Container
OCR
Container
OCR
Container
OCR
Container
OCR
10. Automatic scaling
Architecture [4a]: Serverless Cloud with built-in scaling
PDF
doc
PDF
doc
PDF
doc
PDF
doc
PDF
doc
Pro:
● Scaling is automatic, nearly instant
● No patching, open ports
Con:
● Some limits on size, lifetime
TXT
doc
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
TXT
doc
TXT
doc
TXT
doc
TXT
doc
11. PDF
page
Automatic scaling
Architecture [4b]: Serverless Cloud with built-in scaling
PDF
doc
PDF
doc
PDF
doc
PDF
doc
PDF
doc
PDF
page
PDF
pageLambda
split doc
Lambda
split doc
Lambda
split doc
Lambda
split doc
Lambda
split doc
PDF
page
PDF
page
PDF
page
PDF
page
PDF
page
PDF
page
PDF
page
Automatic scaling
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
Lambda
OCR
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
TXT
doc
With instant, automatic scaling, we can split PDF docs into PDF pages
and OCR each page to text in parallel, with minimal extra effort.
Exploiting parallelism gives us our results much faster at no extra cost.
13. AWS Lambda is a “Function as a Service” (FaaS)
Function as a service (FaaS) is a category of cloud
computing services that provides a platform
allowing customers to develop, run, and manage
application functionalities without the complexity
of building and maintaining the infrastructure
typically associated with developing and launching
an app. Building an application following this
model is one way of achieving a “serverless”
architecture, and is typically used when building
microservices applications.
Wikipedia
FaaS Products
● AWS Lambda
● Google Cloud Functions
● Microsoft Azure Functions
● IBM OpenWhisk
14. Event-Driven Computing: trigger Functions based on events
S3
ObjectCreated
DynamoDB
Row Changed
API Gateway
GET, PUT, POST, DELETE
Lambda Function
processes event
15. Event-Driven Computing: Lambdas can trigger Lambdas
S3
ObjectCreated
DynamoDB
Row Changed
API Gateway
GET, PUT, POST, DELETE
Lambda Function
processes event
Lambda
invoke Sync or Async
16. Event-Driven Computing: allows interesting architectures
Lambda:
return new S3
location
DynamoDB
Row Changed
store info to
DynamoDB
GET /newUpload
{uploadUrl:
s3://bucket/newKey}
PUT /newKey
...data…
S3 ObjectCreated
{bucket: b,
key: newKey}
Application
Search
Engine
Send metadata to application via HTTP
{method: GET,
url: /newUpload,
data: none}
{uploadUrl:
s3://bucket/newKey}
1
2
18. Lambda Benefits
Example application: process 1000 2-second requests/day
● Server: $16.84/month (AWS t2.small, 24x7)
● Lambda: $1.50/month
No Servers to Manage:
no patching, open ports
or logins
Subsecond Metering:
no idle capacity
Continuous Scaling:
high availability
22. EVA OCR Architecture: Big Wins
● Architecture designed for lowest operational cost possible:
○ S3 files removed after 24 hours: minimal data charges, better security
○ no database cost
● Architectural patterns we used instead of database:
○ track progress with directory prefixes
○ propagate information using S3 object metadata
● Lambda autoscaling, fast scaling, pay only for active use
● Serverless Framework simplified deployment
● but see the Gotchas in a few slides...
23. EVA OCR Architecture: securely connect with cloud policies
EDI App
IAM Role:
eva-app-role
EVA OCR S3 bucket
eva-ocr-dev
● /doc_pdf/
● /page_pdf/
● /page_txt/
● /doc_txt/
EVA Search API
on 3x EC2
HTTPS API on port 5333
Security Group:
sg-002: eva-search
● allow from sg-001
● to port 5333
EVA OCR
Lambda Functions
IAM Role:
eva-ocr-dev-us-east-1-lambda
Security Group:
sg-001: ocreva-lambda-output
EVA
code
uploads
PDF to
/doc_pdf/
HTTP POST
{docid: ‘mydocid’,
page: 42,
text: ‘ocr text…’}
Lambdas read/write pdf
and txt in various folders
IAM Role:
eva-app-role
Policies:
● ocreva-s3-write-doc_pdf
● other EDI policies...
IAM Policy:
ocreva-s3-write-doc_pdf
allow write
arn:aws:s3:::eva-ocr-dev/doc_pdf/
No auth servers were harmed
in the making of this service
24. EVA OCR Security Controls
EVA OCR
S3 Storage
Even though Lambda is currently undergoing
FedRAMP certification, cloud security group
provided ATO based on the following controls:
● GovCloud for sensitive data
● IAM policies, roles and Security Groups
restrict access
● Separate VPC for Lambdas
● No VPC network egress for Lambdas
● Security Group allows output of final
Lambda to EDI Search API
● Encrypted data in transit and at rest
● Static Code Analysis
EVA OCR
Autoscaling
Lambdas
Lambda VPC
private IP space
/16 = 65535 IPs
EVA
Data
Integration
systems
EDI VPC
NASA IP space
limited IPs
SG allows
to port 5333
S3
VPC Endpoint
25. Serverless Framework: from the horse’s mouth
Serverless is your toolkit for
deploying and operating
serverless architectures.
Focus on your application,
not your infrastructure.
serverless.com
npm install serverless -g
serverless create --template hello-world
serverless deploy
curl http://xyz.amazonaws.com/hello-world
27. Gotchas!
● Will get duplicate events if Lambda exits unsuccessfully
○ this is a good thing
● May get duplicate events
○ detect and possibly ignore them (idempotent)
● Timeouts if job takes longer than 300 seconds
○ may have to chain Lambdas
● Overloading destinations is likely due to scale
○ detect, back-off
○ may require handling like Timeouts
● Fast scaling can exhaust limited IP addresses in a VPC
○ use separate VPC for Lambda with large private IP space, e.g., /16 with 65,535 IPs
● S3 eventual consistency
○ use UUIDs in S3 keys to force read consistency
28. Happy Customer
“The work you’ve accomplished
is a big step proving out this
new technology for NASA”
Cuong Q Nguyen, JSC/NASA EVA Office
29. Future Challenges and Opportunities
NASA’s Cuong Nguyen has told us he needs to track assembly, subassembly and part
hierarchies. Can we extract structured text?
He also needs to identify inspector and approval “stamps”. This is not OCR but hard
image processing.