Ever received an unexpected huge monthly bill for lambda? Well, pennies do add up! Same as we take great care in provisioning lambdas, it is equally essential that we create cost awareness. Using simple scenarios, I will dismantle the myths to leave everyone better equipped to keep their costs down.
Everything fails all the time! A quote repeated by many everyday. How does it feel when things fail in production? How do you recover from such situations? How can you make sure they don’t repeat? All these discussed with real production incidents and the measures taken to mitigate such failures. We will also look at few of the most common failure possibilities in a serverless ecosystem.
Remember, when everything fails all the time, you must learn something everyday to be operational all the time!
Building resilient serverless systems with non serverless componentsJeremy Daly
Serverless functions (like AWS Lambda, Google Cloud Functions, and Azure Functions) have the ability to scale almost infinitely to handle massive workload spikes. While this is a great solution for compute, it can be a MAJOR PROBLEM for other downstream resources like RDBMS, third-party APIs, legacy systems, and even most managed services hosted by your cloud provider. Whether you’re maxing out database connections, exceeding API quotas, or simply flooding a system with too many requests at once, serverless functions can DDoS your components and potentially take down your application. In this talk, we’ll discuss strategies and architectural patterns to create highly resilient serverless applications that can mitigate and alleviate pressure on “non-serverless” downstream systems during peak load times.
WKS404 7 Things You Must Know to Build Better Alexa SkillsAmazon Web Services
As we add thousands of skills to the skills store our developers have uncovered some basic and more complex tips for building better skills. Whether you are new to Alexa Skill development or if you have created skills that are live today, this session will help you understand and learn best practices. During this session, you’ll build an Alexa skill using more advanced VUI concepts and we’ll cover how to use AWS services like dynamoDB and S3 to implement the best practices we cover.
Presented at ServerlessConf NYC 2016.
What happens when you give 6k developers access to the cloud? Introducing Cloud Custodian, an opensource project from Capital One, which provides a DSL for AWS management that operates in real-time using cloud watch events and lambda. We use it for the gamut of compliance/encryption/cost controls. What can it do for you?
Serverless design considerations for Cloud Native workloadsTensult
We have built a news website with more than a billion views per month and we are sharing the learnings from that experience covering Serverless architectures, Design considerations, and Gotchas.
Crunch Your Data in the Cloud with Elastic Map Reduce - Amazon EMR HadoopAdrian Cockcroft
A introductory discussion of cloud computing and capacity planning implications is followed by a step by step guide to running a Hadoop job in EMR, and finally a discussion of how to write your own Hadoop queries.
Everything fails all the time! A quote repeated by many everyday. How does it feel when things fail in production? How do you recover from such situations? How can you make sure they don’t repeat? All these discussed with real production incidents and the measures taken to mitigate such failures. We will also look at few of the most common failure possibilities in a serverless ecosystem.
Remember, when everything fails all the time, you must learn something everyday to be operational all the time!
Building resilient serverless systems with non serverless componentsJeremy Daly
Serverless functions (like AWS Lambda, Google Cloud Functions, and Azure Functions) have the ability to scale almost infinitely to handle massive workload spikes. While this is a great solution for compute, it can be a MAJOR PROBLEM for other downstream resources like RDBMS, third-party APIs, legacy systems, and even most managed services hosted by your cloud provider. Whether you’re maxing out database connections, exceeding API quotas, or simply flooding a system with too many requests at once, serverless functions can DDoS your components and potentially take down your application. In this talk, we’ll discuss strategies and architectural patterns to create highly resilient serverless applications that can mitigate and alleviate pressure on “non-serverless” downstream systems during peak load times.
WKS404 7 Things You Must Know to Build Better Alexa SkillsAmazon Web Services
As we add thousands of skills to the skills store our developers have uncovered some basic and more complex tips for building better skills. Whether you are new to Alexa Skill development or if you have created skills that are live today, this session will help you understand and learn best practices. During this session, you’ll build an Alexa skill using more advanced VUI concepts and we’ll cover how to use AWS services like dynamoDB and S3 to implement the best practices we cover.
Presented at ServerlessConf NYC 2016.
What happens when you give 6k developers access to the cloud? Introducing Cloud Custodian, an opensource project from Capital One, which provides a DSL for AWS management that operates in real-time using cloud watch events and lambda. We use it for the gamut of compliance/encryption/cost controls. What can it do for you?
Serverless design considerations for Cloud Native workloadsTensult
We have built a news website with more than a billion views per month and we are sharing the learnings from that experience covering Serverless architectures, Design considerations, and Gotchas.
Crunch Your Data in the Cloud with Elastic Map Reduce - Amazon EMR HadoopAdrian Cockcroft
A introductory discussion of cloud computing and capacity planning implications is followed by a step by step guide to running a Hadoop job in EMR, and finally a discussion of how to write your own Hadoop queries.
Serverless on AWS : Understanding the hard parts at Froscon 2019Vadym Kazulkin
In unserem Vortrag tauchen wir tiefer in die Serverless-Welt ein und zeigen wie eine produktionsreife Serverless-Anwendung mithilfe von AWS-Cloud mit dem Technologie-Stack API Gateway, SNS, Lambda and DynamoDB aufgebaut werden kann. Dabei gehen wir auf die Herausforderungen der jeweiligen Services ein, wie "cold start" bei Lamda oder "provisioned throughput" und "adaptice capacity" bei DynamoDB. Dabei zeigen wir, welche Strategien und Wege es gibt, damit umzugehen. Außerdem behandeln wir solche Themen wie Implementierung von Aggregationslogik und (Scheduled) Auto Scaling bei DynamoDB. Am Ende werfen wir einen Blick in die Zukunft und sprechen über die erste relationale serverless Datenbank "Aurora Serverless"
BUILDING Serverless apps with MongoDB AtLAS, AWS Lambda and Step FunctionsRaphael Londner
In this session, AWS Solutions Architect Paul Sears will provide an overview of AWS Lambda functions, including some key integration use cases with MongoDB Atlas. Developer Advocate Raphael Londner will walk you through how to code a Lambda function connected to MongoDB Atlas, with a specific focus on performance optimization. Raphael will then demonstrate how to orchestrate multiple Lambda functions inside a state machine built on top of AWS Step Functions.
While there are many Cloud design patterns for infrastructure, there are also many Cloud design patterns for developers. Come and learn how you can take your software design patterns and apply them to the next generation of cloud applications, or simply modernise your existing software architectures.
Speaker: Arden Packeer, Solutions Architect, Amazon Web Services
Serverless on AWS : Understanding the hard parts at Serverless Meetup Dusseld...Vadym Kazulkin
We will look into the challenges, such as "cold start" with Lamda or "provisioned throughput" with DynamoDB and show you which strategies and options exist. We will also address topics like Tracing of Lambda-Functions and implementation of aggregation logic, Scaling and the Capacity Modes (reserved, provisioned and on-demand) options for DynamoDB. Finally, we'll have a look at the first relational serverless data base "Aurora Serverless" and its new data API.
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
AWS Lambda and Serverless framework: lessons learned while building a serverl...Luciano Mammino
Planet9energy.com is a new electricity company that is building a sophisticated analytics and energy trading platform for the UK market. Since the earliest days of the company we took the unconventional decision to go serverless and finally we are building the product on top of AWS Lambda and the Serverless framework using Node.js. In this talk we will discuss why we took this radical decision, what are the pros and cons of this approach and what are the main issues we faced as a tech team in our design and development experience. We will discuss how normal things like testing and deployment need to be re-thought to work on a serverless fashion but also the benefits of (almost) infinite auto-scalability and the piece of mind of not having to manage hundreds of servers. Finally we will underline how Node.js seems to fit naturally in this scenario and how it makes developing serverless applications extremely convenient.
Thanks to Padraig O'Brien and Luciano Mammino for speaking this month.
Speakers Bio:
Padraig O'Brien
Podge @Podgeypoos79 is a software engineer for over 15 years, most of that was spent developing in .NET and SQL Server, designing and building large scale data intensive applications. Lately he has shifted towards open source technologies and is spending most of his time learning Node.js, Scala and cool data tech like Spark, Cassandra. He is also working on a “super-secret” project called UnicornDB, don’t tell anybody!
In his spare time he helps out with organising some meetups like NodeSchool Dublin, NodeSchool Dun Laoghaire and teaching Kanban via Agile Lean Ireland.
Luciano Mammino
Luciano @loige is a Software Engineer born in 1987, the same year that the Nintendo released “Super Mario Bros” in Europe, which, “by chance” is his favourite game! His primary passion is code and he is extremely fascinated by the web, smart apps and everything that's creative like music, art and design. He started coding at the age of 12 using his father's old i386 provided only with DOS and the qBasic interpreter.He is a senior software developer at Planet9Energy in Dublin and he loves JavaScript (React/Node.js). He is also the co-author of "Node.js design patterns" 2nd edition (Packt, http://amzn.to/1ZF279B).
Hosted by Intercom, sponsored by Nearform and organised by Node.js Dublin (https://www.meetup.com/Dublin-Node-js-Meetup/events/236870576/)
Lambda and serverless - DevOps North East Jan 2017Mike Shutlar
Introduction to AWS Lambda, serverless architectures, & the new AWS Serverless Application Model.
Source code for demo serverless application available here:
https://github.com/infectedsoundsystem/lambda-refarch-webapp
This session will begin with an introduction to non-relational (NoSQL) databases and compare them with relational (SQL) databases. We will also explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service. Learn the fundamentals of DynamoDB and see the new DynamoDB console first-hand as we discuss common use cases and benefits of this high-performance key-value and JSON document store.
(BDT310) Big Data Architectural Patterns and Best Practices on AWSAmazon Web Services
The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.
Scaling on AWS for the First 10 Million Users (ARC206) | AWS re:Invent 2013Amazon Web Services
Cloud computing gives you a number of advantages in being able to scale on demand, easily replace whole parts of your infrastructure, and much more. As a new business looking to use the cloud, you inevitably ask yourself, Where do I start? Join us at this session to understand some of the common patterns and recommended areas of focus you can expect to work through while scaling an infrastructure to handle going from zero to millions of users. From leveraging highly scalable AWS services to making smart decisions on building out your application, you'll learn a number of best practices for scaling your infrastructure in the cloud. The patterns and practices reviewed in this session will get you there.
What does Serverless mean for DevOps, in practical terms? While Serverless does reduce the need for server-centric DevOps, it poses new challenges in many areas including security, app deployment and cloud resource provisioning, partly due to an explosion of "nanoservices". Based on a current project using AWS, we cover relevant tools, techniques and tips to deliver a smooth serverless experience for development through to production.
Delivered at Bristol DevOps meetup, 27 Jun 2018. To see detailed notes covering extra points not on slides, click the Notes link just below (or download the Powerpoint).
Update: here's the correct link for Gojko Adzic talk on the Backendless slide - https://www.youtube.com/watch?v=w7X4gAQTk2E
Building an Amazon Datawarehouse and Using Business Intelligence Analytics ToolsAmazon Web Services
Using AWS has never been easier or more affordable to solve business problems and uncover new opportunities using data. Now, businesses of all sizes and across all industries can take advantage of big data technologies and easily collect, store, process, analyze, and share their data. Gain a thorough understanding of what AWS offers across the big data lifecycle and learn architectural best practices for applying these technologies to your projects. We will also deep dive into how to use AWS services such as Kinesis, DynamoDB, Redshift, and Quicksight to optimize logging, build real-time applications, and analyze and visualize data at any scale.
Serverless on AWS : Understanding the hard parts at Froscon 2019Vadym Kazulkin
In unserem Vortrag tauchen wir tiefer in die Serverless-Welt ein und zeigen wie eine produktionsreife Serverless-Anwendung mithilfe von AWS-Cloud mit dem Technologie-Stack API Gateway, SNS, Lambda and DynamoDB aufgebaut werden kann. Dabei gehen wir auf die Herausforderungen der jeweiligen Services ein, wie "cold start" bei Lamda oder "provisioned throughput" und "adaptice capacity" bei DynamoDB. Dabei zeigen wir, welche Strategien und Wege es gibt, damit umzugehen. Außerdem behandeln wir solche Themen wie Implementierung von Aggregationslogik und (Scheduled) Auto Scaling bei DynamoDB. Am Ende werfen wir einen Blick in die Zukunft und sprechen über die erste relationale serverless Datenbank "Aurora Serverless"
BUILDING Serverless apps with MongoDB AtLAS, AWS Lambda and Step FunctionsRaphael Londner
In this session, AWS Solutions Architect Paul Sears will provide an overview of AWS Lambda functions, including some key integration use cases with MongoDB Atlas. Developer Advocate Raphael Londner will walk you through how to code a Lambda function connected to MongoDB Atlas, with a specific focus on performance optimization. Raphael will then demonstrate how to orchestrate multiple Lambda functions inside a state machine built on top of AWS Step Functions.
While there are many Cloud design patterns for infrastructure, there are also many Cloud design patterns for developers. Come and learn how you can take your software design patterns and apply them to the next generation of cloud applications, or simply modernise your existing software architectures.
Speaker: Arden Packeer, Solutions Architect, Amazon Web Services
Serverless on AWS : Understanding the hard parts at Serverless Meetup Dusseld...Vadym Kazulkin
We will look into the challenges, such as "cold start" with Lamda or "provisioned throughput" with DynamoDB and show you which strategies and options exist. We will also address topics like Tracing of Lambda-Functions and implementation of aggregation logic, Scaling and the Capacity Modes (reserved, provisioned and on-demand) options for DynamoDB. Finally, we'll have a look at the first relational serverless data base "Aurora Serverless" and its new data API.
Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.
AWS Lambda and Serverless framework: lessons learned while building a serverl...Luciano Mammino
Planet9energy.com is a new electricity company that is building a sophisticated analytics and energy trading platform for the UK market. Since the earliest days of the company we took the unconventional decision to go serverless and finally we are building the product on top of AWS Lambda and the Serverless framework using Node.js. In this talk we will discuss why we took this radical decision, what are the pros and cons of this approach and what are the main issues we faced as a tech team in our design and development experience. We will discuss how normal things like testing and deployment need to be re-thought to work on a serverless fashion but also the benefits of (almost) infinite auto-scalability and the piece of mind of not having to manage hundreds of servers. Finally we will underline how Node.js seems to fit naturally in this scenario and how it makes developing serverless applications extremely convenient.
Thanks to Padraig O'Brien and Luciano Mammino for speaking this month.
Speakers Bio:
Padraig O'Brien
Podge @Podgeypoos79 is a software engineer for over 15 years, most of that was spent developing in .NET and SQL Server, designing and building large scale data intensive applications. Lately he has shifted towards open source technologies and is spending most of his time learning Node.js, Scala and cool data tech like Spark, Cassandra. He is also working on a “super-secret” project called UnicornDB, don’t tell anybody!
In his spare time he helps out with organising some meetups like NodeSchool Dublin, NodeSchool Dun Laoghaire and teaching Kanban via Agile Lean Ireland.
Luciano Mammino
Luciano @loige is a Software Engineer born in 1987, the same year that the Nintendo released “Super Mario Bros” in Europe, which, “by chance” is his favourite game! His primary passion is code and he is extremely fascinated by the web, smart apps and everything that's creative like music, art and design. He started coding at the age of 12 using his father's old i386 provided only with DOS and the qBasic interpreter.He is a senior software developer at Planet9Energy in Dublin and he loves JavaScript (React/Node.js). He is also the co-author of "Node.js design patterns" 2nd edition (Packt, http://amzn.to/1ZF279B).
Hosted by Intercom, sponsored by Nearform and organised by Node.js Dublin (https://www.meetup.com/Dublin-Node-js-Meetup/events/236870576/)
Lambda and serverless - DevOps North East Jan 2017Mike Shutlar
Introduction to AWS Lambda, serverless architectures, & the new AWS Serverless Application Model.
Source code for demo serverless application available here:
https://github.com/infectedsoundsystem/lambda-refarch-webapp
This session will begin with an introduction to non-relational (NoSQL) databases and compare them with relational (SQL) databases. We will also explain the fundamentals of Amazon DynamoDB, a fully managed NoSQL database service. Learn the fundamentals of DynamoDB and see the new DynamoDB console first-hand as we discuss common use cases and benefits of this high-performance key-value and JSON document store.
(BDT310) Big Data Architectural Patterns and Best Practices on AWSAmazon Web Services
The world is producing an ever increasing volume, velocity, and variety of big data. Consumers and businesses are demanding up-to-the-second (or even millisecond) analytics on their fast-moving data, in addition to classic batch processing. AWS delivers many technologies for solving big data problems. But what services should you use, why, when, and how? In this session, we simplify big data processing as a data bus comprising various stages: ingest, store, process, and visualize. Next, we discuss how to choose the right technology in each stage based on criteria such as data structure, query latency, cost, request rate, item size, data volume, durability, and so on. Finally, we provide reference architecture, design patterns, and best practices for assembling these technologies to solve your big data problems at the right cost.
Scaling on AWS for the First 10 Million Users (ARC206) | AWS re:Invent 2013Amazon Web Services
Cloud computing gives you a number of advantages in being able to scale on demand, easily replace whole parts of your infrastructure, and much more. As a new business looking to use the cloud, you inevitably ask yourself, Where do I start? Join us at this session to understand some of the common patterns and recommended areas of focus you can expect to work through while scaling an infrastructure to handle going from zero to millions of users. From leveraging highly scalable AWS services to making smart decisions on building out your application, you'll learn a number of best practices for scaling your infrastructure in the cloud. The patterns and practices reviewed in this session will get you there.
What does Serverless mean for DevOps, in practical terms? While Serverless does reduce the need for server-centric DevOps, it poses new challenges in many areas including security, app deployment and cloud resource provisioning, partly due to an explosion of "nanoservices". Based on a current project using AWS, we cover relevant tools, techniques and tips to deliver a smooth serverless experience for development through to production.
Delivered at Bristol DevOps meetup, 27 Jun 2018. To see detailed notes covering extra points not on slides, click the Notes link just below (or download the Powerpoint).
Update: here's the correct link for Gojko Adzic talk on the Backendless slide - https://www.youtube.com/watch?v=w7X4gAQTk2E
Building an Amazon Datawarehouse and Using Business Intelligence Analytics ToolsAmazon Web Services
Using AWS has never been easier or more affordable to solve business problems and uncover new opportunities using data. Now, businesses of all sizes and across all industries can take advantage of big data technologies and easily collect, store, process, analyze, and share their data. Gain a thorough understanding of what AWS offers across the big data lifecycle and learn architectural best practices for applying these technologies to your projects. We will also deep dive into how to use AWS services such as Kinesis, DynamoDB, Redshift, and Quicksight to optimize logging, build real-time applications, and analyze and visualize data at any scale.
Aerospike TCO Vs memory-first architecturesAerospike
See how Aerospike compares to in-memory or caching technologies as you scale. Aerospike TCO is very low in comparison as Flash/SSD technology is used to persist the data.
Deepankar Reddy and Ishan Chhabra (Rocket Fuel)
Rocket Fuel is a marketing technology company that participates in 120+ billion real-time bidding auctions daily to show the right ad to the right user at the right time for our clients. In this talk, we discuss our efforts to systematically identify causes of, and how to decrease, long-tail read latencies.
Cost is often the conversation starter when customers think about moving to the cloud. AWS helps lower costs for customers through its “pay only for what you use” pricing model, frequent price drops, and pricing model choice to support variable & stable workloads. In this session, you will learn about the financial considerations of owning and operating a traditional data center or managed hosting provider versus utilizing AWS. We will detail our TCO methodology and showcase cost comparisons for some common customer use-cases. We’ll also cover a few AWS cost optimization areas, including Spot and Reserved Instances, EC2 Auto Scaling, and consolidated billing.
Slide chia sẻ công nghệ về caching, thông qua slide này bạn sẽ trả lời được những câu hỏi như:
- Caching là gì
- Làm sao sử dụng cũng như xây dựng hệ thống caching
- Tại sao cache giúp tăng tốc ứng dụng lên vài chục, vài trăm lần
- Các hệ thống lớn của Facebook, Twitter, ... đang sử dụng cache thế nào
- ...
Slide chia sẻ về công nghệ về caching, thông qua slide này bạn sẽ trả lời được những câu hỏi như:
- Caching là gì
- Làm sao sử dụng cũng như xây dựng hệ thống caching
- Tại sao cache giúp tăng tốc ứng dụng lên vài chục, vài trăm lần
- Các hệ thống lớn của Facebook, Twitter, ... đang sử dụng cache thế nào
- ...
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Henning Jacobs
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.
(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New FeaturesAmazon Web Services
Learn the specifics of Amazon RDS for PostgreSQL’s capabilities and extensions that make it powerful. This session begins with a brief overview of the RDS PostgreSQL service, how it provides High Availability & Durability and will then deep dive into the new features that we have released since re:Invent 2014, including major version upgrade and newly added PostgreSQL extensions to RDS PostgreSQL. During the session, we will also discuss lessons learned running a large fleet of PostgreSQL instances, including specific recommendations. In addition we will present benchmarking results looking at differences between the 9.3, 9.4 and 9.5 releases.
Cloud Storage Comparison: AWS vs Azure vs Google vs IBMRightScale
As public cloud storage services mature, it becomes easier to make apples-to-apples comparisons. We drill down on the latest specs and features for object, block, archival, and file storage across AWS, Azure, Google, and IBM. We also compare prices for a variety of storage scenarios.
AWS vs Azure vs Google Cloud Storage Deep DiveRightScale
Cloud services keep evolving, and cloud storage is no different. It can be difficult to keep up to date with the latest from each cloud provider and understand how they compare. We’ll drill down on object, block, archival, and file storage for the leading public clouds. We’ll also compare prices for a variety of storage scenarios.
Speaker: Akira Kurogane, Senior Technical Services Engineer, MongoDB
Level: 300 (Advanced)
Track: Performance
One week your active dataset consumes 90% of available RAM. The next week it's 110%. Is that a 10% or 99% performance degradation? Let's discover what it looks like when different hardware capacity limitations are hit. For example, memory vs. disk bottlenecks, the rare CPU bottleneck and network bottlenecks, seeing what happens when you drop a crucial index during peak load, or what happens when you run multiple WiredTiger nodes on the same server without limiting their cache size.
What You Will Learn:
- Performance analysis
- Post-mortem log analysis
- Capacity planning
AWS Webcast - Cost and Performance Optimization in Amazon RDSAmazon Web Services
Amazon RDS makes it easy to set up, operate, and scale relational databases in the cloud. The service offers a variety of options for optimizing the performance level delivered, as well as optimizing your spending. In this webinar, we will show a variety of techniques for implementing the right performance level for your application.
Learning Objectives:
• Understand the Amazon RDS options that change database performance and cost
• Select the appropriate performance and cost level for your specific application Who Should Attend:
• Technical Amazon RDS customers and prospective customers
Measuring Database Performance on Bare Metal AWS InstancesScyllaDB
AWS has recently announced a new type of instance targeted at I/O intensive applications, the i3.metal. That instance does away with the virtualization layer altogether and gives back the resources that would otherwise be used by the hypervisor back to the application.
To use all of those resources — 72 CPUs and 512GB of memory — a database needs to be have the ability to scale both up and out.
In this webinar we will look into the performance of Scylla running in a few of those instances versus Apache Cassandra running in their sweet spot, a larger fleet of smaller instances. We will discuss how much of the gains come from the database design and how much come from the removal of the virtualization layer.
Key takeaways:
How to properly compare two different database technologies while being fair to both
How to choose the optimal setup for your Scylla deployment
How AWS’s bare metal servers enable Scylla users to draw a significant performance boost.
AWS Storage and Database Architecture Best Practices (DAT203) | AWS re:Invent...Amazon Web Services
Learn about architecture best practices for combining AWS storage and database technologies. We outline AWS storage options (Amazon EBS, Amazon EC2 Instance Storage, Amazon S3 and Amazon Glacier) along with AWS database options including Amazon ElastiCache (in-memory data store), Amazon RDS (SQL database), Amazon DynamoDB (NoSQL database), Amazon CloudSearch (search), Amazon EMR (hadoop) and Amazon Redshift (data warehouse). Then we discuss how to architect your database tier by using the right database and storage technologies to achieve the required functionality, performance, availability, and durability—at the right cost.
AWS March 2016 Webinar Series - Managed Database Services on Amazon Web ServicesAmazon Web Services
AWS customers can choose among a variety of managed database services in addition to running databases in Amazon EC2 on their own. Managed database services remove the burden of implementing, managing and maintaining the database and let you focus on your applications.
In this webinar, we will help you understand the differences and common areas of these managed database, and how to choose one or more. We will explain the fundamentals of Amazon RDS, a relational database service in the cloud; Amazon DynamoDB, a fully managed NoSQL database service; Amazon ElastiCache, a fast, in-memory caching service in the cloud; and Amazon Redshift, a fully managed, petabyte-scale data-warehouse solution. We will also cover how each service can help support your application, how much each service costs, and how to get started.
Learning Objectives:
• Understand the Managed Database Service options available on AWS
• Learn how to choose among the Managed Database Services on AWS for your use cases
Who Should Attend:
• IT Professionals, IT Managers, DBAs, Systems Administrators and Developers
Future of computing is boring (and that is exciting!) alekn
We see a trend where computing becomes a metered utility similar to how the electric grid evolved. Initially electricity was generated locally but economies of scale (and standardization) made it more efficient and economical to have utility companies managing the electric grid. Similar developments can be seen in computing where scientific grids paved the way for commercial cloud computing offerings. However, in our opinion, that evolution is far from finished and in this paper we bring forward the remaining challenges and propose a vision for the future of computing. In particular we focus on diverging trends in the costs of computing and developer time, which suggests that future computing architectures will need to optimize for developer time.
Keywords—cloud computing, future, economics, cost
Datadog: a Real-Time Metrics Database for One Quadrillion Points/DayC4Media
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2mAKgJi.
Ian Nowland and Joel Barciauskas talk about the challenges Datadog faces as the company has grown its real-time metrics systems that collect, process, and visualize data to the point they now handle trillions of points per day. They also talk about how the architecture has evolved, and what they are looking to in the future as they architect for a quadrillion points per day. Filmed at qconnewyork.com.
Ian Nowland is the VP Engineering Metrics and Alerting at Datadog. Joel Barciauskas currently leads Datadog's distribution metrics team, providing accurate, low latency percentile measures for customers across their infrastructure.
Enterprise Serverless Adoption. An Experience ReportSheenBrisals
The popularity of Serverless is growing strong every day. Though it is gaining strength in the industry, its adoption in larger enterprises is not on par with others. Unlike start-ups, large organizations look for a systematic and carefully planned approach to adopt serverless. The experience is not readily available nor shared with the larger community. This talk aims to fill in the gaps by sharing a unique serverless adoption story at the LEGO Group. It will take you through the evolution of serverless adoption, ways to grow serverless teams, best practices, and achieving sustainability with serverless.
Is the serverless application you build sustainable? Does the development process promote sustainability? Are you applying the cloud best practices to reduce carbon footprint? These questions become prevalent as serverless adoption grows. This talk addresses these and will guide you to achieve them.
How to Grow a Serverless Team in an EnterpriseSheenBrisals
Many of us agree that adopting serverless in an enterprise has challenges. These challenges increase when the enterprise has no serverless expertise within itself. We know that serverless requires a new mindset and a different way of approaching application development. When business pressure mounts to deliver solutions faster, organisations often fall into the trap of quickly building a team by pulling engineers from different sources in order to satisfy the business goals. What such quick-fix solutions fail to achieve is the basic growth of the serverless knowledge and skills of its employees. Thus, the serverless adoption challenges left unattended.
For a company that credits its own growth on the growth of its people, these quick-fix approaches are not going to offer much help. For a long term gain and to develop a growth culture within the organisation, it is important to recognise the uplift of its serverless expertise. This is where organically growing a serverless team becomes beneficial.
In this talk, taking inspiration from the nature, I will take you through few important phases of growing a serverless team, and discuss how it can bring near term as well as long term benefits to an organisation. Let’s all grow and not build a serverless team!
Enterprise Serverless Adoption. An Experience ReportSheenBrisals
The adoption of Serverless is growing in the industry. However, its adoption in larger enterprises is somewhat slow compared to start-ups and individual developers. This talk tells an enterprise adoption success story and shares insights into the secrets behind its success!
Serverless needs no introduction these days. It is viewed as a magic recipe for organisations moving to cloud and for those moving beyond the container hell.
LEGO.com was migrated from a legacy monolith eCommerce platform onto serverless on AWS. This employed serverless and managed services at its core within an agile development process. Is early success with serverless a springboard to future possibilities? Does serverless really deliver what it promises?
We will look at how serverless helped in the migration and what can it do to the organisation beyond its initial adoption!
Many of us agree that adopting serverless in an enterprise has challenges. These challenges increase when the enterprise has no serverless expertise within itself. We know that serverless requires a new mindset and a different way of approaching application development. When business pressure mounts to deliver solutions faster, organisations often fall into the trap of quickly building a team by pulling engineers from different sources in order to satisfy the business goals. What such quick-fix solutions fail to achieve is the basic growth of the serverless knowledge and skills of its employees. Thus, the serverless adoption challenges left unattended.
For a company that credits its own growth on the growth of its people, these quick-fix approaches are not going to offer much help. For a long term gain and to develop a growth culture within the organisation, it is important to recognise the uplift of its serverless expertise. This is where organically growing a serverless team becomes beneficial.
In this talk, taking inspiration from the nature, I will take you through few important phases of growing a serverless team, and discuss how it can bring near term as well as long term benefits to an organisation. Let’s all grow and not build a serverless team!
Design and Develop Serverless Applications as Set-PiecesSheenBrisals
The emergence of microservices made us rethink how we built business applications. It led us to the migration of complex monolith applications to countless microservices. The cloud adoption and the suitability of the container services helped to revolutionize microservices.
Amidst this adoration for microservices came serverless, the next evolution of the cloud. Serverless brought deeper granularity with its technology offering. It tested our thinking, shifted our minds, and questioned us the way we’ve been building microservices. The agility of event-driven computing and the granularity of serverless allows us to break traditional microservices into multiple pieces.
In this talk, we will see how cloud and serverless help us build those pieces in isolation to achieve acceleration in our modern application development process.
Serverless Microservices Communication with Amazon EventBridgeSheenBrisals
The combination of cloud, serverless and microservices has taken the service implementation to a different level. Though this has accelerated the monolith to microservices transformation, it has also introduced new complexities around service-to-service communication. With every new service added to the system, the order of communications complexity also increases.
Though AWS services such as SNS, SQS and others helped to some extend, they however failed to offer a flexible way to enable filtered routing of messages between microservices. This is where Amazon’s EventBridge makes its mark in alleviating many of these concerns.
AWS EventBridge promotes a hub-and-spoke communication model between microservices. With its flexible and powerful message filtering capability, services can have a renewed way of performing event-driven communication between them. This talk will start by explaining EventBridge and then, with the help of real use-case scenarios, explain how to enable message routing and filtering while working with the event bus.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns
Unlocking Business Potential: Tailored Technology Solutions by Prosigns
Discover how Prosigns, a leading technology solutions provider, partners with businesses to drive innovation and success. Our presentation showcases our comprehensive range of services, including custom software development, web and mobile app development, AI & ML solutions, blockchain integration, DevOps services, and Microsoft Dynamics 365 support.
Custom Software Development: Prosigns specializes in creating bespoke software solutions that cater to your unique business needs. Our team of experts works closely with you to understand your requirements and deliver tailor-made software that enhances efficiency and drives growth.
Web and Mobile App Development: From responsive websites to intuitive mobile applications, Prosigns develops cutting-edge solutions that engage users and deliver seamless experiences across devices.
AI & ML Solutions: Harnessing the power of Artificial Intelligence and Machine Learning, Prosigns provides smart solutions that automate processes, provide valuable insights, and drive informed decision-making.
Blockchain Integration: Prosigns offers comprehensive blockchain solutions, including development, integration, and consulting services, enabling businesses to leverage blockchain technology for enhanced security, transparency, and efficiency.
DevOps Services: Prosigns' DevOps services streamline development and operations processes, ensuring faster and more reliable software delivery through automation and continuous integration.
Microsoft Dynamics 365 Support: Prosigns provides comprehensive support and maintenance services for Microsoft Dynamics 365, ensuring your system is always up-to-date, secure, and running smoothly.
Learn how our collaborative approach and dedication to excellence help businesses achieve their goals and stay ahead in today's digital landscape. From concept to deployment, Prosigns is your trusted partner for transforming ideas into reality and unlocking the full potential of your business.
Join us on a journey of innovation and growth. Let's partner for success with Prosigns.
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I ...Juraj Vysvader
In 2015, I used to write extensions for Joomla, WordPress, phpBB3, etc and I didn't get rich from it but it did have 63K downloads (powered possible tens of thousands of websites).
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Navigating the Metaverse: A Journey into Virtual Evolution"Donna Lenk
Join us for an exploration of the Metaverse's evolution, where innovation meets imagination. Discover new dimensions of virtual events, engage with thought-provoking discussions, and witness the transformative power of digital realms."
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
Accelerate Enterprise Software Engineering with PlatformlessWSO2
Key takeaways:
Challenges of building platforms and the benefits of platformless.
Key principles of platformless, including API-first, cloud-native middleware, platform engineering, and developer experience.
How Choreo enables the platformless experience.
How key concepts like application architecture, domain-driven design, zero trust, and cell-based architecture are inherently a part of Choreo.
Demo of an end-to-end app built and deployed on Choreo.
Globus Connect Server Deep Dive - GlobusWorld 2024Globus
We explore the Globus Connect Server (GCS) architecture and experiment with advanced configuration options and use cases. This content is targeted at system administrators who are familiar with GCS and currently operate—or are planning to operate—broader deployments at their institution.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Globus
Large Language Models (LLMs) are currently the center of attention in the tech world, particularly for their potential to advance research. In this presentation, we'll explore a straightforward and effective method for quickly initiating inference runs on supercomputers using the vLLM tool with Globus Compute, specifically on the Polaris system at ALCF. We'll begin by briefly discussing the popularity and applications of LLMs in various fields. Following this, we will introduce the vLLM tool, and explain how it integrates with Globus Compute to efficiently manage LLM operations on Polaris. Attendees will learn the practical aspects of setting up and remotely triggering LLMs from local machines, focusing on ease of use and efficiency. This talk is ideal for researchers and practitioners looking to leverage the power of LLMs in their work, offering a clear guide to harnessing supercomputing resources for quick and effective LLM inference.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
OpenFOAM solver for Helmholtz equation, helmholtzFoam / helmholtzBubbleFoamtakuyayamamoto1800
In this slide, we show the simulation example and the way to compile this solver.
In this solver, the Helmholtz equation can be solved by helmholtzFoam. Also, the Helmholtz equation with uniformly dispersed bubbles can be simulated by helmholtzBubbleFoam.
3. talk
focus
• why serverless
• how to compute cost of
• lambda function
• API gateway
• dynamoDB
• tools & techniques
• best practices to save cost
• Serverless at LEGO.com
11. cost ❖ number of invocations
❖ 1,000,000 requests free
❖ $0.20 per 1m request
request
❖ based on execution time & memory
❖ 1m requests using 512mb & 100ms
⁼ 1m x 0.1s x (512/1024)gb = 50,000
❖ 400,000 gb-s free
❖ $0.0000166667 per gb-s
compute gb-s
total cost ❖ request cost + compute cost
Freebie
is per
account
12. provisioned
concurrency
❖ number of invocations
❖ $0.20 per 1m request
❖ reservation duration & how many & memory
❖ 1 hour hire. 100 concurrency. 512mb memory
⁼ 3,600s x 100 x (512/1024)gb
⁼ 3,600s x 50gb = 180,000 gb-s
❖ $0.000004167 per gb-s
compute gb-s
total cost ❖ provisioned + request + compute cost
Sorry,
No
Freebies
❖ based on execution time &
memory
❖ $0.0000166667 per gb-s
provisioned cost
14. basic
invoked every
minute
uses 128mb RAM
average 75ms
every minute – 128mb – 100ms cost
request calls 60 x 24 x 30 = 43,200 0.00
compute duration (s) 43,200 x 100ms = 4,320s
gb-s 4,320 x 128mb/1024 = 540gb-s 0.00
total cost $ 0.00
every
minute
75ms
128mb
18. ❖ number of api calls
❖ 1,000,000 requests free
❖ $3.50 per 1m request [REST API]
❖ $1.00 per 1m request [HTTP API]
calls
❖ $0.09/gb for the first 10tb
❖ $0.085/gb for the next 40tb
❖ $0.07/gb for the next 100tb
❖ $0.05/gb for the next 350tb
data transfer out
cache
❖ $0.020/hr for 0.5gb
❖ $0.038/hr for 1.6gb
❖ …..
❖ $3.800/hr for 237gb
api
19. moderate
10 million calls
per month
uses 512mb RAM
average 300ms
10m – 512mb – 300ms cost
lambda as previous $20.14
gateway calls 9m x $3.50/m $31.50
total cost $ 51.64
api gateway
- calls
10 million
300ms
512mb
20. moderate
10 million calls
per month
uses 512mb RAM
average 300ms
10m – 512mb – 300ms cost
lambda as previous $20.14
gateway calls 9m x $3.50/m $31.50
data 9m x 5kb -> 45gb x $0.09 $4.05
total cost $ 55.69
5kb
data
api gateway
- calls
- data
10 million
300ms
512mb
21. moderate
10 million calls
per month
uses 512mb RAM
average 300ms 10m – 512mb – 300ms cost
lambda as previous $20.14
gateway calls 9m x $3.50/m $31.50
data 9m x 5kb -> 45gb x $0.09 $4.05
cache $0.020 x 24 x 30 $14.40
total cost $ 70.09
0.5gb
cache
api gateway
- calls
- data
- cache
5kb
data
10 million
300ms
512mb
22. high
200m api calls
uses 512mb RAM
average 200ms
api gateway
- calls
- data
- cache
100m lambda 100 million
200ms
512mb
200 million
5kb
data
6.1gb
cache
200m api – 100m lambda – 512mb – 200ms cost
lambda calls 99m x $0.20/m $19.80
compute 100m x 0.2s = 20m-s
20m-s x (512/1024) = 10m gb-s
(10m – 400k) x $0.00001667 $160.03
total cost $ 179.83
23. high
200m api calls
uses 512mb RAM
average 200ms 200m api – 100m lambda – 512mb – 200ms cost
lambda as previous $179.83
gateway calls 199m x $3.50/m $696.50
data 200m x 5kb -> 1000gb x $0.09 $90.00
cache $0.200 x 24 x 30 $144.00
total cost (month) $ 1110.33
api gateway
- calls
- data
- cache
100m lambda 100 million
200ms
512mb
200 million
5kb
data
6.1gb
cache
28. reading strong
eventual 4 KB = 1 RCU/RRU
4 KB
4 KB = 2 RCU/RRU
4 KB
4 KB = 4 RCU/RRU
4 KB
TRXN-al
provisioned: $0.0362 per million RCU
on-demand: $0.25 per million RRU
29. writing
standard 1 KB = 1 WCU/WRU
1 KB = 2 WCU/WRU
TRXN-al
provisioned: $0.18056 per million WCU
on-demand: $1.25 per million WRU
30. dynamodb
attributes token_id session date_time
hdsf-der-rrddf Y45ui4-7856-dfsdg-gdfd 732478234334673
hsdsdf-ase-rrddf Yfrui4-77546-df4fg-gdfd 732478234334454
hsdsdf-ase-rrddf Yfrui4-77546-df4fg-gdfd 732478234334454
hsdsdf-ase-rrddf Yfrui4-77546-df4fg-gdfd 732478234334454
hsdsdf-ase-rrddf Yfrui4-77546-df4fg-gdfd 732478234334454
1m writes per day
3m reads per day
1gb data storage
100 bytes item
1 m
writes
3 m
reads
30m writes/month
90m reads/month
31. 90m reads per month
1 day = 3m reads of 4kb data per read
= 35 RCU
1 hour = 125,000 reads
1 second = 35 reads
provisioned on-demand
$0.0000000361 per read OR
$0.0361 per million reads
price: $0.251 per million reads
price:
costs $3.25 / month costs $22.59 / month
FREE
25 RCU
32. 30m writes per month
1 day = 1m writes of 1kb data per write
= 12 WCU
1 hour = 41,667 writes
1 second = 12 writes
provisioned on-demand
$0.000000181 per write OR
$0.181 per million writes
price: $1.25 per million writes
price:
costs $5.42 / month costs $37.50 / month
FREE
25 WCU
46. best
practices
reduce lambda footprint
• don’t use if not needed
• be flexible on fat vs lean lambda
• avoid using for data shifting
• optimise lambda functions
47. Duration: 54908.63 ms Billed Duration: 55000 ms
Memory Size: 2048 MB Max Memory Used: 265 MB
allocated actual
VS
common memory provision mistake
49. best
practices
reduce lambda footprint
• don’t use if not needed
• be flexible on fat vs lean lambda
• avoid using for data shifting
• optimise lambda functions
• lower memory != lower cost
51. best
practices
reduce lambda footprint
• don’t use if not needed
• be flexible on fat vs lean lambda
• avoid using for data shifting
• optimise lambda functions
• lower memory != lower cost
• use async invoke and event-driven
61. best
practices
use built-in integration
• API Gateway native integration
• use HTTP topic subscription for
external service invocation
• EventBridge rules filtering & routing
75. key
takeaways
serverless isn’t always free
reduce lambda usage
use built-in integrations
expire unwanted data
monitor & optimise functions
mindful of all the environments
setup billing alerts
become familiar with the costs
