(SPOT302) Under the Covers of AWS: Core Distributed Systems Primitives That Power Our Platform | AWS re:Invent 2014

•

6 likes•5,371 views

AWS and Amazon.com operate some of the world's largest distributed systems infrastructure and applications. In our past 18 years of operating this infrastructure, we have come to realize that building such large distributed systems to meet the durability, reliability, scalability, and performance needs of AWS requires us to build our services using a few common distributed systems primitives. Examples of these primitives include a reliable method to build consensus in a distributed system, reliable and scalable key-value store, infrastructure for a transactional logging system, scalable database query layers using both NoSQL and SQL APIs, and a system for scalable and elastic compute infrastructure. In this session, we discuss some of the solutions that we employ in building these primitives and our lessons in operating these systems. We also cover the history of some of these primitives; DHTs, transactional logging, materialized views and various other deep distributed systems concepts; how their design evolved over time; and how we continue to scale them to AWS.

Technology

November 13, 2014 | Las Vegas, NV
Al Vermeulen and Swami Sivasubramanian

Trend #1: The race between Computing power
and expectations
Computing systems keep getting more capableBut… expectations are going up even faster

Trend #2: Every application is a distributed app
The number of computers is going up fastMany applications are distributedDistributed systems is not a niche field anymore

Hardware and software trends
Specialized (expensive) hardware
Built in redundancy
Simple Software
Commodity hardware
Smarter software

With commodity hardware and scale -server failures are inevitable! But by using smarter software,
we can build more robust systems
Trend #3: Commodity Hardware and Smarter Software

Cloud –Elasticity is the new normalThis results in fleets being dynamic

Addressing Distributed Computing challengesprimitives

Core Distributed Systems Primitives
Group Membership
Discovery
Metadata Store
Failure Detection
Workflows

Group Membership–
Amazon RDS Multi-AZ
Amazon ElastiCache Group –List of caches in a Memcachegroup

An example… Replica BReplica C
Writes from client AReplica AReplica DNew member in the groupShould I continue to serve reads? Should I start a new quorum? Replica EReplica F
Reads and Writes from client B
Classic Split Brain Issue in Replicated systems leading to lost writes!

Group Membership Fundamentals
Addinga new member to the group
Removinga member from the group
Discoveringwhen the group membership changes
Discoveringroles within the group

Discovery –Metadata store/consensusAmazon DynamoDB

Metadata Store -what are good characteristics?
Simplicity
Availability
Scalability
Amazon DynamoDB –top choice for metadata storage in Amazon

Workflow –What is it?
To execute a series of actions asynchronously

What is not a workflow? synchronousasynchronous

Workflow –Recommended Approach
Activity 1
Activity 2
Activity 3
Activity 4

Workflow –Lessons Learned
Idempotent metadataAmazon Simple Workflow Service

What is the underlying problem?
Group Membership
Discovery
Metadata Store
Failure Detection
Workflows

Paxos and consensus
single point
of failure
Paxos at
the bottom
broken

Group Membership
Discovery
Metadata Store
Failure Detection
Workflows
Lock
Management
Amazon Kinesis
Amazon
DynamoDB
Streams
???

Minimizing customer impact is a key feature in successfully rolling out frequent code updates. Learn how to leverage the AWS cloud so you can minimize bug impacts, test your services in isolation with canary data, and easily roll back changes. Learn to love deployments, not fear them, with a blue/green architecture model. This talk walks you through the reasons it works for us and how we set up our AWS infrastructure, including package repositories, Elastic Load Balancing load balancers, Auto Scaling groups, internal tools, and more to help orchestrate the process. Learn to view thousands of servers as resources at your command to help improve your engineering environment, take bigger risks, and not spend weekends firefighting bad deployments.

Enterprise DevOps at Scale with AWS | AWS Public Sector Summit 2016

Amazon Web Services

Ellucian has been migrating its entire organization from a myriad of software delivery mechanisms, many of them manual, to a highly automated and advanced suite of DevOps tools. Using tools such as Jenkins, Terraform, and Ansible along with native AWS tooling, we have built a highly customized DevOps pipeline on top of the AWS platform. In this talk, we go over some of the challenges we have faced and also discuss our thoughts on the evolution of DevOps and the emerging patterns of managing AWS-based environments.

What's New in Amazon RDS for Open Source and Commercial Databases

Amazon Web Services

Ceate a Scalable Cloud Architecture

Amazon Web Services

(DAT204) NoSQL? No Worries: Build Scalable Apps on AWS NoSQL Services

Amazon Web Services

In this session, we discuss the benefits of NoSQL databases and take a tour of the main NoSQL services offered by AWS—Amazon DynamoDB and Amazon ElastiCache. Then, we hear from two leading customers, Expedia and Mapbox, about their use cases and architectural challenges, and how they addressed them using AWS NoSQL services, including design patterns and best practices. You will walk out of this session having a better understanding of NoSQL and its powerful capabilities, ready to tackle your database challenges with confidence.

AWS Webcast - Intro to DevOps: Using Amazon RDS with AWS OpsWorks

Amazon Web Services

You can now use RDS with AWS OpsWorks, an application deployment and management service. OpsWorks allows you to incorporate DevOps practices to seamlessly deploy code and operate services. DevOps focuses on collaborating across development and operations facilitating continuous software delivery, shared code bases, and automated deploys, which results in faster delivery of features and more stable operating environments. OpsWorks automates deployments and standardizes production environments, a key aspect of DevOps. What this means for an RDS user is OpsWorks can scale your EC2 application servers according to demand and ensure it has the most current code base deployed. It can automatically configure and connect your new server to your RDS database. OpsWorks will ensure consistent provisioning of your resources with no manual configuring.

How to Migrate your Startup to AWS

Amazon Web Services

Deep Dive on Elastic Load Balancing

Amazon Web Services

Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances for fault tolerance and load distribution. In this session, we go into detail about Elastic Load Balancing's configuration and day-to-day management, as well as its use in conjunction with Auto Scaling. We explain how to make decisions about the service and share best practices and useful tips for success.

AWS OpsWorks for Chef Automate provides a fully managed Chef server and suite of automation tools that give you workflow automation for continuous deployment, automated testing for compliance and security, and a user interface that gives you visibility into your nodes and their status. The Chef server gives you full stack automation by handling operational tasks such as software and operating system configurations, package installations, database setups, and more. The Chef server centrally stores your configuration tasks and provides them to each node in your compute environment at any scale, from a few nodes to thousands of nodes. OpsWorks for Chef Automate is completely compatible with tooling and cookbooks from the Chef community and automatically registers new nodes with your Chef server.

Building Highly Scalable Web Applications

IWMW

Configuration Management in the Cloud - AWS Online Tech Talks

Amazon Web Services

AWS re:Invent 2016: Chalk Talk: Succeeding at Infrastructure-as-Code (GPSCT312)

Amazon Web Services

The days of manually managing infrastructure tasks are quickly coming to an end; businesses increasingly need their infrastructure teams to react with the same agility of their development teams. In this session, we discuss various approaches to infrastructure-as-code utilizing AWS solutions across the areas of templated infrastructure provisioning, configuration management, and policy as code. We invite you to bring your questions and join AWS Solutions Architects as we dive deeper into the concepts and best practices behind infrastructure-as-code.

AWS re:Invent 2016: Netflix: Using Amazon S3 as the fabric of our big data ec...

Amazon Web Services

Amazon S3 is the central data hub for Netflix's big data ecosystem. We currently have over 1.5 billion objects and 60+ PB of data stored in S3. As we ingest, transform, transport, and visualize data, we find this data naturally weaving in and out of S3. Amazon S3 provides us the flexibility to use an interoperable set of big data processing tools like Spark, Presto, Hive, and Pig. It serves as the hub for transporting data to additional data stores / engines like Teradata, Redshift, and Druid, as well as exporting data to reporting tools like Microstrategy and Tableau. Over time, we have built an ecosystem of services and tools to manage our data on S3. We have a federated metadata catalog service that keeps track of all our data. We have a set of data lifecycle management tools that expire data based on business rules and compliance. We also have a portal that allows users to see the cost and size of their data footprint. In this talk, we’ll dive into these major uses of S3, as well as many smaller cases, where S3 smoothly addresses an important data infrastructure need. We will also provide solutions and methodologies on how you can build your own S3 big data hub.

Fast, Secure Deployments with Docker on AWS

Amazon Web Services

Today, it is critical that IT teams are able to easily, consistently deploy to production. Running Docker containers on Amazon Web Services makes it possible to engineer a compliant and DevOps-friendly environment from the ground up. Spring Venture Group successfully migrated to AWS with Docker containers and leveraged Logicworks to migrate to AWS and automate infrastructure build-out and deployment. Join our webinar to learn how Spring Venture Group, an innovative insurance brokerage, reduced risk and improved deployment velocity with Logicworks, AWS, and Docker.

Design, Deploy, and Optimize SQL Server on AWS - AWS Online Tech Talks

Amazon Web Services

Enterprises are quickly moving database workloads like SQL Server to the cloud, but with so many options, the best approach isn’t always obvious. You exercise full control of your SQL Server workloads by running them on Amazon EC2 instances, or leverage Amazon RDS for a fully managed database experience. This session will go deep on best practices and considerations for running SQL Server on AWS. We will cover best practices for deploying SQL Server, how to choose between Amazon EC2 and Amazon RDS, ways to optimize the performance of your SQL Server deployment for different applications types. We review in detail how to provision and monitor your SQL Server databases, and how to manage scalability, performance, availability, security, and backup and recovery, in both Amazon RDS and Amazon EC2.

Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...

Amazon Web Services

Organizations are migrating to the cloud in order to increase their agility and eliminate undifferentiated heavy lifting. At the same time, they’re embracing DevOps principles in order to deliver functionality faster and improve operational performance. Taken together, it’s possible to deliver agile, reliable applications with less overhead than ever before. However, it’s not always optimal to emulate traditional approaches to DevOps and configuration management in the cloud. No matter where you are in your DevOps journey, join us in this session to learn how to use AWS application lifecycle management services to focus on your mission, not your tooling.

Building a CICD Pipeline for Container Deployment to Amazon ECS - May 2017 AW...

Amazon Web Services

Learning Objectives: - Learn best practices for creating a CICD pipeline for containerized workloads - Learn how to leverage the Amazon EC2 Container Registry (ECR) to host container images for version control during development and deployment - Understand best practices for scaling CICD Containers can make it easier to scale applications in the cloud. In this session, we will explore how developers can build effective CICD workflows to manage their containerized code deployments using Amazon EC2 Container Service, Amazon EC2 Container Registry, and other common tools such as Github and Jenkins. We will walk through best practices for CICD architectures used by our customers to deploy containers to AWS and we will also deep dive into topics such as creating an accessible CICD platform and Blue-Green deployments. We'll also demo how you can easily use AWS to create a CICD pipeline and deploy containers on Amazon ECS.

Accelerating Application Performance with Amazon ElastiCache (DAT207) | AWS r...

Amazon Web Services

Learn how you can use Amazon ElastiCache to easily deploy a Memcached or Redis compatible, in-memory caching system to speed up your application performance. We show you how to use Amazon ElastiCache to improve your application latency and reduce the load on your database servers. We'll also show you how to build a caching layer that is easy to manage and scale as your application grows. During this session, we go over various scenarios and use cases that can benefit by enabling caching, and discuss the features provided by Amazon ElastiCache.

Creating Your Virtual Data Center: VPC Fundamentals and Connectivity Options

Amazon Web Services

In this session, we will walk through the fundamentals of Amazon Virtual Private Cloud (VPC). First, we will cover build-out and design fundamentals for VPC, including picking your IP space, subnetting, routing, security, NAT, and much more. We will then transition into different approaches and use cases for optionally connecting your VPC to your physical data center with VPN or AWS Direct Connect. This mid-level architecture discussion is aimed at architects, network administrators, and technology decision-makers interested in understanding the building blocks AWS makes available with VPC and how you can connect this with your offices and current data center footprint.

ENT309 Scaling Up to Your First 10 Million Users

Amazon Web Services

Cloud computing gives you a number of advantages, such as the ability to scale your web application or website on demand. If you have a new web application and want to use cloud computing, you might be asking yourself, "Where do I start?" Join us in this session to understand best practices for scaling your resources from zero to millions of users. We show you how to best combine different AWS services, how to make smarter decisions for architecting your application, and how to scale your infrastructure in the cloud.

Achieve Scale & Velocity with AWS OpsWorks for Chef Automate

Amazon Web Services

Companies are frequently using outdated process and tools for delivering infrastructure and applications, and are finding it to be difficult to migrate from on-premises data centers to the cloud. AWS OpsWorks for Chef Automate is a complete solution for the automation of infrastructure, applications, and compliance across the entire technology stack. Join us to learn more how Chef Automate can help by delivering, deploying, and managing infrastructure and apps with increasing speed, improved efficiency, and decreasing risk. Join us to Learn: How to configure, deploy, and scale cloud and on-premises infrastructure simply and securely by automating infrastructure as code. How Chef can help you use configuration management to save time. Best practices of setting up your infrastructure, host configuration and applications. Who Should Attend: Cloud/Solution/Software/Enterprise Architects, DevOps Managers, Senior Technical Managers in Software Development and Operations, Site Reliability Engineers (SREs) AWS Speaker: Isaiah Weiner, Sr. Solutions Architect, AWS Partner Speaker: Sean Carolan, Partner Architect, Chef Customer Speaker: Ronald Lipke , Senior DevOps Engineer, Gannett

Running Microservices on AWS Elastic Beanstalk

Amazon Web Services

DevOps on AWS: Accelerating Software Delivery with the AWS Developer Tools

Amazon Web Services

AWS re:Invent 2016: Proactive Security Testing in AWS: From Early Implementat...

Amazon Web Services

Amazon Aurora for the Enterprise - August 2016 Monthly Webinar Series

Amazon Web Services

Relational databases are a cornerstone of the enterprise IT landscape, powering business-critical applications of many kinds. Though they have been around for a while, current commercial relational databases have lagged behind in innovation. Amazon Aurora, a managed database service built for the cloud, is intended to change that. It fulfils the high-performance, high-availability needs of business-critical applications with an emphasis on cost-effectiveness. In this session, we will look into how Aurora fits the needs of applications built and bought by enterprises to power their business. Learning Objectives: • Explore the overall architecture, capabilities, and cost-effectiveness of Aurora and see how it compares to commercial database offerings • Learn best practices for enterprises adopting Aurora for existing and new workloads, as well as strategies, tools, and techniques for migrating existing databases to Aurora

Deep Dive on Elastic Load Balancing

Amazon Web Services

Introduction to AWS X-Ray

Amazon Web Services

ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...

Amazon Web Services

As both new and established businesses work to increase their customer numbers, revenue and relevance to the market – they are working to deliver software that scales larger than ever before. The challenge of being the "victim of your own success" be it from viral marketing, social media or simply dramatic uptake of a new service; is something that troubles the minds of CIOs and Engineers alike. This session will focus on ways to avoid creating "technical debt" during initial development, and will share well established practices and approaches to building applications that can tolerate and revel in the challenges of scaling to "web scale". Working through a range of architectural dimensions, patterns and pithy examples – attendees will leave this session with useful ideas on how to design new applications, as well as the "retro-fitting" that can be done to existing applications to enable them to scale on AWS.

Bootstrapping a Solution Design in the Cloud - Session Sponsored by Ajilon

Amazon Web Services

AWS Summit 2014 Perth - Breakout 2 The cloud is an incredibly flexible resource with the potential to change the dynamics of project delivery - even for projects that will not run in production in the cloud. Learn how to manage project technical risk and control project dependencies by proving an architecture in the cloud. We will describe how we leveraged Amazon Web Services to solve a range of complicated deployment puzzles to get the approval of the client's infrastructure and security team without waiting for them or depending on them. Drawing on our experience using the cloud to architect a Microsoft Dynamics CRM solution with a fine grained security model federated across seventeen local divisions and integrated with the SAS platform this talk will help you understand the use of cloud services to emulate deployments that are complicated because of network topologies, security requirements, and platform dependencies. Presenter: Ross Kendle, National Competency Lead - Architecture & Bruce Mcdonald, Cloud Evangelist, Ajilon Australia

What's hot

Getting Started with Amazon Aurora

Amazon Web Services

Configuration Management with AWS OpsWorks for Chef Automate

Amazon Web Services

Building Highly Scalable Web Applications

IWMW

Configuration Management in the Cloud - AWS Online Tech Talks

Amazon Web Services

AWS re:Invent 2016: Chalk Talk: Succeeding at Infrastructure-as-Code (GPSCT312)

Amazon Web Services

AWS re:Invent 2016: Netflix: Using Amazon S3 as the fabric of our big data ec...

Amazon Web Services

Fast, Secure Deployments with Docker on AWS

Amazon Web Services

Design, Deploy, and Optimize SQL Server on AWS - AWS Online Tech Talks

Amazon Web Services

Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...

Amazon Web Services

Building a CICD Pipeline for Container Deployment to Amazon ECS - May 2017 AW...

Amazon Web Services

Accelerating Application Performance with Amazon ElastiCache (DAT207) | AWS r...

Amazon Web Services

Creating Your Virtual Data Center: VPC Fundamentals and Connectivity Options

Amazon Web Services

ENT309 Scaling Up to Your First 10 Million Users

Amazon Web Services

Achieve Scale & Velocity with AWS OpsWorks for Chef Automate

Amazon Web Services

Running Microservices on AWS Elastic Beanstalk

Amazon Web Services

DevOps on AWS: Accelerating Software Delivery with the AWS Developer Tools

Amazon Web Services

AWS re:Invent 2016: Proactive Security Testing in AWS: From Early Implementat...

Amazon Web Services

Amazon Aurora for the Enterprise - August 2016 Monthly Webinar Series

Amazon Web Services

Deep Dive on Elastic Load Balancing

Amazon Web Services

Introduction to AWS X-Ray

Amazon Web Services

What's hot (20)

Getting Started with Amazon Aurora

Configuration Management with AWS OpsWorks for Chef Automate

Building Highly Scalable Web Applications

Configuration Management in the Cloud - AWS Online Tech Talks

AWS re:Invent 2016: Chalk Talk: Succeeding at Infrastructure-as-Code (GPSCT312)

AWS re:Invent 2016: Netflix: Using Amazon S3 as the fabric of our big data ec...

Fast, Secure Deployments with Docker on AWS

Design, Deploy, and Optimize SQL Server on AWS - AWS Online Tech Talks

Cloud-Native DevOps: Simplifying application lifecycle management with AWS | ...

Building a CICD Pipeline for Container Deployment to Amazon ECS - May 2017 AW...

Accelerating Application Performance with Amazon ElastiCache (DAT207) | AWS r...

Creating Your Virtual Data Center: VPC Fundamentals and Connectivity Options

ENT309 Scaling Up to Your First 10 Million Users

Achieve Scale & Velocity with AWS OpsWorks for Chef Automate

Running Microservices on AWS Elastic Beanstalk

DevOps on AWS: Accelerating Software Delivery with the AWS Developer Tools

AWS re:Invent 2016: Proactive Security Testing in AWS: From Early Implementat...

Amazon Aurora for the Enterprise - August 2016 Monthly Webinar Series

Deep Dive on Elastic Load Balancing

Introduction to AWS X-Ray

Viewers also liked

ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...

Amazon Web Services

Bootstrapping a Solution Design in the Cloud - Session Sponsored by Ajilon

Amazon Web Services

Day 2 - Delivering Media Mobile Apps Using the AWS Mobile & Javascript SDKs

Amazon Web Services

Businesses around the world are running the infrastructure that supports their websites and mobile applications in the cloud to lower costs, improve time-to-market, and enable rapid scalability. Join this webinar to learn how the AWS Mobile Services and Javascript SDKs make it easy to leverage the power of AWS to provide consistent user state across devices and platforms, authenticate users via public and private login providers, and to grant controlled access to AWS services and features right from your mobile or web application. Using a simple media application we will demonstrate how you can upload, store, repurpose and deliver content with Amazon S3, Amazon CloudFront and Amazon Elastic Transcoder, make efficient use of Amazon DynamoDB, take advantage of Amazon SQS to decouple your application workflow and to send push notifications to mobile devices via Amazon SNS. Reasons to attend: - Learn how you can deliver websites and applications that share state across platforms and devices, using Amazon Elastic Beanstalk and Amazon Cognito. - Learn how to leverage the content repurposing, storage and delivery capabilities of Amazon Elastic Transcoder, Amazon S3 and Amazon CloudFront. - Learn how to use the AWS Mobile and Javascript SDKs to create applications that manage media.

News UK - Our Journey to Cloud

Amazon Web Services

(APP204) NEW LAUNCH: Introduction to AWS Service Catalog | AWS re:Invent 2014

Amazon Web Services

Running an IT department in a large organization is not easy. To provide your internal users with access to the latest and greatest technology so that they can be as efficient and as productive as possible needs to be balanced with the need to set and maintain corporate standards, collect and disseminate best practices, and provide some oversight to avoid runaway spending and technology sprawl. Introducing AWS Service Catalog, a service that allows end users in your organization to easily find and launch products using a personalized portal. You can manage catalogs of standardized offerings and control which users have access to which products, enabling compliance with business policies. Your organization can benefit from increased agility and reduced costs. Attend this session to be one of the first to learn about this new service.

APN Partner Webinar - Having Effective and Critical TCO Conversations

Amazon Web Services

Customers always want to understand how AWS cost models compare to other alternatives. Using the new AWS TCO Calculator, we will outline how AWS breaks down cost drivers when it educates customers who are evaluating cloud vs. looking at other models of computing: on-prem, virtualized, and co-lo. Discussion will also center on best practices to capture the true costs of these alternative computing approaches, and how to have meaningful customer conversations with respect to TCO. • Learn: What is TCO and why it matters • Understand: TCO evaluation Methodology used by AWS • Hear: Best practices around TCO, demonstration of online TCO calculator You can find the recording of this webinar here: http://youtu.be/BaPEf_f0N5U

Aw some day_essentials3.2ish_072214Amazon Web Services

(BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Am...

Amazon Web Services

"In this session we will present some of the key features of Glacier including security, durability and price. You will learn best practices for managing your cold data, including ingest, retrieval and security controls. We will also discuss how to optimize the storage, upload and retrieval costs and help you identify the most applicable workloads and recommend optimizations based on a few sample use cases. "

AWS Public Sector Symposium 2014 Canberra | Big Data in the Cloud: Accelerati...

Amazon Web Services

From distributed caches to in-memory data grids

Max Alexejev

Wordnet IntroductionUniversity of Technology

Introduction to Chainer: A Flexible Framework for Deep Learning

Seiya Tokui

Viewers also liked (12)

ARC205 Building Web-scale Applications Architectures with AWS - AWS re: Inven...

Bootstrapping a Solution Design in the Cloud - Session Sponsored by Ajilon

Day 2 - Delivering Media Mobile Apps Using the AWS Mobile & Javascript SDKs

News UK - Our Journey to Cloud

(APP204) NEW LAUNCH: Introduction to AWS Service Catalog | AWS re:Invent 2014

APN Partner Webinar - Having Effective and Critical TCO Conversations

Aw some day_essentials3.2ish_072214

(BAC307) The Cold Data Playbook: Building the Ultimate Archive Solution in Am...

AWS Public Sector Symposium 2014 Canberra | Big Data in the Cloud: Accelerati...

From distributed caches to in-memory data grids

Wordnet Introduction

Introduction to Chainer: A Flexible Framework for Deep Learning

Similar to (SPOT302) Under the Covers of AWS: Core Distributed Systems Primitives That Power Our Platform | AWS re:Invent 2014

Data Applications and Infrastructure at LinkedIn__HadoopSummit2010

Yahoo Developer Network

Top Technology Trends for Virtualization dallas

InnoTech

Keeping Up with the Adversary: Creating a Threat-Based Cyber Team

Priyanka Aash

Chaos Engineering Without Observability ... Is Just Chaos

Charity Majors

Beat the devil: towards a Drupal performance benchmark

Pedro González Serrano

Relational Databases - Benefits and Challenges

Elvis Saravia

Dynamo Systems - QCon SF 2012 Presentation

Shanley Kane

Amazon SageMaker for Fraud Detection

Amazon Web Services

Realtime search at Yammer

Boris Aleksandrovsky

This presentation is focused on the architecture, scalability concerns, performance bottlenecks, operational characteristics and lessons learned while designing and implementing Yammer distributed real-time search system. Yammer is an enterprise social network SaaS offering with over 100,000 networks (including 85% of the Fortune 100) and nearly 2 million users. The search system we developed scales well up to 1B messages and serves a foundation of knowledge base analysis services Yammer is developing.

Real-time Search at Yammer - By Aleksandrovsky Boris

lucenerevolution

See conference video - http://www.lucidimagination.com/devzone/events/conferences/revolution/2011 This talk will be focused on the architecture, scalability concerns, performance bottlenecks, operational characteristics and lessons learned while designing and implementing Yammer distributed real-time search system. Yammer is an enterprise social network SaaS offering with over 100,000 networks (including 85% of the Fortune 100) and nearly 2 million users. The search system we developed scales well up to 1B messages and serves a foundation of knowledge base analysis services Yammer is developing.

Real Time Search at Yammer

Lucidworks (Archived)

2019 StartIT - Boosting your performance with Blackfire

Marko Mitranić

A workshop held in StartIT as part of Catena Media learning sessions. We aim to dispel the notion that large PHP applications tend to be sluggish, resource-intensive and slow compared to what the likes of Python, Erlang or even Node can do. The issue is not with optimising PHP internals - it's the lack of proper introspection tools and getting them into our every day workflow that counts! In this workshop we will talk about our struggles with whipping PHP Applications into shape, as well as work together on some of the more interesting examples of CPU or IO drain.

Front Range PHP NoSQL Databases

Jon Meredith

http://www.hfadeel.com/Blog/?p=151xlight

Design (Cloud systems) for Failures

Rodolfo Kohn

Distributed systems involve complex interactions among many components. This increases the possibilities of failures that could turn a whole system down. Software architects, designers, and developers need to architect, design, and program functional requirements thinking about possibility of failures and the need for a system to keep running despite failures. This presentation tackles but part of the problem, focusing on redundancy, different types of groups, replication, and eventual consistency, finishing with the presentation of CAP theorem. Presentation delivered at IV Cloud Computing and Big Data Ent at Universdad Nacional de La Plata http://www.jcc.info.unlp.edu.ar/jcc2016/wordpress/index.php/cronograma/

7 Stages of Scaling Web Applications

David Mitzenmacher

UnConference for Georgia Southern Computer Science March 31, 2015

Christopher Curtin

FIM and System Call Auditing at Scale in a Large Container Deployment

Priyanka Aash

This will show how, on a large container deployment, the speaker achieved insight into security events like file events on sensitive files, system call auditing, user level activity trail, network activity, etc., by customizing and plumbing a stack of open source tools that use the underlying Linux’s inotify and kernel audit components and by aggregating these events centrally in Elasticsearch. (Source: RSA Conference USA 2018)

Handling Data in Mega Scale Systems

Directi Group

Azure and cloud design patterns

Venkatesh Narayanan

Similar to (SPOT302) Under the Covers of AWS: Core Distributed Systems Primitives That Power Our Platform | AWS re:Invent 2014 (20)

Data Applications and Infrastructure at LinkedIn__HadoopSummit2010

Top Technology Trends for Virtualization dallas

Keeping Up with the Adversary: Creating a Threat-Based Cyber Team

Chaos Engineering Without Observability ... Is Just Chaos

Beat the devil: towards a Drupal performance benchmark

Relational Databases - Benefits and Challenges

Dynamo Systems - QCon SF 2012 Presentation

Amazon SageMaker for Fraud Detection

Realtime search at Yammer

Real-time Search at Yammer - By Aleksandrovsky Boris

Real Time Search at Yammer

2019 StartIT - Boosting your performance with Blackfire

Front Range PHP NoSQL Databases

http://www.hfadeel.com/Blog/?p=151

Design (Cloud systems) for Failures

7 Stages of Scaling Web Applications

UnConference for Georgia Southern Computer Science March 31, 2015

FIM and System Call Auditing at Scale in a Large Container Deployment

Handling Data in Mega Scale Systems

Azure and cloud design patterns

More from Amazon Web Services

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Amazon Web Services

Il Forecasting è un processo importante per tantissime aziende e viene utilizzato in vari ambiti per cercare di prevedere in modo accurato la crescita e distribuzione di un prodotto, l’utilizzo delle risorse necessarie nelle linee produttive, presentazioni finanziarie e tanto altro. Amazon utilizza delle tecniche avanzate di forecasting, in parte questi servizi sono stati messi a disposizione di tutti i clienti AWS. In questa sessione illustreremo come pre-processare i dati che contengono una componente temporale e successivamente utilizzare un algoritmo che a partire dal tipo di dato analizzato produce un forecasting accurato.

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Amazon Web Services

La varietà e la quantità di dati che si crea ogni giorno accelera sempre più velocemente e rappresenta una opportunità irripetibile per innovare e creare nuove startup. Tuttavia gestire grandi quantità di dati può apparire complesso: creare cluster Big Data su larga scala sembra essere un investimento accessibile solo ad aziende consolidate. Ma l’elasticità del Cloud e, in particolare, i servizi Serverless ci permettono di rompere questi limiti. Vediamo quindi come è possibile sviluppare applicazioni Big Data rapidamente, senza preoccuparci dell’infrastruttura, ma dedicando tutte le risorse allo sviluppo delle nostre le nostre idee per creare prodotti innovativi.

Esegui pod serverless con Amazon EKS e AWS Fargate

Amazon Web Services

Ora puoi utilizzare Amazon Elastic Kubernetes Service (EKS) per eseguire pod Kubernetes su AWS Fargate, il motore di elaborazione serverless creato per container su AWS. Questo rende più semplice che mai costruire ed eseguire le tue applicazioni Kubernetes nel cloud AWS.In questa sessione presenteremo le caratteristiche principali del servizio e come distribuire la tua applicazione in pochi passaggi

Costruire Applicazioni Moderne con AWS

Amazon Web Services

Vent'anni fa Amazon ha attraversato una trasformazione radicale con l'obiettivo di aumentare il ritmo dell'innovazione. In questo periodo abbiamo imparato come cambiare il nostro approccio allo sviluppo delle applicazioni ci ha permesso di aumentare notevolmente l'agilità, la velocità di rilascio e, in definitiva, ci ha consentito di creare applicazioni più affidabili e scalabili. In questa sessione illustreremo come definiamo le applicazioni moderne e come la creazione di app moderne influisce non solo sull'architettura dell'applicazione, ma sulla struttura organizzativa, sulle pipeline di rilascio dello sviluppo e persino sul modello operativo. Descriveremo anche approcci comuni alla modernizzazione, compreso l'approccio utilizzato dalla stessa Amazon.com.

Come spendere fino al 90% in meno con i container e le istanze spot

Amazon Web Services

L’utilizzo dei container è in continua crescita. Se correttamente disegnate, le applicazioni basate su Container sono molto spesso stateless e flessibili. I servizi AWS ECS, EKS e Kubernetes su EC2 possono sfruttare le istanze Spot, portando ad un risparmio medio del 70% rispetto alle istanze On Demand. In questa sessione scopriremo insieme quali sono le caratteristiche delle istanze Spot e come possono essere utilizzate facilmente su AWS. Impareremo inoltre come Spreaker sfrutta le istanze spot per eseguire applicazioni di diverso tipo, in produzione, ad una frazione del costo on-demand!

Open banking as a service

Amazon Web Services

In recent months, many customers have been asking us the question – how to monetise Open APIs, simplify Fintech integrations and accelerate adoption of various Open Banking business models. Therefore, AWS and FinConecta would like to invite you to Open Finance marketplace presentation on October 20th. Event Agenda : Open banking so far (short recap) • PSD2, OB UK, OB Australia, OB LATAM, OB Israel Intro to Open Finance marketplace • Scope • Features • Tech overview and Demo The role of the Cloud The Future of APIs • Complying with regulation • Monetizing data / APIs • Business models • Time to market One platform for all: a Strategic approach Q&A

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

Amazon Web Services

Per creare valore e costruire una propria offerta differenziante e riconoscibile, le startup di successo sanno come combinare tecnologie consolidate con componenti innovativi creati ad hoc. AWS fornisce servizi pronti all'utilizzo e, allo stesso tempo, permette di personalizzare e creare gli elementi differenzianti della propria offerta. Concentrandoci sulle tecnologie di Machine Learning, vedremo come selezionare i servizi di intelligenza artificiale offerti da AWS e, anche attraverso una demo, come costruire modelli di Machine Learning personalizzati utilizzando SageMaker Studio.

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Amazon Web Services

Con l'approccio tradizionale al mondo IT per molti anni è stato difficile implementare tecniche di DevOps, che finora spesso hanno previsto attività manuali portando di tanto in tanto a dei downtime degli applicativi interrompendo l'operatività dell'utente. Con l'avvento del cloud, le tecniche di DevOps sono ormai a portata di tutti a basso costo per qualsiasi genere di workload, garantendo maggiore affidabilità del sistema e risultando in dei significativi miglioramenti della business continuity. AWS mette a disposizione AWS OpsWork come strumento di Configuration Management che mira ad automatizzare e semplificare la gestione e i deployment delle istanze EC2 per mezzo di workload Chef e Puppet. Scopri come sfruttare AWS OpsWork a garanzia e affidabilità del tuo applicativo installato su Instanze EC2.

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Amazon Web Services

Vuoi conoscere le opzioni per eseguire Microsoft Active Directory su AWS? Quando si spostano carichi di lavoro Microsoft in AWS, è importante considerare come distribuire Microsoft Active Directory per supportare la gestione, l'autenticazione e l'autorizzazione dei criteri di gruppo. In questa sessione, discuteremo le opzioni per la distribuzione di Microsoft Active Directory su AWS, incluso AWS Directory Service per Microsoft Active Directory e la distribuzione di Active Directory su Windows su Amazon Elastic Compute Cloud (Amazon EC2). Trattiamo argomenti quali l'integrazione del tuo ambiente Microsoft Active Directory locale nel cloud e l'utilizzo di applicazioni SaaS, come Office 365, con AWS Single Sign-On.

Computer Vision con AWS

Amazon Web Services

Database Oracle e VMware Cloud on AWS i miti da sfatare

Amazon Web Services

Amazon Web Services e VMware organizzano un evento virtuale gratuito il prossimo mercoledì 14 Ottobre dalle 12:00 alle 13:00 dedicato a VMware Cloud ™ on AWS, il servizio on demand che consente di eseguire applicazioni in ambienti cloud basati su VMware vSphere® e di accedere ad una vasta gamma di servizi AWS, sfruttando a pieno le potenzialità del cloud AWS e tutelando gli investimenti VMware esistenti. Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali.

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

Amazon Web Services

Molte aziende oggi, costruiscono applicazioni con funzionalità di tipo ledger ad esempio per verificare lo storico di accrediti o addebiti nelle transazioni bancarie o ancora per tenere traccia del flusso supply chain dei propri prodotti. Alla base di queste soluzioni ci sono i database ledger che permettono di avere un log delle transazioni trasparente, immutabile e crittograficamente verificabile, ma sono strumenti complessi e onerosi da gestire. Amazon QLDB elimina la necessità di costruire sistemi personalizzati e complessi fornendo un database ledger serverless completamente gestito. In questa sessione scopriremo come realizzare un'applicazione serverless completa che utilizzi le funzionalità di QLDB.

API moderne real-time per applicazioni mobili e web

Amazon Web Services

Con l’ascesa delle architetture di microservizi e delle ricche applicazioni mobili e Web, le API sono più importanti che mai per offrire agli utenti finali una user experience eccezionale. In questa sessione impareremo come affrontare le moderne sfide di progettazione delle API con GraphQL, un linguaggio di query API open source utilizzato da Facebook, Amazon e altro e come utilizzare AWS AppSync, un servizio GraphQL serverless gestito su AWS. Approfondiremo diversi scenari, comprendendo come AppSync può aiutare a risolvere questi casi d’uso creando API moderne con funzionalità di aggiornamento dati in tempo reale e offline. Inoltre, impareremo come Sky Italia utilizza AWS AppSync per fornire aggiornamenti sportivi in tempo reale agli utenti del proprio portale web.

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Amazon Web Services

Molte organizzazioni sfruttano i vantaggi del cloud migrando i propri carichi di lavoro Oracle e assicurandosi notevoli vantaggi in termini di agilità ed efficienza dei costi. La migrazione di questi carichi di lavoro, può creare complessità durante la modernizzazione e il refactoring delle applicazioni e a questo si possono aggiungere rischi di prestazione che possono essere introdotti quando si spostano le applicazioni dai data center locali. In queste slide, gli esperti AWS e VMware presentano semplici e pratici accorgimenti per facilitare e semplificare la migrazione dei carichi di lavoro Oracle accelerando la trasformazione verso il cloud, approfondiranno l’architettura e dimostreranno come sfruttare a pieno le potenzialità di VMware Cloud ™ on AWS.

Tools for building your MVP on AWSAmazon Web Services

How to Build a Winning Pitch DeckAmazon Web Services

Building a web application without serversAmazon Web Services

Fundraising EssentialsAmazon Web Services

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...Amazon Web Services

Introduzione a Amazon Elastic Container Service

Amazon Web Services

Amazon Elastic Container Service (Amazon ECS) è un servizio di gestione dei container altamente scalabile, che semplifica la gestione dei contenitori Docker attraverso un layer di orchestrazione per il controllo del deployment e del relativo lifecycle. In questa sessione presenteremo le principali caratteristiche del servizio, le architetture di riferimento per i differenti carichi di lavoro e i semplici passi necessari per poter velocemente migrare uno o più dei tuo container.

More from Amazon Web Services (20)

Come costruire servizi di Forecasting sfruttando algoritmi di ML e deep learn...

Big Data per le Startup: come creare applicazioni Big Data in modalità Server...

Esegui pod serverless con Amazon EKS e AWS Fargate

Costruire Applicazioni Moderne con AWS

Come spendere fino al 90% in meno con i container e le istanze spot

Open banking as a service

Rendi unica l’offerta della tua startup sul mercato con i servizi Machine Lea...

OpsWorks Configuration Management: automatizza la gestione e i deployment del...

Microsoft Active Directory su AWS per supportare i tuoi Windows Workloads

Computer Vision con AWS

Database Oracle e VMware Cloud on AWS i miti da sfatare

Crea la tua prima serverless ledger-based app con QLDB e NodeJS

API moderne real-time per applicazioni mobili e web

Database Oracle e VMware Cloud™ on AWS: i miti da sfatare

Tools for building your MVP on AWS

How to Build a Winning Pitch Deck

Building a web application without servers

Fundraising Essentials

AWS_HK_StartupDay_Building Interactive websites while automating for efficien...

Introduzione a Amazon Elastic Container Service

Recently uploaded

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

JMeter webinar - integration with InfluxDB and Grafana

RTTS

Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application. In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics. Length: 30 minutes Session Overview ------------------------------------------- During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana: - What out-of-the-box solutions are available for real-time monitoring JMeter tests? - What are the benefits of integrating InfluxDB and Grafana into the load testing stack? - Which features are provided by Grafana? - Demonstration of InfluxDB and Grafana using a practice web application To view the webinar recording, go to: https://www.rttsweb.com/jmeter-integration-webinar

When stars align: studies in data quality, knowledge graphs, and machine lear...

Elena Simperl

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Product School

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Jeffrey Haguewood

Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows. We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases. This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams. Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Ramesh Iyer

In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.

Elevating Tactical DDD Patterns Through Object Calisthenics

Dorra BARTAGUIZ

After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

Product School

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

How world-class product teams are winning in the AI era by CEO and Founder, P...

Product School

Essentials of Automations: Optimizing FME Workflows with Parameters

Safe Software

Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place. Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects. Here’s what you’ll gain: - Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows. - Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy. - Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency. - Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity. We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic. Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

Product School

PCI PIN Basics Webinar from the Controlcase Team

ControlCase

Generating a custom Ruby SDK for your web service or Rails API using Smithy

g2nightmarescribd

Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

Recently uploaded (20)

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

Accelerate your Kubernetes clusters with Varnish Caching

JMeter webinar - integration with InfluxDB and Grafana

When stars align: studies in data quality, knowledge graphs, and machine lear...

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...

Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...

Elevating Tactical DDD Patterns Through Object Calisthenics

Unsubscribed: Combat Subscription Fatigue With a Membership Mentality by Head...

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Epistemic Interaction - tuning interfaces to provide information for AI support

How world-class product teams are winning in the AI era by CEO and Founder, P...

Essentials of Automations: Optimizing FME Workflows with Parameters

AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...

PCI PIN Basics Webinar from the Controlcase Team

Generating a custom Ruby SDK for your web service or Rails API using Smithy

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

(SPOT302) Under the Covers of AWS: Core Distributed Systems Primitives That Power Our Platform | AWS re:Invent 2014

1. November 13, 2014 | Las Vegas, NV Al Vermeulen and Swami Sivasubramanian

3. Trend #1: The race between Computing power and expectations Computing systems keep getting more capableBut… expectations are going up even faster

5. Trend #2: Every application is a distributed app The number of computers is going up fastMany applications are distributedDistributed systems is not a niche field anymore

7. Hardware and software trends Specialized (expensive) hardware Built in redundancy Simple Software Commodity hardware Smarter software

8. With commodity hardware and scale -server failures are inevitable! But by using smarter software, we can build more robust systems Trend #3: Commodity Hardware and Smarter Software

10. Online Online Online Online Online

11. Online Online Online Online Online

12. Online Online Online Online Online

13. Cloud –Elasticity is the new normalThis results in fleets being dynamic

14. Our World

15. Challenges

16. Addressing Distributed Computing challengesprimitives

17. Core Distributed Systems Primitives Group Membership Discovery Metadata Store Failure Detection Workflows

18. Group Membership

19. Group Membership– Amazon RDS Multi-AZ Amazon ElastiCache Group –List of caches in a Memcachegroup

20. An example… Replica BReplica C Writes from client AReplica AReplica DNew member in the groupShould I continue to serve reads? Should I start a new quorum? Replica EReplica F Reads and Writes from client B Classic Split Brain Issue in Replicated systems leading to lost writes!

21. Group Membership Fundamentals Addinga new member to the group Removinga member from the group Discoveringwhen the group membership changes Discoveringroles within the group

22. Discovery

23. Discovery

24. Discovery –Configuration File

25. Discovery -DNS

26. Discovery –DNS (cons)

27. Discovery –Gossip Protocol

28. Cons) Discovery –Gossip Protocol (Con)

29. Discovery –Metadata store/consensusAmazon DynamoDB

30. Metadata Store

31. Metadata Store

32. Metadata Store -what are good characteristics? Simplicity Availability Scalability Amazon DynamoDB –top choice for metadata storage in Amazon

33. Metadata Store –Lessons Learned