Downloaded 42 times
Uploaded byMahesh Raj
SNS SQS SWF and Kinesis
Amazon Simple Notification Service (SNS) allows publishers to send messages to subscribers through topics. Publishers communicate asynchronously by sending messages to topics, which are communication channels. Subscribers receive notifications based on the protocol they use, such as SQS, HTTP/S, email or SMS. Amazon Simple Queue Service (SQS) offers a reliable and scalable hosted message queue. It allows applications and microservices to communicate asynchronously by sending and receiving messages. SQS provides features like dead-letter queues and poison message handling. Amazon Simple Workflow Service (SWF) allows building distributed applications as workflows. Workflows coordinate activities that can run asynchronously across devices. Activities represent tasks, and a decider determines the order and
SNS SQS SWF and Kinesis
- 1. Mahesh TR SNS, SQS,SWF & KINESIS
- 2. Mahesh TR Amazon SimpleNotification Service Amazon Simple Notification Service (Amazon SNS) is a web service that coordinates and manages the delivery or sending of messages to subscribing endpoints or clients. In Amazon SNS, there are two types of clients—publishers and subscribers—also referred to as producers and consumers. Publishers communicate asynchronously with subscribers by producing and sending a message to a topic, which is a logical access point and communication channel. Subscribers (i.e., web servers, email addresses, Amazon SQS queues, AWS Lambda functions) consume or receive the message or notification over one of the supported protocols (i.e., Amazon SQS, HTTP/S, email, SMS, Lambda) when they are subscribed to the topic.
- 3.
- 4. Mahesh TR Amazon SimpleQueue Service Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly-scalable hosted queue for storing messages as they travel between applications or microservices. It moves data between distributed application components and helps you decouple these components. Amazon SQS provides familiar middleware constructs such as dead-letter queues and poison-pill management. It also provides a generic web services API and can be accessed by any programming language that the AWS SDK supports. Amazon SQS supports both standard and FIFO queues.
- 5. Mahesh TR Consumer Read,Process and Delete Messages ✓ Visibility timeout ✓ Message Retention
- 6. Mahesh TR Amazon SWFWorkflows ✓ Using the Amazon Simple Workflow Service (Amazon SWF), you can implement distributed, asynchronous applications as workflows. Workflows coordinate and manage the execution of activities that can be run asynchronously across multiple computing devices and that can feature both sequential and parallel processing. ✓ When designing a workflow, you analyze your application to identify its component tasks. In Amazon SWF, these tasks are represented by activities. The order in which activities are performed is determined by the workflow's coordination logic. Simple way ✓ The Amazon Simple Workflow Service (Amazon SWF) makes it easy to build applications that use Amazon's cloud to coordinate work across distributed components. In Amazon SWF, a task represents a logical unit of work that is performed by a component of your workflow.
- 7. Mahesh TR Modeling YourWorkflow and Its Activities
- 8. Mahesh TR SWF Actors •Workflow Starter : An application that can initiate (start) a workflow could be an e-commerce website when placing an order • Deciders : Control the flow of activity tasks in a workflow execution. If something has finished in workflow (or fails) a Decider decided what to do next. • Activity Workers : Carry out the activity tasks.
- 9. Mahesh TR Amazon Kinesis AmazonKinesis makes it easy to collect, process, and analyze video and data streams in real time. Use Kinesis Video Streams to capture, process, and store video streams for analytics and machine learning. ✓Use Kinesis Data Streams to build custom applications that analyze data streams using popular stream processing frameworks. ✓Use Kinesis Data Firehose to load data streams into AWS data stores. ✓Use Kinesis Data Analytics to analyze data streams with SQL. Process large amount of data Redshift for business Intelligence Elastic Map Reduce for Big Data Processing.
- 10. Mahesh TR By defaultdata is stored for 24 hours, but can be increased to 7 days
- 11. Mahesh TR Stream Terminology Adata record is the unit of data stored in an Kinesis stream. Data records are composed of a sequence number, partition key, and data blob, which is an immutable sequence of bytes. Kinesis Streams does not inspect, interpret, or change the data in the blob in any way. A data blob can be up to 1 MB.
- 12. Mahesh TR Shards A shardis a uniquely identified group of data records in a stream. A stream is composed of one or more shards, each of which provides a fixed unit of capacity. Each shard can support up to 5 transactions per second for reads, up to a maximum total data read rate of 2 MB per second and up to 1,000 records per second for writes, up to a maximum total data write rate of 1 MB per second (including partition keys). If your data rate increases, you can increase or decrease the number of shards allocated to your stream. Partition Keys A partition key is used to group data by shard within a stream. The Kinesis Streams service segregates the data records belonging to a stream into multiple shards, using the partition key associated with each data record to determine which shard a given data record belongs to. Sequence Numbers Each data record has a sequence number that is unique within its shard. The sequence number is assigned by Kinesis Streams after you write to the stream with client.putRecords or client.putRecord. Sequence numbers for the same partition key generally increase over time; the longer the time period between write requests, the larger the sequence numbers become.
- 13. Mahesh TR Difference betweenSNS, SQS and Kinesis
- 14.
- 15.
- 16.
- 17.
- 18. Mahesh TR Ops Works ✓Orchestrationservice that uses Chef ✓Chef consist of recipes to maintain a consistent state ✓Look for the term “chef” or “recipes” or “cook books” and think Ops Works
- 19. Mahesh TR Elastic Transcoder ✓Mediatranscoder in the cloud ✓Convert Media files from their original source format to different formats that will play on smartphones, tablets, PC’s, etc.. ✓Provides transcoding presents for popular output formats, which means that you don’t need to guess about which settings work best on particular device ✓Pay base on the minutes that you transcode and the resolution at which you transcode.
- 20.