Download as PDF, PPTX
Uploaded byYan Cui
SQS ingress for AWS Lambda
The document discusses the configuration and cost considerations of using AWS Lambda with SQS as an ingress source, highlighting the importance of dead letter queues and parallelism. It emphasizes that while SQS is useful, it may not be the appropriate solution for every scenario, and manual or autoscaling alternatives could be needed for unpredictable traffic. Additionally, the document references links to further resources on processing SQS with Lambda.
SQS ingress for AWS Lambda
- 1. for AWS LAMBDAAWS LAMBDA SQSingressSQS ingress
- 2.
- 4. AWS user since2009
- 7. can I hazSQS as ingress source?
- 8. DO IT we can DOIT ourselves!
- 11. retried 3 times deadletter queue configurable dead letter queue
- 12. retried 3 times deadletter queue one invocation per message configurable dead letter queue no. of pollers
- 13. retried 3 times deadletter queue one invocation per message highest parallelism! configurable dead letter queue no. of pollers
- 14. retried 3 times deadletter queue one invocation per message consider impact on downstream systems consider Lambda throttling which is per region, so other functions can get throttled too configurable dead letter queue no. of pollers
- 16. retried until success/expiry NOdead letter queue configurable dead letter queue
- 17. retried until success/expiry NOdead letter queue configurable dead letter queue
- 18. retried until success/expiry NOdead letter queue one invocation per shard configurable dead letter queue no. of pollers
- 19. retried until success/expiry NOdead letter queue one invocation per shard charged by shard hour + reqs configurable dead letter queue no. of pollers charged by reqs
- 20. retried until success/expiryconfigurable NO dead letter queue dead letter queue one invocation per shard no. of pollers charged by shard hour + reqs charged by reqs high baseline cost (1 shard) but order of magnitude cheaper at scale
- 21. 1 msg/s fora month (1K msgs) 1 x 60s x 60m x 24hr x 30days @ $0.4 per mil = $1.0368 1 x 60s x 60m x 24hr x 30days @ $0.014 per mil + 24hrs x 30days x $0.015 per hr = $10.836
- 22. 1000 msg/s fora month (1K msgs) 1000 x 60s x 60m x 24hr x 30days @ $0.4 per mil = $1036.80 1000 x 60s x 60m x 24hr x 30days @ $0.014 per mil + 24hrs x 30days x $0.015 per hr = $47.088
- 23. “none of themis the right choice for every situation”
- 24. SQS is rightchoice sometimes, no? “none of them is the right choice for every situation”
- 25.
- 26.
- 27.
- 28.
- 29. unpredictable traffic leads tounpredictable latency
- 30. unpredictable traffic leads tounpredictable latency manual scaling
- 31.
- 32.
- 34.
- 35.
- 36.
- 37. predictable latency even whentraffic is unpredictable
- 38.
- 39. unhandled exception killsloop cost
- 41.
- 42.
- 43. token last_heartbeat my_loop_1 2017-05-1720:47:32.142 my_loop_2 2017-05-17 20:47:29.923 … … passed to a recursive function at the start
- 44. every X mins lookfor dead loops
- 45. every X mins lookfor dead loops recurse start
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52. Process SQS withLambda part 1 : http://bit.ly/2rvXYdW part 2 : http://bit.ly/2pY0Grj
- 53.