Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (15)
Similar to Streetlife's real time analytics stack
Similar to Streetlife's real time analytics stack (20)
Recently uploaded
Recently uploaded (20)
Streetlife's real time analytics stack
- 1. ANALYTICS
- 2. • Local social network • Web, iOS and Android • 1.3M users in uk. • 1.5B events
- 8. analytics are core for our business
- 9. we track millions of events a day
- 14. Subject: Here's what your neighbours are talking about... To: user@domain.com
- 15. Subject: Here's what your neighbours are talking about... To: user@domain.com
- 16. Subject: Here's what your neighbours are talking about... To: user@domain.com
- 17. Susan P.
- 18. Susan P.
- 19. Susan P.
- 20. we currently use the batch pipeline Redshift
- 21. we gather insights once a day Redshift user_stats message_stats SQL
- 22. last_visit last_email_open 2016/01/15 12:33:54 2016/07/12 22:13:10 2016/07/10 04:52:01 2016/03/11 21:01:03 2016/07/11 05:34:45 2016/07/12 14:30:21
- 23. day impressions 2016/01/15 engagmentsid 1 u_impressions u_engagments 345 122 299 99 2016/01/152 123 99 94 84 2016/01/153 934 845 843 789 2016/01/141 899 777 744 645 2016/01/142 754 543 634 433 2016/01/143 103 99 91 77 2016/01/131 499 301 382 235 2016/01/132 1893 1400 991 1099
- 24. With this data we give businesses insights Your message got 890 impressions and 350 engagements.
- 25. We tune each message's reach dynamically 2000 target 1200 current >
- 26. It works great, but the data has several hours of lag event ingest Load SQL View S3 Load
- 27. We are going to start using the realtime pipeline realtime analytics realtime content router
- 30. The latency from events happening to us acting on them would be nearly zero event ingest Kinesis Lambda DynamoDB Load
- 31. AWS Lambdas: 20,000 feet view .py .js .java DEPENDENCIES ZIP AWS S3 λ
- 32. λ • Zero-administration compute platform • Connect Lambdas with AWS services • Kinesis, DynamoDB, APIGateway, S3, CloudWatch, Cron… • Pricing based on usage: • ~$0.0000002 per/run • ~$0.000000208 per/100ms λλ AWS Lambdas: 20,000 feet view
- 33. There is a ‘but’ Deploying/Managing them is a pitaλ
- 35. • Tool to create, wire and deploy AWS Lambdas using CloudFormation • Python/Javascript/Java/Golang/Scala runtimes… • Supported integrations • APIGateway, CloudWatch, Dynamodb, Kinesis, S3
- 37. We are hiring! Thank you! Software Engineer (data team) Software Engineer (core team) Infrastructure Engineer Data Scientist