Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Handle your Lambdas:
From event-based processing to Continuous
Integration
Sergii Khomenko, Data Scientist
sergii.khomenko...
Sergii Khomenko
2
Data scientist at one of the biggest fashion communities, Stylight.
Data analysis and visualisation hobb...
Profitable Leads
Stylight provides its
partners with high-
quality leads enabling
partner shops to
leverage Stylight as a
...
Stylight – acting on a global scale
Experienced & Ambitious Team
Innovative cross-
functional organisation
with flat hierarchy builds a 

unique team spirit.
...
6
A Brief History of Computing
7
8
9
Charles Babbage's machine - 1832
10
1940 – 1956: First Generation – Vacuum Tubes
111956 – 1963: Second Generation – Transistors
121964 – 1971: Third Generation – Integrated Circuits
131972- … : Fourth Generation – Microprocessors
14
15
Immutable infrastructure
16
Infrastructure as Code
17
N e e d t o u p g r a d e ? N o p r o b l e m .
B u i l d a n e w , u p g r a d e d s y s t e m a n d
t h r o w t h e o...
18
19
20
21
CloudFormation
22
Terraform
23
24
Terraform
Kubernetes and Docker {Swarm, Compose}
Serverless architecture
25
26
27
28
29
30
31
Installation
32
export GOPATH=$HOME/go
export PATH=$PATH:$GOPATH/bin
go get github.com/apex/apex/cmd/apex
export AWS_REGIO...
33
34
35
36
main.go
37
main.py
Python dependencies
38
pip install requests -t functions/dependency
39
apex invoke
Outcome
40
• all Lambdas in one place with version control
• integration tests with real events
• proper CI/CD setup
41
CircleCI deployments
42
CircleCI deployments
43
CircleCI deployments
44
Cloud functions
Use-case
CloudWatch alerting
45
46
47
48
Use-case of
outlier detection
49
50
51
custom
unification
pipeline
Departments
Business
Intelligence
internal processes variety of event types
and structures
52
The Future is Now
53
54
AWS Lambda in FRA
55
AWS Lambda in FRA
www.stylight.com
sergii.khomenko@stylight.com
@lc0d3r
Related links
57
1. The Babbage Engine
2. The Five Generations of Computers
3. IBM System 360
4. https://github.com/cloudt...
58
USE CODE - AWS20
Upcoming SlideShare
Loading in …5
×

Handle your Lambdas - From event-based processing to Continuous Integration / Munich AWS User Group - Mar 17, 2016

918 views

Published on

Handle your Lambdas - From event-based processing to Continuous Integration / Munich AWS User Group - Apr Mar, 2016

Published in: Data & Analytics
  • Be the first to comment

Handle your Lambdas - From event-based processing to Continuous Integration / Munich AWS User Group - Mar 17, 2016

  1. 1. Handle your Lambdas: From event-based processing to Continuous Integration Sergii Khomenko, Data Scientist sergii.khomenko@stylight.com, @lc0d3r Munich AWS User Group - Mar 17 2016
  2. 2. Sergii Khomenko 2 Data scientist at one of the biggest fashion communities, Stylight. Data analysis and visualisation hobbyist, working on problems not only in working time but in free time for fun and personal data visualisations. Speaker at Berlin Buzzwords 2014, ApacheCon Europe 2014, Puppet Camp London 2015, Berlin Buzzwords 2015 , Tableau Conference on Tour 2015, Budapest BI Forum 2015, Crunchsconf 2015, FOSDEM 2016, PyData Amsterdam 2016
  3. 3. Profitable Leads Stylight provides its partners with high- quality leads enabling partner shops to leverage Stylight as a ROI positive traffic channel. Inspiration Stylight offers shoppable inspiration that makes it easy to know what to buy and how to style it. Branding & Reach Stylight offers a unique opportunity for brands to reach an audience that is actively looking for style online. Shopping Stylight helps users search and shop fashion and lifestyle products smarter across hundreds of shops. 3 Stylight – Make Style Happen Core Target Group Stylight help aspiring women between 18 and 35 to evolve their style through shoppable inspiration.
  4. 4. Stylight – acting on a global scale
  5. 5. Experienced & Ambitious Team Innovative cross- functional organisation with flat hierarchy builds a 
 unique team spirit. • +200 employees • 40 Engineers/PhDs • 28 years average age • 63% female • 23 nationalities • 0 suits 5
  6. 6. 6
  7. 7. A Brief History of Computing 7
  8. 8. 8
  9. 9. 9 Charles Babbage's machine - 1832
  10. 10. 10 1940 – 1956: First Generation – Vacuum Tubes
  11. 11. 111956 – 1963: Second Generation – Transistors
  12. 12. 121964 – 1971: Third Generation – Integrated Circuits
  13. 13. 131972- … : Fourth Generation – Microprocessors
  14. 14. 14
  15. 15. 15
  16. 16. Immutable infrastructure 16 Infrastructure as Code
  17. 17. 17 N e e d t o u p g r a d e ? N o p r o b l e m . B u i l d a n e w , u p g r a d e d s y s t e m a n d t h r o w t h e o l d o n e a w a y . N e w a p p r e v i s i o n ? S a m e t h i n g . B u i l d a s e r v e r ( o r i m a g e ) w i t h a n e w r e v i s i o n a n d t h r o w a w a y t h e o l d o n e s .
  18. 18. 18
  19. 19. 19
  20. 20. 20
  21. 21. 21 CloudFormation
  22. 22. 22 Terraform
  23. 23. 23
  24. 24. 24 Terraform Kubernetes and Docker {Swarm, Compose}
  25. 25. Serverless architecture 25
  26. 26. 26
  27. 27. 27
  28. 28. 28
  29. 29. 29
  30. 30. 30
  31. 31. 31
  32. 32. Installation 32 export GOPATH=$HOME/go export PATH=$PATH:$GOPATH/bin go get github.com/apex/apex/cmd/apex export AWS_REGION=eu-west-1
  33. 33. 33
  34. 34. 34
  35. 35. 35
  36. 36. 36 main.go
  37. 37. 37 main.py
  38. 38. Python dependencies 38 pip install requests -t functions/dependency
  39. 39. 39 apex invoke
  40. 40. Outcome 40 • all Lambdas in one place with version control • integration tests with real events • proper CI/CD setup
  41. 41. 41 CircleCI deployments
  42. 42. 42 CircleCI deployments
  43. 43. 43 CircleCI deployments
  44. 44. 44 Cloud functions
  45. 45. Use-case CloudWatch alerting 45
  46. 46. 46
  47. 47. 47
  48. 48. 48
  49. 49. Use-case of outlier detection 49
  50. 50. 50
  51. 51. 51 custom unification pipeline Departments Business Intelligence internal processes variety of event types and structures
  52. 52. 52
  53. 53. The Future is Now 53
  54. 54. 54 AWS Lambda in FRA
  55. 55. 55 AWS Lambda in FRA
  56. 56. www.stylight.com sergii.khomenko@stylight.com @lc0d3r
  57. 57. Related links 57 1. The Babbage Engine 2. The Five Generations of Computers 3. IBM System 360 4. https://github.com/cloudtools/troposphere 5. https://www.terraform.io/ 6. (Docker Compose + Docker Swarm) or Kubernetes 7. Google Cloud Functions 8. https://github.com/apex/apex 9. https://github.com/apex/apex/tree/master/_examples/python/functions/dependency 10.Streaming Data Processing with Amazon Kinesis and AWS Lambda
  58. 58. 58 USE CODE - AWS20

×