Microservice architecture gives flexibility and ability to change components independently. At Sumo Logic we use microservices running on AWS cloud infrastructure. That allowed us to replace one microservice with more cost effiecient implementation. The talk will be a case study, how we are saving $23k / month by taking the advantage of this pattern. We were able to remove one legacy cluster and replace it with the NoSQL database as a service (Amazon DynamoDB). I'll highlight what we learned from this project and what issues we faced during such migration.

1. A D A M S Z N A J D E R
AW S C O S T
O P T I M I S AT I O N
F R O M R I G H T M I N D S E T TO
R E A L S AV I N G S
F O L L O W M E : @ A S Z N A J D E R

2. C L O U D S O F T WA R E
E N G I N E E R
S I L I C O N VA L L E Y
S TA R T U P
B I G M A C H I N E D ATA
A N A LY S I S
H U G E AW S
B I L L S
F O L L O W M E : @ A S Z N A J D E R

3. WA R S AW T E A M
C O S T S - D R I V E N
D E V E L O P M E N T
F O L L O W M E : @ A S Z N A J D E R

4. C L O U D S U P P O R T S
I N N O VAT I O N
F O L L O W M E : @ A S Z N A J D E R

5. B U R N I N G
L O T S O F $ $ $
V E RY L O W
AWA R E N E S S
M I S S I N G T E C H N I C A L
K N O W L E D G E
L O W
P R I O R I T Y
F O L L O W M E : @ A S Z N A J D E R

6. F O L L O W M E : @ A S Z N A J D E R
R I G H T
M I N D S E T
I S C R I T I C A L

7. F O L L O W M E : @ A S Z N A J D E R
R E S P O N S I B I L I T Y
D I F F I C U LT
D E C I S I O N S
G E T T I N G
S H * T D O N E

8. F O L L O W M E : @ A S Z N A J D E R
K N O W Y O U R B I L L S
T R A C K
C O S T
D ATA - D R I V E N
A P P R O A C H

9. F O L L O W M E : @ A S Z N A J D E R
A U TO M AT E
P E O P L E A R E
L A Z Y
A U TO M AT I C C O S T
G U A R D S

10. F O L L O W M E : @ A S Z N A J D E R
H O W TO C U T
C O S T S I N P R A C T I C E

11. PAY U P F R O N T
C O S T R E D U C T I O N
S AV E 3 1 - 7 6 %
L O W H A N G I N G
F R U I T
F O L L O W M E : @ A S Z N A J D E R

12. S 3
3 T Y P E S
S M A R T S TO R A G E
I N F R E Q U E N T
A C C E S S
F O L L O W M E : @ A S Z N A J D E R

13. AW S TO O L B O X
U S E R I G H T
TO O L
D B S , C A C H E S . . .
F O L L O W M E : @ A S Z N A J D E R

14. D Y N A M O D B
N O S Q L
K E Y- VA L U E
S TO R E
O N D E M A N D
C A PA C I T Y
F O L L O W M E : @ A S Z N A J D E R

15. $ 2 3 K L E S S E A C H M O N T H !

16. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
C O N S I S T E N T H A S H I N G
N O D E 2
N O D E 3
CustomerId
Timestamp
Count

17. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
K E Y I S H A S H E D
N O D E 2
N O D E 3
CustomerId
Timestamp
Count
hashValue
h(key)

18. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
F I N D I N G PA R T I T I O N
N O D E 2
N O D E 3
CustomerId
Timestamp
Count
hashValue
h(key)
N1 < <= N2

19. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
1 0 0 W C U ! = 1 0 0 W C U
N O D E 2
N O D E 3
Partition WCU = Bought WCU / Partitions Number

20. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
1 0 0 W C U ! = 1 0 0 W C U
N O D E 2
N O D E 3
Partition WCU = Bought WCU / Partitions Number
33.3 WCU = 100 WCU / 3

21. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
H O T PA R T I T I O N
N O D E 2
N O D E 3
CustomerId
Timestamp
Count
hashValue
h(key)
N1 < <= N2
100 Writes / sec
Only 33.3 WCU!

22. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
H O T PA R T I T I O N
N O D E 2
N O D E 3
CustomerId
Timestamp
Count
hashValue
h(key)
N1 < <= N2
100 Writes / sec
Increase to 100 WCU

23. N O D E 1
N O D E 2
N O D E 3
Hash Space N1-N2
Hash Space N3-N1 Hash Space N2-N3
H O T PA R T I T I O N
N O D E 2
N O D E 3
CustomerId
Timestamp
Count
hashValue
h(key)
N1 < <= N2
100 Writes / sec
Increase to 100 WCU
TOTAL COST = Hot Partition WCU * Number of partitions

24. M I S TA K E S C O S T A L O T !

25. M I S TA K E S C O S T A L O T !
Paid

26. M I S TA K E S C O S T A L O T !
Paid
Used
Paid
Used
= ~100

27. A U TO S C A L I N G
J U S T E N O U G H
R E S O U R C E S
F O L L O W M E : @ A S Z N A J D E R

28. N O D E 2
N O D E 3
R E A L E X A M P L E
Cluster CPU

29. N O D E 2
N O D E 3
TA K I N G A D VA N TA G E
SAVED $$$
Old Cluster CPU
New Cluster CPU

30. P R O B L E M S TO S O LV E
S TAT E L E S S
S E RV I C E
S C A L I N G
S T R AT E G Y
C L U S T E R
M A N A G E M E N T
F O L L O W M E : @ A S Z N A J D E R

31. S P O T I N S TA N C E S
E C 2
M A R K E T
G E N E R A L LY
C H E A P E R
S P I K E S H U R T
F O L L O W M E : @ A S Z N A J D E R

32. B AT C H P R O C E S S I N G
C H E A P
E C 2 L O S S
S AV E PA R T I A L
R E S U LT S
M A C H I N E
L E A R N I N G
F O L L O W M E : @ A S Z N A J D E R

33. P R E D I C T S P I K E S
D I F F I C U LT
B U T P O S S I B L E
U S E D W I S E LY
V E RY P O W E R F U L
F O L L O W M E : @ A S Z N A J D E R

34. A U TO S C A L E
U S I N G S P O T S
U S E R I
A S A B A S E
F O L L O W M E : @ A S Z N A J D E R
S P O T S + A U TO S C A L I N G

35. N O D E 2
N O D E 3
S P O T S + A U TO S C A L I N G
Reserved Instances
Spots

36. L E A R N H O W
D O E S Y O U R
S Y S T E M W O R K
F O L L O W M E : @ A S Z N A J D E R

37. F I N D C O N S T R A I N T S
C P U
M E M O RY
N E T W O R K
F O L L O W M E : @ A S Z N A J D E R

38. F I N D T H E R O O T S O F Y O U R
P R O B L E M S
F O L L O W M E : @ A S Z N A J D E R

39. R E A L E X A M P L E
M AT C H I N G P R O C E S S I N G
Messages
Search Expressions
Matching Messages

40. T H I S R E A L LY W O R K S !

41. T H A N K YO U
C A P TA I N
O B V I O U S
F O L L O W M E : @ A S Z N A J D E R

42. Things rarely get stuck because of lack of
time. They get stuck because the doing of
them has not been defined.
David Allen, Getting Things Done
TA K E H O M E
F O L L O W M E : @ A S Z N A J D E R

43. Q U E S T I O N S ?
L E T ' S M E E T !
F O L L O W M E : @ A S Z N A J D E R

44. ADAM SZNAJDER
adam.sznajder@gmail.com
@asznajder
asznajder.github.io

45. H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / PA S Z C Z A K 0 0 0 / 1 3 4 4 6 6 0 2 8 4 4 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / E S C H I P U L / 8 3 5 8 0 8 6 4 9 3 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / A I G L E _ D O R E / 2 4 0 6 5 8 3 9 2 9 9 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / S O L D I E R S M E D I A C E N T E R / 11 9 8 6 9 8 9 5 2 3
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / 3 1 0 2 9 8 6 5 @ N 0 6 / 1 4 8 1 0 9 0 3 2 8 0 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / R E D N U H T / 4 7 9 3 7 0 0 8 8 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / W O O D H E A D / 2 1 2 0 3 3 6 7 0 2 4
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / V I N O T H C H A N D A R / 5 7 9 3 0 5 9 5 8 0 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / A R T- S A R A H / 5 1 8 1 9 0 4 6 1 5 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / 1 3 2 0 8 4 5 2 2 @ N 0 5 / 1 7 0 8 6 5 7 0 2 1 8 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / C E N T R A L A S I A N / 3 9 8 8 7 5 8 4 4 2 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / O L A F _ S C H R E I B E R / 8 0 3 2 2 6 5 8 2 5 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / Z A P T H E D I N G B AT / 3 6 3 4 8 8 4 1 5 5
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / M A N O F TA S T E - D E / 9 4 8 3 6 0 2 8 1 7 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / F L O R I A N R I C / 7 2 6 3 3 8 2 5 5 0 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / B R U C E _ A L D R I D G E / 8 3 7 8 9 2 2 3 9 3 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / K 1 L LY R I D 0 L S / 4 5 4 7 8 4 3 6 4 4 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / F I S C H L E R / 6 1 8 1 2 9 5 8 3 8 /
H T T P S : / / W W W. F L I C K R . C O M / P H O TO S / V I L L A G E 9 9 9 1 / 7 5 1 9 4 9 6 6 4 6 /
I M A G E S S O U R C E S