Download to read offline
![Motivation
• Works presented either app-lvl [1] or sys-
lvl [2]
• Lack of consistent comparison between
these services
• No implementation in accordance with the
SAF standard
13](https://image.slidesharecdn.com/performanceevaluationbetweencheckpointservicesinmulti-tierstateful-170105203646/85/Performance-evaluation-between-checkpoint-services-in-multi-tier-stateful-13-320.jpg)
![Discussion
• Availability Analysis in a year
• Mean Time To Recovery (MTTR) as
failover time
• Mean Time To Failure (MTTF) as Apache
Server (788.4h/year) [3]
• Assuming that the failover time is 50 times
greater
• High Availability (HA) = 99.999% (five
nines) 43](https://image.slidesharecdn.com/performanceevaluationbetweencheckpointservicesinmulti-tierstateful-170105203646/85/Performance-evaluation-between-checkpoint-services-in-multi-tier-stateful-43-320.jpg)
![References
• [1] KANSO, Ali; LEMIEUX, Yves. Achieving High Availability at
the Application Level in the Cloud. In: 2013 IEEE Sixth
International Conference on Cloud Computing. IEEE, 2013. p.
778-785.
• [2] LI, Wubin; KANSO, Ali; GHERBI, Abdelouahed. Leveraging
linux containers to achieve high availability for cloud services. In:
Cloud Engineering (IC2E), 2015 IEEE International Conference
on. IEEE, 2015. p. 76-83
• [3] MELO, R. M. D. et al. Redundant vod streaming service in a
private cloud: availability modeling and sensitivity analysis.
Mathematical Problems in Engineering, Hindawi Publishing
Corporation, v. 2014, 2014
57](https://image.slidesharecdn.com/performanceevaluationbetweencheckpointservicesinmulti-tierstateful-170105203646/85/Performance-evaluation-between-checkpoint-services-in-multi-tier-stateful-57-320.jpg)
Uploaded byDemis Gomes
Performance evaluation between checkpoint services in multi tier stateful
The document evaluates the performance of checkpoint services in multi-tier stateful applications, comparing system-level and application-level implementations. It analyzes the impact of various recovery modes on failover time, checkpoint time, and resource consumption, concluding that while system-level and application-level services are comparable in collocated warm situations, significant differences exist in non-collocated modes. Future work includes studying service scalability and resource consumption in experimental instances.
More Related Content
![[India Merge World Tour] Electric Cloud](https://cdn.slidesharecdn.com/ss_thumbnails/india-mergeworldtourelectriccloud-130819053645-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
![Perforce webinar clear-case_jb[2]](https://cdn.slidesharecdn.com/ss_thumbnails/perforce-webinarclearcasejb2-171107142005-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Performance evaluation between checkpoint services in multi tier stateful
Performance evaluation between checkpoint services in multi tier stateful
- 1. Performance Evaluation Between Checkpoint Services inMulti-tier Stateful Applications Demis Gomes Advisor: Glauco Gonçalves Co-Advisor: Patricia Endo
- 2.
- 3.
- 4. Introduction • Multi-tier statefulapplications 4
- 5. Introduction • It isimportant keep an application in a PaaS running as long as possible • A downtime causes many financial losses 5
- 6. Introduction • The averagecost of a critical application failure per hour is $500,000 to $1 million. Source: https://devops.com/2015/02/11/real-cost-downtime/ . Last access 11 out. 2016 6 Checkpoint Services!
- 7.
- 8. Background • A checkpointservice is divided into three mechanisms – Checkpoint saving – Failure detection – Failover 8
- 9. Background • Checkpoint Service 9 App ActiveStandby CheckpointService App State App State App State Failover Checkpoint Saving App Failure Detection
- 10. Background • Service AvailabilityForum (SAF) • Three different implementations: – Non-collocated – Collocated warm – Collocated hot 10
- 11.
- 12. Checkpoint Services 12 CS Application-levelCS System-level App Agent State-aware application App Agent HA-agnostic application Container Checkpoint Manager Checkpoint Manager
- 13. Motivation • Works presentedeither app-lvl [1] or sys- lvl [2] • Lack of consistent comparison between these services • No implementation in accordance with the SAF standard 13
- 14. Motivation • Carry outa performance evaluation between system and application checkpoint services, where these models follow the SAF standard and evaluate the impact of different recovery modes in time and resource consumption 14
- 15. Answer three questions •System-level ~= App-level? • Impact of changing from non-collocated to collocated? • Bottlenecks of the system-level and application-level? 15
- 16.
- 17. Application • State-aware application •A multi-tier stateful chat – Frontend: provides interface and saves user’s data – Backend: saves room messages – Database: stores information related to rooms and users 17 App Agent GET /state 200 OK
- 18. Application • State providedvia JSON (backend) 18
- 19. CS System-level • Weused well-known tools: – LXC as container – NFS as file system – rsync to transfer files between instances – CRIU to establish checkpoint and restore containers 19 CS: Checkpoint Service! :D
- 20. CS System-level • Wedid not implement collocated hot because CRIU does not allow restore in a running instance 20
- 21. CS System-level • Checkpointin non-collocated 21 App Checkpoint Manager Agent App Agent Standby Instance Active Instance Container Container
- 22. CS System-level • Checkpointin collocated warm 22 App Checkpoint Manager Agent App Agent Standby Instance Active Instance rsync Container Container
- 23. Container CS System-level • Failoverin non-collocated 23 App Checkpoint Manager Agent App Agent Standby Instance Active Instance Container
- 24. Container CS System-level • Failoverin collocated warm 24 App Checkpoint Manager Agent App Agent Standby Instance Active Instance Container rsync
- 25. CS App-level • CSat application-level was developed from scratch for this work • REST resources 25 Remember, CS: Checkpoint Service! :D GET http://{manager_ip}:{manager_port}/config RESPONSE 200 OK Content-type: application/json
- 26. CS App-level • Checkpointat Application-level 26 App Checkpoint Manager Agent App Agent Standby Instance Active Instance State-aware application Non-collocated Collocated warm Collocated hot
- 27. CS App-level • Failoverin non-collocated 27 App Checkpoint Manager Agent App Agent Standby Instance Active Instance
- 28. CS App-level • Failoverin collocated warm 28 App Checkpoint Manager Agent App Agent Standby Instance Active Instance
- 29. CS App-level • Failoverin collocated hot 29 App Checkpoint Manager Agent App Agent Standby Instance Active Instance
- 30.
- 31. Evaluation • Two evaluationswere conducted – Evaluation I: Failover time comparison – Evaluation II: Checkpoint time and resources consumption comparison 31
- 32. Evaluation 32Physical Machines: 16GB RAM, 8 cores, Gigabit Interface
- 33. Evaluation I • Methodology –Backend with 1, 5,10,15,20 and 25 MB of state sizes – Experiment Manager starts the experiment and generates a failure alert – Failover process is executed – Failover time is collected 33
- 34. Failover time –Non collocated 34 Application-level has a greater failover time The growth is linear
- 35. Failover time –Non collocated 35 We estimate the failover time with state size increasing until 100 MB App lvl would be 66% faster
- 36. Failover time –Collocated 36 Application-level collocated warm is greatly impacted with increase of state size The values of app lvl collocated hot and sys lvl collocated warm are very similar
- 37. Failover time –Collocated 37 Linear regression shows: High increase of app lvl collocated warm Slight increase on sys lvl collocated warm Constant values to collocated hot
- 38. Evaluation II • Methodology –Similarly to the previous experiment, states are saved in same state sizes – Experiment Manager triggers a checkpoint process – Checkpoint time is collected – Resources consumption are evaluated 38
- 39. Evaluation II • Methodology –Resources consumption metrics 39 Metrics Measured in Checkpoint Time s CPU Load % Memory Occupation % Network I/O Throughput Mbps Disk I/O Throughput b/s
- 40.
- 41. Evaluation II –Active Instance 41 At 25MB CPU Memory Network (I/O) Disk (W) Sys-lvl collocated warm 6,8% 9,4% 0/59,8 Mbps 1300 b/s App-lvl collocated warm 2,7% 9,1% 0/8,8 Mbps 9220 b/s App-lvl collocated hot 2,53% 9,5% 0/8,64 Mbps 8340 b/s At 25MB CPU Memory Network (I/O) Disk (W) Sys-lvl non- collocated 6% 9,1% 0/81 Mbps 1780b/s App-lvl non- collocated 2% 8,92% 0/11,6 Mbps 2410 b/s
- 42. Evaluation II –Standby Instance 42 At 25 MB CPU Memory Network (I/O) Disk (W) Sys-lvl collocated warm 1,8% 10,3% 5,1/0 Mbps 12500 b/s App-lvl collocated warm 2,5% 11,9% 8,5/8,5 Mbps 7280 b/s App-lvl collocated hot 4,1% 12,4% 8,35/8,35 Mbps 6900 b/s At 25 MB CPU Memory Network (I/O) Disk (W) Sys-lvl non- collocated 0,16% 9,8% 0/0 Mbps 800 b/s App-lvl non- collocated 0,2% 11,4% 0/0 Mbps 2600 b/s
- 43. Discussion • Availability Analysisin a year • Mean Time To Recovery (MTTR) as failover time • Mean Time To Failure (MTTF) as Apache Server (788.4h/year) [3] • Assuming that the failover time is 50 times greater • High Availability (HA) = 99.999% (five nines) 43
- 44. Discussion MTTR in 25 MB(s) MTTR in 25 MB with factor 50 (s) MTTF(s) Availability with factor 50 (%) System-level collocated warm 0.38636 19.318 2838240 99.9993 Application-level collocated warm 1.27823 63.9115 2838240 99.997 Application-level collocated hot 0.25802 12.901 2838240 99.9995 System-level non-collocated 3.5441 177.205 2838240 99.9937 Application-level non-collocated 1.38795 69.3975 2838240 99.997 44 Availability analysis (25 MB)
- 45. Discussion MTTR in 100 MB (s) MTTRin 100 MB with factor 50 (s) MTTF(s) Availability with factor 50 (%) System-level collocated warm 0.5902 29.51 2838240 99.9989 Application-level collocated warm 3.8621 193.1 2838240 99.993 Application-level collocated hot 0.2677 13.385 2838240 99.9995 System-level non-collocated 9.7999 498.995 2838240 99.9824 Application-level non-collocated 4.321 216.05 2838240 99.9923 45 Availability analysis (prediction until 100 MB)
- 46.
- 47. Conclusions Answering the questions •System-level ~= App-level? Yes! In collocated warm 47
- 48. Conclusions • Impact ofchange from non-collocated to collocated? – Failover: great decrease – Checkpoint: great increase – Resources Consumption: Similar, except of CPU and disk (greater on collocated) 48
- 49. Conclusions • Bottlenecks ofthe system-level and application-level? – App : disk, CPU in standby (hot) and development time – Sys: CPU, network and NFS 49
- 50. Conclusions • CS Application-level –Private PaaS – App with large state size and high rate of checkpoints (massive online applications) 50
- 51. Conclusions • CS System-level –PaaS with legacy applications – App with less state size and higher checkpoint intervals 51
- 52. Conclusions • PaaS BusinessModel – Non-collocated: Free plans – Collocated: Premium plans 52
- 53. Contributions • Short paperapproved with results of Experiment I, entitled: “Failover Time Evaluation Between Checkpoint Services in Multi-tier Stateful Applications” IM-2017, Exp. Session (Qualis B1) 53
- 54. Future Works As futureworks, we will study • Scalability of services • Resources consumption on Experiment Instance 54
- 55.
- 56.
- 57. References • [1] KANSO,Ali; LEMIEUX, Yves. Achieving High Availability at the Application Level in the Cloud. In: 2013 IEEE Sixth International Conference on Cloud Computing. IEEE, 2013. p. 778-785. • [2] LI, Wubin; KANSO, Ali; GHERBI, Abdelouahed. Leveraging linux containers to achieve high availability for cloud services. In: Cloud Engineering (IC2E), 2015 IEEE International Conference on. IEEE, 2015. p. 76-83 • [3] MELO, R. M. D. et al. Redundant vod streaming service in a private cloud: availability modeling and sensitivity analysis. Mathematical Problems in Engineering, Hindawi Publishing Corporation, v. 2014, 2014 57
- 58.
- 59. Agenda • Introduction • CheckpointServices • Evaluation – Experiment I – Experiment II • Conclusion and Future Works • Acknowledgments 59
- 60. Introduction • PaaS containsseveral challenges, where one is the availability of your services • Multi-tier stateful applications 60
- 61. Introduction • Many PaaSnot have a mechanism that handles failures on application • Some offers a backup but is not transparent 61
- 62. Introduction 62 Tsuru only restarts application,not saving your last state
- 63. VM x Container •VMs • Containerization 63
- 64. Objectives • General – Carryout a consistent comparison between checkpoint in system and application levels • Specifics – Develop the two modes following SAF standard – Compare the services among following metrics: • Failover time • Checkpoint time • Load generated in application 64
- 65. Application • Application generatesnew base states if – threshold defined by developer has reached – A time limit has reached 65 App 20 new messages! App 120 seconds without updates!
- 66.
- 67. CS System-level • Checkpoint/RestoreIn Userspace (CRIU) • Saves memory context • Freezes processes reading memory • Restores processes in machines with same filesystem 67
- 68.
- 69. Checkpoint Services Implementation • URLSimplemented by chat 69
- 70. Checkpoint Services • CSApplication-level 70 App Checkpoint Manager Agent App Agent Standby Instance Active Instance State-aware application Non-collocated Collocated warm Collocated hot
- 71. VM/Container Checkpoint Services • CSSystem-level 71 App Checkpoint Manager Agent App Agent Standby Instance Active Instance HA-agnostic application Non-collocated Collocated warm Collocated hot VM/Container
- 72. CS System-level • LXCmust be configured to allow CRIU make checkpoint and restore 72
- 73. Evaluation II • Methodology –Checkpoint time is presented as means with 95% Confidence Interval (CI) – Resource consumption are means with 95% CI related to active and standby instances 73
- 74. CS System-level • Checkpointprocess is established in non- collocated – saving container via CRIU and storing your memory context in a shared file system between Manager and Agent • In collocated: – saving container via CRIU and send state via rsync to all standby instances 74
- 75.
- 76. CS System-level • Failoverprocess (non-collocated) 76
- 77. CS System-level • Failoverprocess (collocated warm) 77
- 78.
- 79. CS App-level • Infailover process (non-collocated) 79
- 80. CS App-level • Infailover process (collocated warm) 80
- 81. CS App-level • Infailover process (collocated hot) 81
- 82. Evaluation I 82 • T-testbetween app collocated hot and sys collocated warm
- 83. Evaluation II 83 Network received(collocated modes)
- 84. Evaluation II 84 Network received(non-collocated)
- 85. Evaluation II 85 CPU Load(collocated modes)
- 86. Evaluation II 86 CPU Load(non-collocated)
- 87. Evaluation II 87 Memory occupation(collocated modes)
- 88. Evaluation II 88 Memory occupation(non-collocated)
- 89. Evaluation II 89 Network sent(collocated modes)
- 90. Evaluation II 90 Network sent(non-collocated)
- 91. Evaluation II 91 Disk written(collocated modes)
- 92. Evaluation II 92 Disk written(non-collocated)
- 93. Acknowledgments 93 • Family • Friends •Creators • UFRPE • Advisors (the bests) • CNPq and FACEPE
Editor's Notes
- #4 Ressaltar que um PaaS tem módulos já configurados e um provedor deve sempre manter a confiabilidade do serviço
- #5 Provides web page where users choose products. ***to users** Tentar ressaltar que o stateful é a parte mais importante em vez do multitier. Se conseguir, dizer que o multitier dificulta o checkpoint da aplicação
- #6 Lembrar da confiabilidade do PaaS provider em relação ao desenvolvedor
- #8 I said maybe...
- #10 Dar ênfase principalmente ao Checkpoint Saving e ao Failover, para que a galera não se esqueça Tentar terminar esse slide antes dos 6 min
- #11 Se der, falar a definição de disponibilidade
- #13 Passar mais rápido
- #15 Avaliar a performance, não propor SAF, não OpenSAf
- #19 Dizer que focamos no backend devido a maior troca e armazenamento de mensagens
- #21 Colocar o sys level antes do app level, dizendo que usamos ferramentas existentes como o CRIU, RSYNC e etc
- #22 Para não perder muito tempo, talvez fazer uma animação explicando o funcionamento tanto do app lvl quanto do sys lvl
- #23 Para não perder muito tempo, talvez fazer uma animação explicando o funcionamento tanto do app lvl quanto do sys lvl
- #24 Fazer animações dos failovers
- #25 Fazer animações dos failovers
- #26 Desgraçado, lembrar que foi você que fez o app-lvl
- #27 Apenas inserir as chamadas REST para diferenciar os modos
- #28 Adicionar as chamadas REST aqui tbm destacando o failover
- #29 Adicionar as chamadas REST aqui tbm destacando o failover
- #30 Adicionar as chamadas REST aqui tbm destacando o failover
- #34 Figura do cenário Explicar em topicos o que foi feito
- #36 Add os numeros em porcentagem
- #42 Tabela resumindo os resultados. Destacar os principais resultados
- #43 Tabela resumindo os resultados
- #44 Acrescentar discussões dos outros resultados
- #45 Acrescentar discussões dos outros resultados
- #46 Acrescentar discussões dos outros resultados
- #54 Faz teu nome Desenvolvimento da app Comparacão dos services Artigo
- #58 Topicalizar Template Explicar o CRIU Tabela para sumarizar Resumir títulos Verificar os subtópicos Aumentar as legendas Tabela Mencionar que os dados são compactados PaaS com sistemas legados Test t com slide de backup
- #60 Tirar
- #62 Seria massa nós pesquisarmos os PaaS existentes e colocarmos uma figura informando quais contém checkpoint. Talvez aquele doc do projeto contenha mais informações
- #63 Tomar cuidado com o “fazer checkpoint”, e sim fazer checkpoint da aplicação Fazer diferentes níveis: os que reiniciam a app quando falha e os que fazem checkpoint/failover CloudFoundry: No checkpoint/ No Restart application Tsuru: No checkpoint/ Yes restart application Deis: No checkpoint / No restart application Flynn: No checkpoint / No restart application Openshift: No checkpoint / No restart application
- #64 Passar mais rápido
- #66 Outra animação aqui seria bom
- #67 I said maybe...
- #69 Dá uma explicada no CRIU
- #71 Para não perder muito tempo, talvez fazer uma animação explicando o funcionamento tanto do app lvl quanto do sys lvl
- #72 Para não perder muito tempo, talvez fazer uma animação explicando o funcionamento tanto do app lvl quanto do sys lvl
- #75 Se der, adicionar outra animação
- #79 Deixar de backup para reduzir Usar REST e não URL
- #87 Analisar as figuras para colocar na mesma escala