Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Grammarly 

engineering
Who am I?
Aleksey Yashchenko
Software Engineer at Grammarly
@tuxslayer
The Company
Ukrainian startup
60 engineers
Offices in Kyiv and San-Francisco
The Product
Grammar
Spell
Style
Plagiarism

… checker
4.5M weekly active users on Chrome and growing
100k words/sec
400 instances on AWS
10M+ WebSocket connections/day
The Prod...
Dozens of services in 

Java, Lisp, Scala and Python
6 UI applications in 

JS, TypeScript and F#
The Product
Our Apps
Funnels on www.grammarly.com
Browser Plugins: Chrome, Safari, FF
Web App
M$ Office Add-in
Desktop Editor
Extension =>
<= Synonym cards
Mistake cards =>
Low development pace
Buggy releases =>
regular outages =>
high downtime
The Problem
…but why?
Horizontal and service teams
&
Too much manual work
The Problem
The Problem
Backend
QAFrontEnd
Ops
silos, tasks ping-pong, no DRIs
Horizontal
teams
The Problem
Buggy releases,
regular outages, 

high downtime
Slow
development
Manual work
Our Solution
Organization Technical
Our Solution
Engineers at
Cross-functional Teams
Fully own services and releases, while
Tools solve all the hassle
Cross-func Teams
QA => SE in Test
QA Engineers => 

Feature Teams
Solution /1
Continuos Delivery
Monitoring
ORG: crossfit T...
Solution /1
Feature Cross-functional Teams
no-QA: From QA to SET
Catch bugs
Build for quality
Write code
… well… catch bugs
CI/CD is easy…
Feature branches
CI Server (TeamCity)
Unit / Integration / Load / … tests
Continuous Deployment
CI/CD is easy…
Feature branches
CI Server (TeamCity)
Unit / Integration / Load …tests
Continuous Deployment
Doesn’t work
Monitor all the things!
Monitor all the things!
Make it an engineering practice
Measure everything, choose what to
monitor then
Validate monitorin...
Monitor all the things!
Logs (servers, FE, access logs) =>
Kafka (queuing) =>
Logstash (processing) =>
ElasticSearch (stor...
Monitor all the things!
Metrics (timers, counters, gauges ) =>
StatsD (aggregation) =>
Graphite (storage, querying) =>
Gra...
Monitor all the things!
Logs & Metrics (storage) =>
Sensu Client (monitoring, alerting) =>
OpsGeniue (night calls)
More: w...
Results /1
Zero ping-pong time between teams
Developers can release “at will”
QA-people are involved from day 1
Everybody ...
Results /1
Time to release from 1-2h to 1-10 min
Rollback instead of fixing => 

Short outages
Whole team does not get dist...
Release Calendar
Questions /1
?
Platform Team
Embedded Ops
in Teams
Solution /2
docker
rocker
rocker-compose
ORG: no-Ops TECH: containers
Platform Team
Not a “service” team
PaaS
Strategic rather than tactical
Software Engineers + DevOps
Embedded Ops for Featur...
Solution /2
The Funnel
Huge project
Multiple releases per day
Static site generator
1k lines of nginx configs
Static content
Nginx confi...
The Funnel
Static content
Nginx configs
Nginx
+
Dev Team Ops Kitchen
blah-blah
Chef
Container Platform
The Funnel
Static content
Nginx configs
Nginx
Responsible Dev Team Happy Ops Kitchen
ship it!
Container
...
Docker Rocks… almost
Custom tools:
rocker (building)
rocker-compose (deployment)
More: https://goo.gl/wXUbHa
Results /2
Wait no more for DevOps tasks
Reproducible environments
Safe deployments
Configuration as a code (versioning, au...
A Victory
Engineers at
Cross-functional Teams
Fully own services and releases, while
Tools solve all the hassle
Take aways
Tech should support org
Org should enforce [hi-]tech
Responsibility matters
Automate everything
Questions?
More Info
Rocker: Breaking the limits of Dockerfile
How We Deploy Containers at Grammarly
Frontend Monitoring at Grammarly
Thank you!
Aleksey Yashchenko
Software Engineer at Grammarly
@tuxslayer
Построение продукта и организации в Grammarly: no-QA & Continuous Delivery
Upcoming SlideShare
Loading in …5
×

Построение продукта и организации в Grammarly: no-QA & Continuous Delivery

203 views

Published on

Алексей Ященко, Engineering Manager Grammarly

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Построение продукта и организации в Grammarly: no-QA & Continuous Delivery

  1. 1. Grammarly 
 engineering
  2. 2. Who am I? Aleksey Yashchenko Software Engineer at Grammarly @tuxslayer
  3. 3. The Company Ukrainian startup 60 engineers Offices in Kyiv and San-Francisco
  4. 4. The Product Grammar Spell Style Plagiarism
 … checker
  5. 5. 4.5M weekly active users on Chrome and growing 100k words/sec 400 instances on AWS 10M+ WebSocket connections/day The Product
  6. 6. Dozens of services in 
 Java, Lisp, Scala and Python 6 UI applications in 
 JS, TypeScript and F# The Product
  7. 7. Our Apps Funnels on www.grammarly.com Browser Plugins: Chrome, Safari, FF Web App M$ Office Add-in Desktop Editor
  8. 8. Extension =>
  9. 9. <= Synonym cards Mistake cards =>
  10. 10. Low development pace Buggy releases => regular outages => high downtime The Problem …but why?
  11. 11. Horizontal and service teams & Too much manual work The Problem
  12. 12. The Problem Backend QAFrontEnd Ops silos, tasks ping-pong, no DRIs
  13. 13. Horizontal teams The Problem Buggy releases, regular outages, 
 high downtime Slow development Manual work
  14. 14. Our Solution Organization Technical
  15. 15. Our Solution Engineers at Cross-functional Teams Fully own services and releases, while Tools solve all the hassle
  16. 16. Cross-func Teams QA => SE in Test QA Engineers => 
 Feature Teams Solution /1 Continuos Delivery Monitoring ORG: crossfit TECH: CI
  17. 17. Solution /1 Feature Cross-functional Teams
  18. 18. no-QA: From QA to SET Catch bugs Build for quality Write code … well… catch bugs
  19. 19. CI/CD is easy… Feature branches CI Server (TeamCity) Unit / Integration / Load / … tests Continuous Deployment
  20. 20. CI/CD is easy… Feature branches CI Server (TeamCity) Unit / Integration / Load …tests Continuous Deployment Doesn’t work
  21. 21. Monitor all the things!
  22. 22. Monitor all the things! Make it an engineering practice Measure everything, choose what to monitor then Validate monitoring data!
  23. 23. Monitor all the things! Logs (servers, FE, access logs) => Kafka (queuing) => Logstash (processing) => ElasticSearch (storage/querying) => Kibana (UI)
  24. 24. Monitor all the things! Metrics (timers, counters, gauges ) => StatsD (aggregation) => Graphite (storage, querying) => Graphana (UI)
  25. 25. Monitor all the things! Logs & Metrics (storage) => Sensu Client (monitoring, alerting) => OpsGeniue (night calls) More: www.slideshare.net/tuxSlayer/frontend-monitoring-grammarly
  26. 26. Results /1 Zero ping-pong time between teams Developers can release “at will” QA-people are involved from day 1 Everybody writes tests Everybody* can fix a bug * almost
  27. 27. Results /1 Time to release from 1-2h to 1-10 min Rollback instead of fixing => 
 Short outages Whole team does not get distracted by accidents
  28. 28. Release Calendar
  29. 29. Questions /1 ?
  30. 30. Platform Team Embedded Ops in Teams Solution /2 docker rocker rocker-compose ORG: no-Ops TECH: containers
  31. 31. Platform Team Not a “service” team PaaS Strategic rather than tactical Software Engineers + DevOps Embedded Ops for Feature Teams
  32. 32. Solution /2
  33. 33. The Funnel Huge project Multiple releases per day Static site generator 1k lines of nginx configs Static content Nginx configs Nginx + +
  34. 34. The Funnel Static content Nginx configs Nginx + Dev Team Ops Kitchen blah-blah Chef
  35. 35. Container Platform The Funnel Static content Nginx configs Nginx Responsible Dev Team Happy Ops Kitchen ship it! Container Container Container …
  36. 36. Docker Rocks… almost Custom tools: rocker (building) rocker-compose (deployment) More: https://goo.gl/wXUbHa
  37. 37. Results /2 Wait no more for DevOps tasks Reproducible environments Safe deployments Configuration as a code (versioning, automation, templating etc) Multi-service deployments (e.g. DB+Redis+App+Nginx) … and…
  38. 38. A Victory Engineers at Cross-functional Teams Fully own services and releases, while Tools solve all the hassle
  39. 39. Take aways Tech should support org Org should enforce [hi-]tech Responsibility matters Automate everything
  40. 40. Questions?
  41. 41. More Info Rocker: Breaking the limits of Dockerfile How We Deploy Containers at Grammarly Frontend Monitoring at Grammarly
  42. 42. Thank you! Aleksey Yashchenko Software Engineer at Grammarly @tuxslayer

×