Successfully reported this slideshow.
Your SlideShare is downloading. ×

Open sourcing a successful internal project - Reversim 2021

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 23 Ad

Open sourcing a successful internal project - Reversim 2021

Download to read offline

About a year ago data streams team at Wix has released to open-source its Kafka client SDK wrapper called Greyhound.

Greyhound offers rich functionality like message processing parallelisation and batching, various fault tolerant retry policies and much more.

This talk will show how the team designed Greyhound with a layered architecture to allow both public and private parts and also different levels of flexible configuration.
How it automatically syncs only relevant code from private repo to public one and also how it securely accepts public PRs back to the private repo.

Outline:
* Quick intro on what Greyhound is and its history at Wix
* Greyhound layered architecture design to allow both public and private parts and also different levels of flexible configuration.
* How it automatically syncs only relevant code from private repo to public one using Copybara tool
* how it securely accepts public PRs back to the private repo.

About a year ago data streams team at Wix has released to open-source its Kafka client SDK wrapper called Greyhound.

Greyhound offers rich functionality like message processing parallelisation and batching, various fault tolerant retry policies and much more.

This talk will show how the team designed Greyhound with a layered architecture to allow both public and private parts and also different levels of flexible configuration.
How it automatically syncs only relevant code from private repo to public one and also how it securely accepts public PRs back to the private repo.

Outline:
* Quick intro on what Greyhound is and its history at Wix
* Greyhound layered architecture design to allow both public and private parts and also different levels of flexible configuration.
* How it automatically syncs only relevant code from private repo to public one using Copybara tool
* how it securely accepts public PRs back to the private repo.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to Open sourcing a successful internal project - Reversim 2021 (20)

Advertisement

More from Natan Silnitsky (20)

Recently uploaded (20)

Advertisement

Open sourcing a successful internal project - Reversim 2021

  1. 1. Open Sourcing a Successful Internal Project Natan Silnitsky Backend Infra TL, Wix.com natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
  2. 2. To developers of the World Open Source a successful project
  3. 3. Wix developers To developers of the World Open Source a successful project
  4. 4. 1st Way To Open Source Public Private move Open Sourcing a Successful Internal Project
  5. 5. 2nd Way To Open Source Public Private sync Open Sourcing a Successful Internal Project
  6. 6. Open Sourcing a Successful Internal Project What is Greyhound? 2000
  7. 7. Greyhound Wraps Kafka Open Sourcing a Successful Internal Project Kafka Consumer Kafka Producer Kafka Broker Wix Microservices
  8. 8. Abstract so that it’s easy to change for everyone Simplify APIs, with additional features Kafka Broker Open Sourcing a Successful Internal Project Greyhound Kafka Consumer Kafka Producer Wix Microservices
  9. 9. Greyhound Kafka Consumer Kafka Producer ✔ Parallel Consumption ✔ Batch Consumer ✔ Consumer Retry Strategies ✔ Resilient Producer ✔ Context Propagation ✔ Metrics reporting Self Service ❤ Wix developers Kafka Broker Open Sourcing a Successful Internal Project
  10. 10. To developers of the World 2015 Greyhound 2019 Open Source Greyhound Self Service ❤ Rewrite!
  11. 11. Open Sourcing Greyhound A Layered Architecture ------ 2019 Open Sourcing a Successful Internal Project Allow different access to different parts
  12. 12. Kafka Producer Kafka Consumer Greyhound Kafka Broker Multiple APIs for Java, Scala and Wix devs Scala Future ZIO Java Zio Core Open Sourcing a Successful Internal Project
  13. 13. Kafka Producer Kafka Consumer Greyhound Kafka Broker Scala Future ZIO Java Zio Core Wix Interop Open Sourcing a Successful Internal Project Public Private
  14. 14. Open Sourcing Greyhound A Layered Architecture ------ 2019 Auto Sync Private → Public end of 2019 Open Sourcing a Successful Internal Project I don’t want to worry about tech debt
  15. 15. Kafka Producer Kafka Consumer Greyhound Kafka Broker Scala Future ZIO Java Zio Core Wix Interop Wix Public Github Repo Open Sourcing a Successful Internal Project Wix Private Github Repo Sync!
  16. 16. Wix Public Github Repo Open Sourcing a Successful Internal Project Wix Private Github Repo Sync! Merge PR to master How oSync Works → Transforms part of it with Copybara → Creates a (new) PR → Runs Tests > Merges
  17. 17. Open Sourcing Greyhound A Layered Architecture ------ 2019 Auto Sync Private → Public end of 2019 Accept Public PRs ------ 2020 Open Sourcing a Successful Internal Project I want to be open to contributions
  18. 18. Wix Public Github Repo Open Sourcing a Successful Internal Project Wix Private Github Repo Sync! → New PR (close original PR) → Reviewer > Merge PR opened in public → Slack alert → Reviewer > Slack approval
  19. 19. Can make frequent Changes Private (auto) sync Public developers of the World Wix developers Open Sourcing a Successful Internal Project
  20. 20. Private (auto) sync Public developers of the World Wix developers Open Sourcing a Successful Internal Project Layers & Public PR checks Contributions
  21. 21. Open Sourcing a Successful Internal Project
  22. 22. Thank You! 👉 slideshare.net/NatanSilnitsky Any questions? natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil
  23. 23. Q&A natansil.com twitter@NSilnitsky linkedin/natansilnitsky github.com/natansil 👉 slideshare.net/NatanSilnitsky Any questions?

×