Worldwide Scalable and Resilient Messaging Services by CQRS and Event Sourcin...DataWorks Summit
ChatWork is one of major business communication platforms in Japan. We keep growing up for 5+ years since our service inception. Now, we hold 110k+ of customer organizations which includes large organizations like telecom companies and the service is widely used across 200+ countries and regions.
Nowadays we have faced drastic increase of message traffic. But, unfortunately, our conventional backend was based on traditional LAMP architecture. Transforming traditional backend into highly available, scalable and resilient backend was imperative.
To achieve this, we have applied “Command Query Responsibility Segregation (CQRS) and Event Sourcing” as a heart of its architecture. The simple idea of segregation brings us independent command-side and query-side system components and it can subsequently achieve highly available, scalable and resilient systems. It is desirable property for messaging services because, for example, even if command-side was down, user can keep reading messages unless query-side was down. Event Sourcing is another key technique to enable us to build optimized systems to handle heterogeneous write/read load. This means that we can choose optimized storage platform for each side. Moreover, the event data can be the rich source for real-time analysis of user’s communication behavior. We have chosen Kafka as a command-side event storage, HBase as a query-side storage, Kafka Streams as a core library to give eventual consistency between the two sides. In application layer, Akka has been chosen as a core framework. Akka can be a good choice as an abstraction layer to build highly concurrent, distributed, resilient and message-driven application effectively. Backpressure introduced by Akka Stream can be important technology to prevent from overflow of data flows in our backend, which contributes system stability very well.
In this session, we talk about how above architecture works, how we concluded above architectural decisions on many trade-offs, what was achieved by this architecture, what was the pain points (e.g. how to guarantee eventual consistency, how to migrate systems in the real project, etc.) and several TIPS we learned for realizing our highly distributed and resilient messaging systems.
ChatWork is a business communication platform for global teams. Our four main features are enterprise-grade group chat, file sharing, task management and video chat. NTT DATA is one of biggest solution provider in Japan and providing technical support about Open Source Software and distributed computing. The project has been conducted with cooperation of ChatWork and NTT DATA.
Worldwide Scalable and Resilient Messaging Services by CQRS and Event Sourcin...DataWorks Summit
ChatWork is one of major business communication platforms in Japan. We keep growing up for 5+ years since our service inception. Now, we hold 110k+ of customer organizations which includes large organizations like telecom companies and the service is widely used across 200+ countries and regions.
Nowadays we have faced drastic increase of message traffic. But, unfortunately, our conventional backend was based on traditional LAMP architecture. Transforming traditional backend into highly available, scalable and resilient backend was imperative.
To achieve this, we have applied “Command Query Responsibility Segregation (CQRS) and Event Sourcing” as a heart of its architecture. The simple idea of segregation brings us independent command-side and query-side system components and it can subsequently achieve highly available, scalable and resilient systems. It is desirable property for messaging services because, for example, even if command-side was down, user can keep reading messages unless query-side was down. Event Sourcing is another key technique to enable us to build optimized systems to handle heterogeneous write/read load. This means that we can choose optimized storage platform for each side. Moreover, the event data can be the rich source for real-time analysis of user’s communication behavior. We have chosen Kafka as a command-side event storage, HBase as a query-side storage, Kafka Streams as a core library to give eventual consistency between the two sides. In application layer, Akka has been chosen as a core framework. Akka can be a good choice as an abstraction layer to build highly concurrent, distributed, resilient and message-driven application effectively. Backpressure introduced by Akka Stream can be important technology to prevent from overflow of data flows in our backend, which contributes system stability very well.
In this session, we talk about how above architecture works, how we concluded above architectural decisions on many trade-offs, what was achieved by this architecture, what was the pain points (e.g. how to guarantee eventual consistency, how to migrate systems in the real project, etc.) and several TIPS we learned for realizing our highly distributed and resilient messaging systems.
ChatWork is a business communication platform for global teams. Our four main features are enterprise-grade group chat, file sharing, task management and video chat. NTT DATA is one of biggest solution provider in Japan and providing technical support about Open Source Software and distributed computing. The project has been conducted with cooperation of ChatWork and NTT DATA.
新技術的導入,由0到1通常是最難的一步。尤其在大型企業內,要導入一門新技術不是那麼容易,從接觸 Windows 容器到導入,就像是從離開舒適圈一樣,一開始的路走得跌跌撞撞,鼻青臉腫。但隨著關關難過關關過的精神與毅力,一但突破了那0的臨界點,才能享受到1的技術美好。本場次分享在學習與導入 Windows 容器上的心路歷程,有高興、有期待、有失望、有憤怒,試著用不一樣的角度來分享 Windows 容器的苦與樂。
新技術的導入,由0到1通常是最難的一步。尤其在大型企業內,要導入一門新技術不是那麼容易,從接觸 Windows 容器到導入,就像是從離開舒適圈一樣,一開始的路走得跌跌撞撞,鼻青臉腫。但隨著關關難過關關過的精神與毅力,一但突破了那0的臨界點,才能享受到1的技術美好。本場次分享在學習與導入 Windows 容器上的心路歷程,有高興、有期待、有失望、有憤怒,試著用不一樣的角度來分享 Windows 容器的苦與樂。
1. ASP.NET Web API 2內建兩個強大的功能,批次處理與使用OAuth 2.0進行社交登入,本場次將會完整解析這兩個不為人知的功能。
2. ASP.NET Web API 2發行之後,很快速又提供2.1版的更新,在最新的Web API 2.1.2中提供一些很棒的新功能與改善,我會為各位完整介紹Web API 2.1.2的新功能。
3. 開發Web API服務卡卡的?本場次會介紹一些「好工具」,「好工具」能帶我們上天堂,我將深入介紹「好工具」協助各位在開發與測試的路上一路順暢。
OData只定規格,不限制實作,當然,它由微軟提出,ASP.NET Web API v1 就支援 OData,在 ASP.NET Web API v2 一路支援至 OData v3(與有限的v4)。我們談 OData 規格也談 ASP.NET Web API 實作,如何利用 OData 來擴充你的 ASP.NET Web API,讓你開發出來的 RESTFul API 能應付多變的需求,以提升加速開發(少寫一行扣,就少一隻蟲!)。