Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
What is LINE LIVE?
Video streaming and hosting service
Watch streams from celebrities and companies
Host or watch streams ...
History of LINE LIVE
Feb 2014
LINE LIVE CAST was released
History of LINE LIVE
Feb 2014
LINE LIVE CAST was released
Sep 2015
Kick off meeting for LINE LIVE
History of LINE LIVE
Feb 2014
LINE LIVE CAST was released
Sep 2015
Kick off meeting for LINE LIVE
Dec 2015
LINE LIVE was r...
History of LINE LIVE
Feb 2014
LINE LIVE CAST was released
Sep 2015
Kick off meeting for LINE LIVE
Dec 2015
LINE LIVE was r...
Tools and Staff
Java, Nginx, MySQL, Redis, Elasticsearch, Fluentd, CDN, etc.
Server (media 260+, chat 110+, API and other ...
LINE LIVE Architecture
OBS
Inhouse Storage
System
Hosting
Viewing
RTMP Media
Servers
API
Servers
LINE
Billing
CMS
Chat
Web...
Relationship with LINE
Data Analysis
For details, see the presentation,
“New Stream Processing Platform with Apache Flink"
Data Labs 荻林裕憲
Real-ti...
Video Stream Delivery
HLS
(HTTP Live Streaming)
HLS
live-1_001.ts
live-1_002.ts
live-1_003.ts
live-1_004.ts
live-1_005.ts
(HTTP Live Streaming)
HLS
All compiled into
one video
live-1_001.ts
live-1_002.ts
live-1_003.ts
live-1_004.ts
live-1_005.ts
(HTTP Live Streaming)
HLS
live-1.m3u8
・live-1_001.ts
・live-1_002.ts
・live-1_003.ts
・live-1_004.ts
・live-1_005.ts
...
live-1_001.ts
live-1_002.ts...
HLS
live-1.m3u8
・live-1_001.ts
・live-1_002.ts
・live-1_003.ts
・live-1_004.ts
・live-1_005.ts
...
live-1_001.ts
live-1_002.ts...
HLS
live-1_720.m3u8
・live-1_720_001.ts
・live-1_720_002.ts
・live-1_720_003.ts
...
live-1.m3u8
・live-1_720.m3u8
・live-1_480....
HLS
live-1_720.m3u8
・live-1_720_001.ts
・live-1_720_002.ts
・live-1_720_003.ts
...
live-1.m3u8
・live-1_720.m3u8
・live-1_480....
Architecture
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
RTMP
Architecture
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
RTMP
Upload HLS Files
Architecture
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
Send Events
On JSON API
RTMP
...
Architecture
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
Send Events
On JSON API
RTMP
...
Architecture
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
Send Events
On JSON API
Cache...
HLS Good Hack
on LINE LIVE
HLS Good Hack for Data LIVE
http://〜/{encrypt(objectId, startAt, durationSec)}_720_xxx.ts
http://〜/{encrypt(objectId, star...
HLS Good Hack for Data LIVE
GET 200 GET 200
18:00 21:0019:00 20:00
http://〜/FooBar.m3u8
http://〜/FooBar_720_xxx.ts
HLS Good Hack for Data LIVE
Example : startAt=19:00
durationSec=3600
GET 404 GET 404 GET 404 GET 404
18:00 21:0019:00 20:0...
HLS Good Hack for Auto play
http://〜/{encrypt(objectId, startAt, durationSec)}.m3u8
http://〜/FooBar_720_xxx.ts
Prepare enc...
HLS Good Hack for Auto play
Example : startAt=2sec
durationSec=6
001.ts 002.ts 003.ts 004.ts 005.ts xxx.ts …
GET 200 GET G...
Flow When Stream Is Terminated
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
RTMP
Upload...
Stream Terminated
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
RTMP
Upload HLS Files
Stream Terminated
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
2. Cache
RTMP
Upload HLS...
Stream Terminated
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
3. 404 4. 404
RTMP
Uploa...
Standby Until Recovery
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
3. 404 4. 404
RTMP
...
Recovery Flow
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
1. RTMP
restart
Upload HLS F...
Recovery Flow
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
1. RTMP
restart
Upload HLS F...
Recovery Flow
Media
Servers
API
Servers
OBS
Inhouse Storage System
CDN
Akamai
Hosting Viewing
Cache 5. Download
HLS Files
...
Improved Hosting Performance
GPUCPU
720p 1080p
220%
267%
Chat
Chat Overview
CMS
Hosting Viewing
WebSocket
CHAT
API
Servers
JSON API
JSON API
JSON API
WebSocket
Chat Overview
Server 1 Server 2 Server 3
Room 1 Room 1
Redis Cluster
Publish/Subscribe
Chat Overview
Server 1 Server 2 Server 3
Room 1 Room 1
Room 2 Room 2
Redis Cluster
Room 2
Log Archive
live-1.chatlog
・live-1_001.json
・live-1_002.json
・live-1_003.json
・live-1_004.json
・live-1_005.json
...
live-1...
Issues
Issues
Over 10,000 messages per minute
LOVE
“Love-mashing”
is fun
"Love-mashing”
is communication
Love
Clients
Store
(Increment)
Flush
Burst
API
Redis
Cluster
MySQL
Clients
- ranking (ZSET)
- Total count (NUM)
Data
(log)...
“Love” Total Ranking
Get
API Redis
Cluster
Clients
Using ZREVRANGE
“Love” Total Ranking
Get
(Archive)
API Redis
Cluster
Clients
MySQL
Using ZREVRANGE
Example 1
11 million Loves in 120 minutes
Example 2
2.3 million Loves in 30 minutes
Chat and “Love” Issues Are
Also Explained on our Blog
http://developers.linecorp.com/blog/ja/?p=3653
http://developers.lin...
Summary
Provided overview of LINE LIVE
Talked about video streaming methods
Talked about implementation of traffic-invitin...
Any Questions?
A 7 architecture sustaining line live
A 7 architecture sustaining line live
A 7 architecture sustaining line live
Upcoming SlideShare
Loading in …5
×

A 7 architecture sustaining line live

11,416 views

Published on

TBD

Published in: Technology
  • Be the first to comment

A 7 architecture sustaining line live

  1. 1. What is LINE LIVE? Video streaming and hosting service Watch streams from celebrities and companies Host or watch streams from other users Chat communication Showing appreciation with “Loves”
  2. 2. History of LINE LIVE Feb 2014 LINE LIVE CAST was released
  3. 3. History of LINE LIVE Feb 2014 LINE LIVE CAST was released Sep 2015 Kick off meeting for LINE LIVE
  4. 4. History of LINE LIVE Feb 2014 LINE LIVE CAST was released Sep 2015 Kick off meeting for LINE LIVE Dec 2015 LINE LIVE was released
  5. 5. History of LINE LIVE Feb 2014 LINE LIVE CAST was released Sep 2015 Kick off meeting for LINE LIVE Dec 2015 LINE LIVE was released Aug 2016 LINE LIVE Version 2 is out
  6. 6. Tools and Staff Java, Nginx, MySQL, Redis, Elasticsearch, Fluentd, CDN, etc. Server (media 260+, chat 110+, API and other 100+) Over 20 core engineers Over 30 other related engineers (LINE app, storage, network, etc.)
  7. 7. LINE LIVE Architecture OBS Inhouse Storage System Hosting Viewing RTMP Media Servers API Servers LINE Billing CMS Chat WebSocket JSON API CDN Akamai LINE ServersUpload HLS Files Cache Send Operation On JSON API Send Events On JSON API
  8. 8. Relationship with LINE
  9. 9. Data Analysis For details, see the presentation, “New Stream Processing Platform with Apache Flink" Data Labs 荻林裕憲 Real-time viewer aggregation Streaming program KPI measurement, etc.
  10. 10. Video Stream Delivery
  11. 11. HLS (HTTP Live Streaming)
  12. 12. HLS live-1_001.ts live-1_002.ts live-1_003.ts live-1_004.ts live-1_005.ts (HTTP Live Streaming)
  13. 13. HLS All compiled into one video live-1_001.ts live-1_002.ts live-1_003.ts live-1_004.ts live-1_005.ts (HTTP Live Streaming)
  14. 14. HLS live-1.m3u8 ・live-1_001.ts ・live-1_002.ts ・live-1_003.ts ・live-1_004.ts ・live-1_005.ts ... live-1_001.ts live-1_002.ts live-1_003.ts live-1_004.ts live-1_005.ts (HTTP Live Streaming)
  15. 15. HLS live-1.m3u8 ・live-1_001.ts ・live-1_002.ts ・live-1_003.ts ・live-1_004.ts ・live-1_005.ts ... live-1_001.ts live-1_002.ts live-1_003.ts live-1_004.ts live-1_005.ts (HTTP Live Streaming) CDN
  16. 16. HLS live-1_720.m3u8 ・live-1_720_001.ts ・live-1_720_002.ts ・live-1_720_003.ts ... live-1.m3u8 ・live-1_720.m3u8 ・live-1_480.m3u8 ・live-1_360.m3u8 ... live-1_480.m3u8 ・live-1_480_001.ts ・live-1_480_002.ts ・live-1_480_003.ts ... live-1_360.m3u8 ・live-1_360_001.ts ・live-1_360_002.ts ・live-1_360_003.ts ... live-1_720_001.ts live-1_720_002.ts live-1_720_003.ts live-1_480_001.ts live-1_480_002.ts live-1_480_003.ts live-1_360_001.ts live-1_360_002.ts live-1_360_003.ts
  17. 17. HLS live-1_720.m3u8 ・live-1_720_001.ts ・live-1_720_002.ts ・live-1_720_003.ts ... live-1.m3u8 ・live-1_720.m3u8 ・live-1_480.m3u8 ・live-1_360.m3u8 ... live-1_480.m3u8 ・live-1_480_001.ts ・live-1_480_002.ts ・live-1_480_003.ts ... live-1_360.m3u8 ・live-1_360_001.ts ・live-1_360_002.ts ・live-1_360_003.ts ... live-1_720_001.ts live-1_720_002.ts live-1_720_003.ts live-1_480_001.ts live-1_480_002.ts live-1_480_003.ts live-1_360_001.ts live-1_360_002.ts live-1_360_003.ts
  18. 18. Architecture Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing RTMP
  19. 19. Architecture Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing RTMP Upload HLS Files
  20. 20. Architecture Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing Send Events On JSON API RTMP Upload HLS Files
  21. 21. Architecture Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing Send Events On JSON API RTMP Send a Push Upload HLS Files
  22. 22. Architecture Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing Send Events On JSON API Cache Download HLS Files RTMP JSON API Upload HLS Files
  23. 23. HLS Good Hack on LINE LIVE
  24. 24. HLS Good Hack for Data LIVE http://〜/{encrypt(objectId, startAt, durationSec)}_720_xxx.ts http://〜/{encrypt(objectId, startAt, durationSec)}.m3u8 http://〜/FooBar_720_xxx.ts http://〜/FooBar.m3u8 Example : startAt=19:00 durationSec=3600 Prepare encrypted URL on server in advance
  25. 25. HLS Good Hack for Data LIVE GET 200 GET 200 18:00 21:0019:00 20:00 http://〜/FooBar.m3u8 http://〜/FooBar_720_xxx.ts
  26. 26. HLS Good Hack for Data LIVE Example : startAt=19:00 durationSec=3600 GET 404 GET 404 GET 404 GET 404 18:00 21:0019:00 20:00 http://〜/FooBar.m3u8 http://〜/FooBar_720_xxx.ts
  27. 27. HLS Good Hack for Auto play http://〜/{encrypt(objectId, startAt, durationSec)}.m3u8 http://〜/FooBar_720_xxx.ts Prepare encrypted URL on server in advance Example : startAt=2sec durationSec=6 001.ts 002.ts 003.ts 004.ts 005.ts xxx.ts …
  28. 28. HLS Good Hack for Auto play Example : startAt=2sec durationSec=6 001.ts 002.ts 003.ts 004.ts 005.ts xxx.ts … GET 200 GET GET GET http://〜/AcmeAcme.m3u8 foo-bar.m3u8 ・002.ts ・003.ts ・004.ts
  29. 29. Flow When Stream Is Terminated Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing RTMP Upload HLS Files
  30. 30. Stream Terminated Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing RTMP Upload HLS Files
  31. 31. Stream Terminated Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing 2. Cache RTMP Upload HLS Files 1. GET HLS2. Fetch Origin
  32. 32. Stream Terminated Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing 3. 404 4. 404 RTMP Upload HLS Files 1. GET HLS2. Fetch Origin
  33. 33. Standby Until Recovery Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing 3. 404 4. 404 RTMP Upload HLS Files Burst API Server 6. SUSPEND 5. Polling SUSPEND Event 1. GET HLS2. Fetch Origin
  34. 34. Recovery Flow Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing 1. RTMP restart Upload HLS Files Burst API Server 2. RESTART event
  35. 35. Recovery Flow Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing 1. RTMP restart Upload HLS Files Burst API Server 4. RESTART 3. Polling 2. RESTART event
  36. 36. Recovery Flow Media Servers API Servers OBS Inhouse Storage System CDN Akamai Hosting Viewing Cache 5. Download HLS Files 1. RTMP restart Upload HLS Files Burst API Server 4. RESTART 3. Polling 2. RESTART event
  37. 37. Improved Hosting Performance GPUCPU 720p 1080p 220% 267%
  38. 38. Chat
  39. 39. Chat Overview CMS Hosting Viewing WebSocket CHAT API Servers JSON API JSON API JSON API WebSocket
  40. 40. Chat Overview Server 1 Server 2 Server 3 Room 1 Room 1 Redis Cluster Publish/Subscribe
  41. 41. Chat Overview Server 1 Server 2 Server 3 Room 1 Room 1 Room 2 Room 2 Redis Cluster Room 2
  42. 42. Log Archive live-1.chatlog ・live-1_001.json ・live-1_002.json ・live-1_003.json ・live-1_004.json ・live-1_005.json ... live-1_001.json live-1_002.json live-1_003.json live-1_004.json live-1_005.json Upload to the OBS
  43. 43. Issues
  44. 44. Issues Over 10,000 messages per minute
  45. 45. LOVE “Love-mashing” is fun "Love-mashing” is communication
  46. 46. Love Clients Store (Increment) Flush Burst API Redis Cluster MySQL Clients - ranking (ZSET) - Total count (NUM) Data (log) Using Lua Script
  47. 47. “Love” Total Ranking Get API Redis Cluster Clients Using ZREVRANGE
  48. 48. “Love” Total Ranking Get (Archive) API Redis Cluster Clients MySQL Using ZREVRANGE
  49. 49. Example 1 11 million Loves in 120 minutes
  50. 50. Example 2 2.3 million Loves in 30 minutes
  51. 51. Chat and “Love” Issues Are Also Explained on our Blog http://developers.linecorp.com/blog/ja/?p=3653 http://developers.linecorp.com/blog/ja/?p=3771
  52. 52. Summary Provided overview of LINE LIVE Talked about video streaming methods Talked about implementation of traffic-inviting chat and “Love” features
  53. 53. Any Questions?

×