Softwares
on Large Scale Internet Services
2013/07/18 at Tokyo Univ.
SATOSHI TAGOMORI (@tagomoris)
LINE Corp.
topics
About LINE Corp.
Service scales
Large is worse
Internet (Web) service architecture
Case studies on LINE Corp.
Q&A
TAGOMORI Satoshi (@tagomoris)
LINE corp.
tagomoris: 田籠 聡
-2005.03:
東京大学計数工学科システム情報工学コース
新研究室
2005.04-2006.03: 朝日ネット
2006.03-2010.06: NTTデータ先端技術
2010.08-:
ライブドア → N...
Service scales
small, middle, large and huge scale
Scale(1): Users, traffics, data amount, market share, ...
Scale(2): Serv...
Service scales
small middle large huge
users ∼100k ∼1m ∼10m 100m∼
servers 1, 2 or 3 ... tens handreds
thousands
or more
ex...
More/less servers
Servers
less: computing power-- / management cost--
more: computing power++ / management cost++
Server t...
Large is worse:
more traffics needs more computing power
less server types more server types
less servers
more servers
smal...
Small scale internet (web) service
server-side architecture
Users (PC, smartphone, mobilephone, ...)
Database
(RDBMS, NoSQ...
Middle scale internet (web) service
server-side architecture
Users (PC, smartphone, mobilephone, ...)
Database
(RDBMS, NoS...
Large scale internet (web) service
server-side architecture
Users (PC, smartphone, mobilephone, ...)
Database
(RDBMS, NoSQ...
Huge scale internet (web) service
server-side architecture
Users (PC, smartphone, mobilephone, ...)
Distributed datastore
...
Software engineering:
for large scale services
Performance: application, database, storage, cpu
Stability: network, high a...
Our scope in server side
Users (PC, smartphone, mobilephone, ...)
Database
(RDBMS, NoSQL)
External API
Application Server
...
Our scope in server side
Users (PC, smartphone, mobilephone, ...)
Database
(RDBMS, NoSQL)
External API
Application Server
...
BREAK.
CASE STUDIES OF
LINE CORP.
LINE secret sale:
traffic spike
Heavy traffic on sale time
transactions
remaining updates
image outbounds
SSD
Application tu...
LINE manga:
content delivery
content (image) delivery
STF
distributed object
storage
livedoor blog:
heavy traffic
STF based image storage
+ cache
CDN
MySQL database cluster
Server Side Include (SSI)
Access an...
system administration:
server setup, deployment, monitoring, ...
server setup / deployment:
automation(kickstart), testing...
Service monitoring:
Hadoop, stream processing, ...
Daily service metrics (hadoop+hive)
Pageviews, Unique users, ...
(Semi-...
micro level tunings
Log archive server
linux md raid-6 → hardware raid-1+0
I/O performance dramatically decreased
I/O sche...
Our scope in server side
Users (PC, smartphone, mobilephone, ...)
Database
(RDBMS, NoSQL)
External API
Application Server
...
Our scope of computer stack
PC/smartphone browser iOS/Android App
web servers / application servers
application code
langu...
Softwares
on large scale internet services
All of available things!
linux, lvs, haproxy, apache, nginx, squid, varnish,
me...
ALL OPTIONS
ARE AVAILABLE
IN
SERVICE PROVIDERS.
Softwares on Large Scale Internet Services
Upcoming SlideShare
Loading in …5
×

Softwares on Large Scale Internet Services

1,586 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,586
On SlideShare
0
From Embeds
0
Number of Embeds
114
Actions
Shares
0
Downloads
9
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Softwares on Large Scale Internet Services

  1. 1. Softwares on Large Scale Internet Services 2013/07/18 at Tokyo Univ. SATOSHI TAGOMORI (@tagomoris) LINE Corp.
  2. 2. topics About LINE Corp. Service scales Large is worse Internet (Web) service architecture Case studies on LINE Corp. Q&A
  3. 3. TAGOMORI Satoshi (@tagomoris) LINE corp.
  4. 4. tagomoris: 田籠 聡 -2005.03: 東京大学計数工学科システム情報工学コース 新研究室 2005.04-2006.03: 朝日ネット 2006.03-2010.06: NTTデータ先端技術 2010.08-: ライブドア → NHN Japan → LINE
  5. 5. Service scales small, middle, large and huge scale Scale(1): Users, traffics, data amount, market share, ... Scale(2): Servers, programmers, lines of code, ...
  6. 6. Service scales small middle large huge users ∼100k ∼1m ∼10m 100m∼ servers 1, 2 or 3 ... tens handreds thousands or more ex. … … livedoor blog cookpad Ameba, 楽天, ... Google, Twitter Facebook Amazon, ...
  7. 7. More/less servers Servers less: computing power-- / management cost-- more: computing power++ / management cost++ Server types (complexity) less: efficiency-- / management cost-- more: efficiency++ / management cost++
  8. 8. Large is worse: more traffics needs more computing power less server types more server types less servers more servers small middle huge large
  9. 9. Small scale internet (web) service server-side architecture Users (PC, smartphone, mobilephone, ...) Database (RDBMS, NoSQL) Web Server + Application Server (Apache + PHP, nginx + passenger(ruby), ...) External API
  10. 10. Middle scale internet (web) service server-side architecture Users (PC, smartphone, mobilephone, ...) Database (RDBMS, NoSQL) External API Application Server (Java, Perl, Ruby, ...) Caches (memory, disk) Web Server (Apache, nginx, ...)
  11. 11. Large scale internet (web) service server-side architecture Users (PC, smartphone, mobilephone, ...) Database (RDBMS, NoSQL) External API Application Server (Java, Perl, Ruby, ...) Caches (memory, disk) CDN Storage Service Web Server (Apache, nginx, ...) Data analysis
  12. 12. Huge scale internet (web) service server-side architecture Users (PC, smartphone, mobilephone, ...) Distributed datastore (GFS+BigTable, HDFS+HBase, ...) Application server farm (IaaS, PaaS) Web server cluster + CDN
  13. 13. Software engineering: for large scale services Performance: application, database, storage, cpu Stability: network, high availability, monitoring, logging Low costs: deployment, continuous testing Incomes: response time, data analysis, visualization
  14. 14. Our scope in server side Users (PC, smartphone, mobilephone, ...) Database (RDBMS, NoSQL) External API Application Server (Java, Perl, Ruby, ...) Caches (memory, disk) CDN Storage Service Web Server (Apache, nginx, ...) Data analysis
  15. 15. Our scope in server side Users (PC, smartphone, mobilephone, ...) Database (RDBMS, NoSQL) External API Application Server (Java, Perl, Ruby, ...) Caches (memory, disk) CDN Storage Service Web Server (Apache, nginx, ...) Data analysis
  16. 16. BREAK.
  17. 17. CASE STUDIES OF LINE CORP.
  18. 18. LINE secret sale: traffic spike Heavy traffic on sale time transactions remaining updates image outbounds SSD Application tuning transactions cache updates
  19. 19. LINE manga: content delivery content (image) delivery STF distributed object storage
  20. 20. livedoor blog: heavy traffic STF based image storage + cache CDN MySQL database cluster Server Side Include (SSI) Access analysis tools
  21. 21. system administration: server setup, deployment, monitoring, ... server setup / deployment: automation(kickstart), testing(serverspec) itempotent deployment(chef, puppet) transactional operation monitoring: server/switch metrics CDN API integration
  22. 22. Service monitoring: Hadoop, stream processing, ... Daily service metrics (hadoop+hive) Pageviews, Unique users, ... (Semi-)Real-time service metrics (fluentd+norikra) Req/sec, Error rate, Response time percentile, ... + system integration autoscaling, service status for end users, ...
  23. 23. micro level tunings Log archive server linux md raid-6 → hardware raid-1+0 I/O performance dramatically decreased I/O schedular: CFQ → deadline Stream processing node (ruby, fluentd) Increasing memory usage (fragmentation) Memory allocator: glibc malloc → jemalloc
  24. 24. Our scope in server side Users (PC, smartphone, mobilephone, ...) Database (RDBMS, NoSQL) External API Application Server (Java, Perl, Ruby, ...) Caches (memory, disk) CDN Storage Service Web Server (Apache, nginx, ...) Data analysis
  25. 25. Our scope of computer stack PC/smartphone browser iOS/Android App web servers / application servers application code language runtime object stores RDBMS / KVS OS server, NIC, memory, HDD/SSD, ... network, firewall, load balancing authentication(policies), audit logger / watchdogs / cron / ... HTML/CSS, Javascript ...
  26. 26. Softwares on large scale internet services All of available things! linux, lvs, haproxy, apache, nginx, squid, varnish, memcached, mysql, kyoto tycoon, mongodb, cassandra, redis, riak, perl, ruby, php, python, java, javascript, node.js, hadoop, hbase, hive, fluentd, ... except for global-scale distributed systems processor design
  27. 27. ALL OPTIONS ARE AVAILABLE IN SERVICE PROVIDERS.

×