This document summarizes the experiences of operating Rails websites at stable scale. It discusses using virtualization to run multiple websites on a single server. It provides tips for optimizing performance including database tuning, adding indexes, log rotation, and restarting Mongrel processes periodically. New Rails sites are launched using the newest framework versions and different web servers like Passenger are tested for performance.

1. Experiences of
Rails site stabilized
operation
Hirotomo Oi
Chief Architect
Media Technology Lab., Recruit Co.,Ltd.

2. Agenda
• Our “Rails” websites
• Software
• httpd
• Server virtualization
• Tips

3. About Me
• Hirotomo Ōi
• Chief Architect of infrastructure of web.
• And “engekilife.com” tech guy.
• And...planning/developing new website.

4. About MTL
• MTL(Media Technology Lab.) is an
applied study institute to promote future
media communication, and to seek for a
sympathetic world for all of us.
• One of our focus:
Develop prototype web sites, and verify
responses and feedbacks through beta
releases.

5. MTL Websites

6. MTL Websites

7. MTL Websites

8. Software
• Httpd for Ruby Apps.
• Virtualization

9. Rails Version
• Use various versions.
• 1.2.x, 2.0.x, 2.1.x, 2.2.x, 2.3,x...
• We choose the newest version at
launch time of each site.

10. httpd
• ...Donʼt fix, now trying...
• Mongrel
• Thin
• Passenger
• Some sites were changed from
Mongrel.

11. Benchmark
Time per request(ms):
31.00
23.25
15.50
7.75
0
Mongrel Passenger

12. Ruby Version
• 1.8.6
• If apps didnʼt work, use another vers.
• Ruby Enterprise Edition
• for Passenger

13. Virtualization

14. Virtualization
+

15. Virtualize servers
• Two or more servers
can be running on one
Web,
hardware. App DB cache
(memory-
(CPU- (IO-bound)
bound)
• Generally, different bound)
resources are put
together is good
efficiency.

16. virtualize webservers
• We are virtualizing
web+app tier.
• Generally web site A
(ruby)
site B
(php)
site C
(perl)
apps are CPU-
bound.
• RoR apps need
memory.

17. How many VMs?
• Hardware
• 16GB memory
• 2x Quad Core Xeon
• VM Spec
• 1GB memory
• 10GB disk space
• 8 CPU cores
available

18. How many VMs?
• Hardware
• 16GB memory
• 2x Quad Core Xeon
• VM Spec 13
• 1GB memory VMs / Hard
• 10GB disk space
• 8 CPU cores
available

19. How many
instances?

20. How many
instances?
• 10 Mongrel instances / 1GB mem VM.
• 24 Mongrel instances / 2GB mem VM

21. Tips
• DB Tuning
• Log rotation
• Restart Mongrel

22. DB Tuning
• Developers often forget DB tuning when
launch the site.
• Developers use ActiveRecord, not
connect DB directly.

23. DB Tuning
• Developers often forget DB tuning when
launch the site.
• Developers use ActiveRecord, not
connect DB directly.
DO NOT FORGET DATABASE!!

24. Get Slow Query Log
• Step 1: know slow query.
• my.cnf
[mysqld]
log-slow-queries
long_query_time=1

25. EXPLAIN
• Step 2: Analysis of slow query
• Use “EXPLAIN”
mysql> EXPLAIN SELECT...Your Slow Query;

26. EXPLAIN
• Step 2: Analysis of slow query
• Use “EXPLAIN”
mysql> EXPLAIN SELECT...Your Slow Query;

27. Fix Slow Query
• Add Index
• Examine to change (ex:divide) the
query
• Donʼt use join...
• Donʼt use subquery...

28. ex)
mysql> show create table rubykaigi_demo
G
| rubykaigi_demo | CREATE TABLE
`rubykaigi_demo` (
`id` int(11) NOT NULL auto_increment,
`type` varchar(255) default NULL,
`bukken_id` int(11) default NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |

29. 1) Know Slow Query
mysql> SELECT * FROM `rubykaigi_demo`
WHERE `type` = 'PlayguideTicketInfoShow'
and `bukken_id` = 435;
+-------+-------------------------+-----------+
| id | type | bukken_id |
+-------+-------------------------+-----------+
| 13139 | PlayguideTicketInfoShow | 435 |
+-------+-------------------------+-----------+
1 row in set (0.01 sec)

30. 2) EXPLAIN
mysql> EXPLAIN SELECT * FROM
`rubykaigi_demo` WHERE `type` =
'PlayguideTicketInfoShow' and `bukken_id`
= 435G
id: 1
select_type: SIMPLE
table: rubykaigi_demo
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL
rows: 2299
Extra: Using where

31. 2) EXPLAIN
mysql> EXPLAIN SELECT * FROM
`rubykaigi_demo` WHERE `type` =
'PlayguideTicketInfoShow' and `bukken_id`
= 435G
id: 1
select_type: SIMPLE
table: rubykaigi_demo
type: ALL
possible_keys: NULL
key: NULL
key_len: NULL
ref: NULL Not Using Index
rows: 2299
Extra: Using where

32. 3) ADD INDEX
mysql> ALTER TABLE `rubykaigi_demo`
ADD INDEX( `type`, `bukken_id`);
Query OK, 1977 rows affected (0.18 sec)
Records: 1977 Duplicates: 0 Warnings: 0
or migration:
add_index "type_bukken", ["type",
"bukken_id"], :name => "type_bukken"

33. 4) RESULT
mysql> EXPLAIN SELECT * FROM
`rubykaigi_demo` WHERE `type` =
'PlayguideTicketInfoShow' and `bukken_id`
= 435G
id: 1
select_type: SIMPLE
table: rubykaigi_demo
type: ref
possible_keys: type
key: type
key_len: 773
ref: const,const
rows: 1
Extra: Using where; Using index

34. 4) RESULT
mysql> EXPLAIN SELECT * FROM
`rubykaigi_demo` WHERE `type` =
'PlayguideTicketInfoShow' and `bukken_id`
= 435G
id: 1
select_type: SIMPLE
table: rubykaigi_demo
type: ref
possible_keys: type
key: type
key_len: 773
ref: const,const Using Index
rows: 1
Extra: Using where; Using index

35. Log rotation
• “production.log” increase continuously.
• Itʼs necessary to rotate logs.
ex) logrotate.d conf file
(Your Rails App Log Dir)/*.log {
daily
missingok
rotate 30
compress
notifempty
copytruncate
create 0644 user group
}

36. Log rotation
• “production.log” increase continuously.
• Itʼs necessary to rotate logs.
ex) logrotate.d conf file
(Your Rails App Log Dir)/*.log {
daily
missingok
rotate 30
compress
notifempty
copytruncate Important!
create 0644 user group
}

37. Restart Mongrel
• Restart periodically for fix memory leak
of Mongrel processes.
• Using swap, not only the VM but all
VMs on same HW are slow down...

38. Restart Mongrel
• Cron
• Logrotate.d
• Monit

39. Agenda
• Our “Rails” websites
• Softwares
• httpd, virtualization,...
• Tips
• Donʼt forget DB Tuning
• Donʼt forget log rotation
• Restart Mongrel

40. ...one more thing

41. Comming Soon!!
http://mashupaward.jp/

42. Thank you
Please contact : http://mtl.recruit.co.jp/