Zabbix䛸JobScheduler㐃ᦠ 
䛻䜘䜛㐠⏝䝅䝇䝔䝮OSS໬䛾ᐇ⌧ 
ࠥHyClops JobMonitoring䛾䛤⤂௓ࠥ 
TIS ᰴᘧ఍♫ 
ᡓ␎ᢏ⾡䝉䞁䝍䞊 
ụ⏣䚷኱㍜ 
2014/9/5 OSC2014.Enterprise@Osaka
⮬ᕫ⤂௓ 
ụ⏣䚷኱㍜ 
TISᰴᘧ఍♫ ᡓ␎ᢏ⾡䝉䞁䝍䞊 
@ike_dai 
https://www.facebook.com/ikedai 
Zabbix,AWS,fluentd,Chef,JobScheduler... 
ྡ๓ 
ᡤᒓ 
Twitter 
Facebook 
⯆࿡ 
AWS䚸VMware┘ど⏝䝥䝷䜾䜲䞁䛄HyClops for Zabbix䛅㛤Ⓨ 
ᢏ⾡ホㄽ♫HP : http://gihyo.jp/book/2014/978-4-7741-6288-1 (http://tech-sketch.github.io/hyclops/jp/) 
Amazon : http://www.amazon.co.jp/dp/4774162884
TIS OSS䝃䝫䞊䝖䝃䞊䝡䝇 
OSS䛾฼⏝䜢䝖䞊䝍䝹䛻䝞䝑䜽䜰䝑䝥 
ಖᏲ䝃䝫䞊䝖 
䝁䞁䝃䝹䝔䜱䞁䜾 䝬䜲䜾䝺䞊䝅䝵䞁
TIS OSS䝃䝫䞊䝖䝃䞊䝡䝇 
ᑐ㇟OSS 
アプリケーション 
稼働基盤 
運用基盤 
インフラ基盤 
䈜௒ᚋ䝃䝫䞊䝖ᑐ㇟ணᐃ
TIS OSS䝃䝫䞊䝖䝃䞊䝡䝇 
ᑐ㇟OSS 
アプリケーション 
稼働基盤 
運用基盤 
インフラ基盤 
䈜௒ᚋ䝃䝫䞊䝖ᑐ㇟ணᐃ 
䈜TIS䛿Zabbix♫䛾ㄆᐃ䝟䞊䝖䝘䞊䛷䛩䚹
㐠⏝䜢ྲྀ䜚ᕳ䛟≧ἣ䛾ኚ໬ 
䝅䝇䝔䝮䛾௬᝿໬䞉䜽䝷䜴䝗໬䛜ຍ㏿ 
⟶⌮ᑐ㇟ᶵჾ䛾኱つᶍ໬ ⟶⌮ᑐ㇟ᶵჾ䛾ືⓗ䛺ኚ໬ 
ᡭື㐠⏝䛷䜹䝞䞊䛷䛝䜛䝅䝇䝔䝮䛜ῶᑡ 
“㐠⏝⮬ື໬”䛜┈䚻ồ䜑䜙䜜䜛≧ἣ䛷 
䜘䜚Ᏻᐃ䛧䛯䝅䝇䝔䝮䜢ప౯᱁䛷㎿㏿䛻ᥦ౪
㐠⏝䝅䝇䝔䝮䛻 
OSS䜢ྲྀ䜚ධ䜜䜛䛣䛸䛾䝯䝸䝑䝖 
Ᏻᐃ 
ά⏝஦౛➼䜒䜸䞊䝥䞁䛻䛥䜜䜛䛣䛸䛜ከ䛟䚸 
Ᏻᐃ㐠⏝䛻ྥ䛡䛯ຠᯝⓗ䛺฼⏝䝜䜴䝝䜴䛾ඹ᭷䛜ྍ⬟ 
฼⏝䛾㝿䛾䝷䜲䝉䞁䝇㈝䜢పῶ 
(≉䛻㐠⏝⟶⌮ᑐ㇟ᶵჾ䛾ᩘ䛻ᑐ䛧䛶ᙉ䜏䛜ฟ䛶䛟䜛) 
䜸䞊䝥䞁䛰䛛䜙䛣䛭㛤Ⓨ䜒άⓎ 
㐠⏝⮬ື໬䛻ྥ䛡䛯ᵝ䚻䛺ᶵ⬟䜢䝒䞊䝹⮬㌟䛻㏣ຍᐇ⿦ 
䝃䞊䝗䝟䞊䝔䜱〇䛸䛧䛶䝋䝸䝳䞊䝅䝵䞁ᥦ౪䛥䜜䜛䛺䛹άⓎ 
ప౯᱁ 
㎿㏿
䝆䝵䝤⟶⌮䛸┘ど䛾䛒䜚᪉ 
䝆䝵䝤⟶⌮䛻ồ䜑䜙䜜䜛䛣䛸䠛 
䝆䝵䝤䛜䜚䛺䛟䝇䜿䝆䝳䞊䝹㏻䜚䛻ṇᖖᐇ⾜䛥䜜䜛䛣䛸 
● 䝆䝵䝤ᐇ⾜䛜᫬㛫㏻䜚䛻ᐇ⾜䛥䜜䜛 
● 䝆䝵䝤䛜ṇᖖ⤊஢䛧䛶䛔䜛 
௒ᚋᐇ⾜䛥䜜䜛䝆䝵䝤䝇䜿䝆䝳䞊䝹䛜☜ᐇ䛻ไᚚ䛥䜜䜛䛣䛸 
● ௒ᚋ䛒䜛䝆䝵䝤䜢ᐇ⾜䛩䜛㝿䛻䝃䞊䝞䝸䝋䞊䝇䛜㊊䜚䜛䛛 
● ௒ᚋ䛒䜛䝆䝵䝤䜢ᐇ⾜䛩䜛㝿䛻☜ᐇ䛻ணᐃ㏻䜚᏶஢䛷䛝䜛䛛 
“┘ど”䛜㔜せ
╔┠䛧䛯䛾䛿㧗ᶵ⬟&ᣑᙇᛶ䛾㧗䛥 
⤫ྜ┘どOSS 䝆䝵䝤⟶⌮OSS
OSS⤫ྜ┘ど䝋䝣䝖䜴䜵䜰Zabbix 
OSS⤫ྜ┘ど䝒䞊䝹 (GPL ver.2) 
䝷䝖䝡䜰䛾ZabbixSIA♫䛜㛤Ⓨ 
2014/9/5᫬Ⅼ䛾᭱᪂Ᏻᐃ∧2.2.6 (2014/8/27䝸䝸䞊䝇) 
┘どᑐ㇟ᶵჾ 
- Linux 
- Windows 
- NWᶵჾ 
➼ 
┘ど᪉ἲ 
- Zabbix Agent┘ど 
- SNMP┘ど 
- IPMI┘ど 
- 䜶䞊䝆䜵䞁䝖䝺䝇┘ど 
➼ 
┘どෆᐜ 
- 䝸䝋䞊䝇┘ど 
- Ṛά┘ど 
- ICMPPing 
- 䝥䝻䝉䝇 
- Web┘ど 
- 䝻䜾┘ど 
- HW┘ど 
- SQL┘ど 
➼
Zabbix䛿┘どᡭἲ䛜㇏ᐩ 
snmp server 
snmp server 
௵ព䛾᝟ሗ䜢pushᆺ䛷Zabbix䛻Ⓩ㘓 
SNMP Trap 
SNMP 
ssh server 
ssh server 
Zabbix Agent 
௬᝿໬HV 
Cloud 
ODBC 
DB 
JMX 
Java 
䜰䝥䝸 
௬᝿⎔ቃAPI 
䜽䝷䜴䝗API 
Zabbix Sender 
Zabbix Server 
SSH᥋⥆䛻䜘䜛 
ෆ㒊䝁䝬䞁䝗ᐇ⾜┘ど 
ZabbixAgent 
䛻䜘䜛ෆ㒊┘ど 
(䝸䝋䞊䝇䜔䝻䜾➼) 
ODBC䛻䜘䜛DB┘ど 
Zabbix Java Gateway䛻䜘䜛Java┘ど 
እ㒊䝏䜵䝑䜽䝇䜽䝸䝥䝖┘どᶵ⬟䛻䜘䜛 API┘ど
Zabbix䛿⮬ື໬ᶵ⬟䛜඘ᐇ 
┘どタᐃ䝟䝍䞊䞁໬ ┘どタᐃ⮬ື໬ 㐠⏝⮬ື໬ 
䝔䞁䝥䝺䞊䝖ᶵ⬟ ZabbixAgent⮬ືⓏ㘓ᶵ⬟ 
䝻䞊䝺䝧䝹䝕䜱䝇䜹䝞䝸ᶵ⬟ 
䜰䜽䝅䝵䞁ᶵ⬟ 
┘どタᐃ䜢䝃䞊䝞䛾⏝㏵ẖ䚸 
✀ูẖ➼䜎䛸䜑䜛ᶵ⬟ 
Agent㉳ື᫬䛻⮬㌟䛾᝟ሗ䜢 
Zabbix Server䛻⮬ື㐃ᦠ 
┘どᑐ㇟䝃䞊䝞ෆ䛾᝟ሗ䜢཰㞟䛧䚸 
䛭䛾ෆᐜ䛻ྜ䜟䛫䛶⮬ືタᐃ 
᮲௳䜢タᐃ䛧䛶䛒䜙䛛䛨䜑Ⓩ㘓䛧 
䛯㐠⏝సᴗฎ⌮ᐇ⾜䝁䝬䞁䝗䜢 
⮬ືᐇ⾜ 
Zabbix APIᶵ⬟ 
䝥䝻䜾䝷䝮䛛䜙 Zabbix䛾タᐃ䜔 
┘ど⤖ᯝ᝟ሗྲྀᚓ䜢ᐇ⌧
ཧ⪃᝟ሗ: 䝻䞊䝺䝧䝹䝕䜱䝇䜹䝞䝸 
Zabbix Server 
䝕䜱䝇䜹䝞䝸䝹䞊䝹 
/ /data 
ᐃᮇⓗ䛻᥈⣴ 
䜰䜲䝔䝮䝥䝻䝖䝍䜲䝥 
䜰䜲䝔䝮 
vfs.fs.size[{#FSNAME},free] 
{ 
"data":[ 
䚷䚷 { 
䚷䚷 "{#FSNAME}":"/", 
䚷 "{#FSTYPE}":"ext3"}, 
䚷 { 
䚷 "{#FSNAME}":"/data", 
䚷 "{#FSTYPE}":"ext3"}, 
䚷 䞉䞉䞉┬␎ 
} 
䜰䜲䝔䝮 
vfs.fs.size[/,free] 
vfs.fs.size[/data,free] 
ά⏝౛䠖 
JobScheduler䛻Ⓩ㘓䛥䜜䛶䛔䜛Job୍ぴ᝟ሗ䛻ᇶ䛵䛔䛶䜰䜲䝔䝮⮬ືⓏ㘓䛸䛛
ཧ⪃᝟ሗ: ZabbixAPI 
■ 䛷䛝䜛䛣䛸 
䚷- Zabbix䛻ᑐ䛩䜛᧯స(タᐃኚ᭦䚸䝩䝇䝖㏣ຍ䚸䜾䝹䞊䝥సᡂ➼)ᐇ᪋ 
䚷- Zabbix䛷⟶⌮䛧䛶䛔䜛┘ど⤖ᯝ䛾ྲྀᚓ(history,event,IT service᝟ሗ) 
■ API䛾ᙧᘧ: JSON-RPC 
䚷- URL: http://<zabbix-server>/zabbix/api_jsonrpc.php 
䚷- 䝯䝋䝑䝗: host.create䚸host.update䚸item.get䚸history.get䛺䛹 
䚷- 䝟䝷䝯䞊䝍: 䝯䝋䝑䝗ẖ䛻ᚲせ䛺䝟䝷䝯䞊䝍䜢Ώ䛩 
host.get䛾౛ 
䝺䝇䝫䞁䝇 
{ 
"auth": "xxxxxxx", 
"method": "host.get", 
"params": { 
"output": "extend", 
"filter": {"host":"Zabbix server"} 
}, 
"jsonrpc": "2.0", 
"id": 1 
} 
{ 
"jsonrpc":"2.0", 
"result":[{ 
"maintenances":[], 
"hostid":"10084", 
"host":"Zabbix server", 
䞉䞉䞉␎ 
}], 
"id":1 
} 
䝸䜽䜶䝇䝖䝟䝷䝯䞊䝍
OSS䝆䝵䝤⟶⌮䝋䝣䝖䜴䜵䜰JobScheduler 
OSS䝆䝵䝤⟶⌮䝒䞊䝹 (GPL ver.2) 
䝗䜲䝒 䝧䝹䝸䞁䛾SOS♫䛜㛤Ⓨ 
2014/9/5᫬Ⅼ䛾᭱᪂Ᏻᐃ∧1.7.4241 (2014/8/29䝸䝸䞊䝇) 
ᑐᛂ䝥䝷䝑䝖䝣䜷䞊䝮 
- Linux 
- Windows 
- Solaris 
- AIX 
- HP-UX 
䝆䝵䝤ᐇ⾜᪉ἲ 
- JS༢య䛷䛾䝆䝵䝤ᐇ⾜ 
- JS Agent䛻䜘䜛 
䝸䝰䞊䝖䝃䞊䝞䜈䛾䝆䝵䝤ᐇ⾜ 
- SSH᥋⥆䛻䜘䜛 
䝸䝰䞊䝖䝃䞊䝞䜈䛾䝆䝵䝤ᐇ⾜ 
ᑐᛂྍ⬟ฎ⌮ 
- 䝣䝻䞊(JobChain)ᐇ⾜ 
- 䝇䜿䝆䝳䞊䝹ᐇ⾜ 
- ከ㔜ᐇ⾜䞉᤼௚ไᚚ 
- ඛ⾜䞉ᚋ⥆ไᚚ 
- ඃඛᗘタᐃ 
- 䝆䝵䝤ᐇ⾜┘ど 
- 䝯䞊䝹㏻▱ 
➼ 
䈜 JS = JobScheduler 
䈜 JS Agent = JobScheduler Agent
JobScheduler䛾ᇶᮏᵓᡂ 
JOC 
JOE 
䈜Job䜶䝕䜱䝍 
Job Job 
Chain 
JobScheduler Agent 
JobScheduler Agent 
SSH Server 
䈜Job⟶⌮WebGUI 
Job Job 
Chain 
DB 
䈜Jobᐇ⾜ᒚṔ䚸 
䚷Job䝇䜿䝆䝳䞊䝸䞁䜾᝟ሗ 
䚷➼ 
Schedule Process 
Class 
Schedule Process 
Class 
JobScheduler Engine
JobScheduler䛾Job䛾⪃䛘᪉ 
Standalone Job 
Job Chain 
Order Job 
Order Job 
Schedule 
• Standalone Job 
– ༢య䛷✌ື䛩䜛䝆䝵䝤䜢ᐃ⩏ 
• Job Chain 
– 」ᩘ䝆䝵䝤䜢㡰ᗎ㛵ಀ䜢⪃៖䛧䛶 
⤌䜏ྜ䜟䛫䜛䝆䝵䝤(order job)䜢ᐃ⩏ 
• Schedule 
– 䝆䝵䝤䛾✌ືணᐃ䜢ᐃ⩏ 
• 䝆䝵䝤ᐃ⩏䛿ᵝ䚻䛺グ㏙䛜ྍ⬟ 
– ShellScript, Windows bat,Java, 
JavaScript,VBScript, javax.script, 
Perl 
JobScheduler䛾≉ᚩ 
● 䝆䝵䝤㛵㐃䛾඲䛶䛾ᐃ⩏䛜XML䝣䜯䜲䝹䛷⟶⌮ 
● ᵝ䚻䛺᧯స䜢⡆༢䛻⾜䛘䜛䜘䛖䛻䛩䜛ᶵ⬟䛜ᥦ౪䛥䜜䛶䛔䜛 
○ 䝣䜯䜲䝹᧯స䚸䝣䜯䜲䝹㌿㏦䚸DBไᚚ䚸SQL᧯స䚸䝺䝫䞊䝔䜱䞁䜾ᶵ⬟䚷䛺䛹䛺䛹
JobScheduler༢య䛰䛸䛷䛝䛺䛔䛣䛸 
● 䝆䝵䝤䛾ኻᩋ᫬䛾㏻▱䛿༢⣧䛺䝯䞊䝹㏦ಙ䛾䜏 
(ኻᩋ᫬䛾ฎ⌮䜢䝆䝵䝤䛸䛧䛶ᐃ⩏䛩䜜䜀௵ព䛾ฎ⌮䛿ྍ⬟䛰䛜) 
● JobScheduler䛻䛿䝴䞊䝄⟶⌮ᶵ⬟䛜䛺䛔 
● 䝆䝵䝤䛾ᐇ⾜⤖ᯝ䛿䝔䜻䝇䝖䛸䛧䛶ಖᏑ䛥䜜䛶䛔䜛䛾䜏 
(⾲♧䛿䝔䜻䝇䝖䝧䞊䝇䛾୍ぴ⾲♧) 
● ┘どᶵ⬟䛿䛺䛔䛾䛷䝆䝵䝤ᐇ⾜᫬䛾䝃䞊䝞䝸䝋䞊䝇᝟ሗ䛿⟶⌮୙ྍ
⤯㈶㛤Ⓨ୰䟿 
㧗ᶵ⬟䛺┘ど䜢ᐇ⌧䛩䜛Zabbix 
㧗ᶵ⬟䛺䝆䝵䝤⟶⌮䜢ᐇ⌧䛩䜛JobScheduler
HyClops JobMonitoring 
ᴫせ 
● 䝆䝵䝤ᐇ⾜≧ἣ䜢Zabbix䛻䛶┘ど䛩䜛䝒䞊䝹 
● Zabbix䜔JobScheduler⮬య䛾ᨵኚ䛿䛫䛪䛭䛾䜎䜎฼⏝ྍ⬟ 
● Zabbix䛾┘どᶵ⬟䚸JobScheduler䛾䝆䝵䝤ᐇ⾜ᶵ⬟䜢ά⏝ 
● JobScheduler฼⏝᫬䛻୙㊊䛩䜛┘どᶵ⬟䜢Zabbix䛷⿵䛖 
● OSS䛸䛧䛶බ㛤ணᐃ 
(㗦ព㛤Ⓨ୰䛻䛴䛝බ㛤䜎䛷䛧䜀䛧䛚ᚅ䛱䜢)
HyClops JobMonitoring 
3䛴䛾䝯䝸䝑䝖 
䛭䛾1. 䝆䝵䝤䛾ኻᩋ䜔㐜ᘏ᝟ሗ䜢Zabbix䛷䜻䝱䝑䝏䜰䝑䝥 
TimeLimit! 
Zabbix 
㏻▱䜢Zabbix䛾䜰䜽䝅䝵䞁タᐃ䛻ᇶ䛵䛔䛶ᐇ⾜䛷䛝䜛䛯䜑䚸䜘䜚ᰂ㌾䛺ᑐᛂ䛜ྍ⬟䛻
HyClops JobMonitoring 
3䛴䛾䝯䝸䝑䝖 
䛭䛾2. 䝆䝵䝤ᐇ⾜᫬䛾≧ែ䛾᥎⛣䜢Zabbix䛷⟶⌮ 
Zabbix䛾DB䛻䛶᥎⛣≧ἣ䜢⟶⌮䛷䛝䜛䛾䛷䝆䝵䝤ᐇ⾜䛾ศᯒ䛜ྍ⬟䛻 
ྛ䝆䝵䝤䛾ᐇ⾜᫬䛾䝃䞊䝞䝸䝋䞊䝇䛾≧ἣ䛸⣣௜䛡䛶䛾⟶⌮䜒ྍ⬟䛻 
1ᖺ๓ 
Finish! 
1᫬㛫 
⌧ᅾ 
Finish! 
3᫬㛫
HyClops JobMonitoring 
3䛴䛾䝯䝸䝑䝖 
䛭䛾3. 䝆䝵䝤ᐇ⾜᫬䛾㧗㈇Ⲵ䛻ഛ䛘䛯┘どタᐃኚ᭦ 
ᖹᖖ᫬䛾CPU㈇Ⲵ Jobᐇ⾜᫬䛾CPU㈇Ⲵ 
㜈್ 㜈್ 
Job䛾ᐇ⾜䛻䜘䜚䛒䜙䛛䛨䜑᝿ᐃ䛥䜜䜛㈇Ⲵ䜈䛾ᑐᛂ䛜ྍ⬟
HyClopsJM䛾ᶵ⬟ヲ⣽ 
䝆䝵䝤ኻᩋ≧ἣ䛾᳨▱ 
Zabbix HyClops JM JobScheduler Engine 
ᐇ⾜ 
Error⤊஢ 
䝯䞊䝹䜢 
䝣䝑䜽 
Zabbix㐃ᦠฎ⌮ mail㏦ಙ 
䜘䜚ᰂ㌾䛺㏻▱ 
䈜JobScheduler Agentẖ䛻⟶⌮ 
Zabbix Sender 
Zabbix䜰䜽䝅䝵䞁タᐃ 
Zabbix䝖䝸䜺䞊タᐃ 
Job✌ാ䝇䝔䞊䝍䝇 
Jo┘bど✌䜰ാ䜲䝇䝔䝔䝮䞊䝍䝇 
┘ど䜰䜲䝔䝮
HyClopsJM䛾ᶵ⬟ヲ⣽ 
䝆䝵䝤ኻᩋ㏻▱タᐃ౛ 
᫬㛫㍈ 
JobChain1 
ኻᩋ 
JobChain1 
ኻᩋ 
JobChain1 
ኻᩋ 
JobChain1 
ኻᩋ 
Skip! Skip!
Jobᐃ⩏ 
HyClopsJM䛾ᶵ⬟ヲ⣽ 
䝆䝵䝤✌ാ᫬㛫䛾┘ど 
Zabbix 
䝩䝇䝖 
server-01 
server-02 
䜰䜲䝔䝮 
JobA✌ാ᫬㛫 
Job1✌ാ᫬㛫 
Job3✌ാ᫬㛫 
JobScheduler Engine 
server-01 
JobScheduler Agent 
server-02 
JobScheduler Agent 
HyClops JM 
ᐃ⩏᝟ሗ䜢䜒䛸䛻 
タᐃ⮬ື㐃ᦠ 
ᐇ⾜䝻䜾䜢ゎᯒ䛧 
✌ാ᫬㛫⮬ືⓏ㘓 
⮬ືⓏ㘓 
Job2✌ാ᫬㛫 
Job4✌ാ᫬㛫 
JobA 
JobChain1 
Job1 
Job2 
JobChain2 
Job3 
Job4 
䝻䞊䝺䝧䝹䝕䜱䝇䜹䝞䝸 
Zabbix Sender
HyClopsJM䛾ᶵ⬟ヲ⣽ 
✌ാ᫬㛫᥎⛣䜾䝷䝣౛
HyClopsJM䛾ᶵ⬟ヲ⣽ 
䝖䝸䜺䞊㜈್䛾⮬ືኚ᭦ 
Zabbix 
㜈್ኚ᭦Job 
䝯䜲䞁Job 
᚟ᖐJob 
Job䝔䞁䝥䝺䞊䝖 
㜈್ኚ᭦Job 
䝔䞁䝥䝺䞊䝖 
᚟ᖐJob 
䝔䞁䝥䝺䞊䝖 
᪤Ꮡ䝖䝸䜺䞊 
↓ຠ໬ 
Jobᐇ⾜୰䛾䜏 
᭷ຠ䛺䝖䝸䜺䞊 
᪂つసᡂ 
౛: CPU౑⏝⋡>70%䛺䜙㞀ᐖ 
౛: CPU౑⏝⋡>90%䛺䜙㞀ᐖ 
Zabbix API
HyClopsJM䛾ᶵ⬟ヲ⣽ 
䝖䝸䜺䞊㜈್䛾⮬ືኚ᭦ 
Zabbix 
㜈್ኚ᭦Job 
䝯䜲䞁Job 
᚟ᖐJob 
Job䝔䞁䝥䝺䞊䝖 
㜈್ኚ᭦Job 
䝔䞁䝥䝺䞊䝖 
᚟ᖐJob 
䝔䞁䝥䝺䞊䝖 
᪤Ꮡ䝖䝸䜺䞊 
᭷ຠ໬ 
Jobᐇ⾜୰䛾䜏 
᭷ຠ䛺䝖䝸䜺䞊 
๐㝖 
౛: CPU౑⏝⋡>70%䛺䜙㞀ᐖ 
౛: CPU౑⏝⋡>90%䛺䜙㞀ᐖ 
Zabbix API
HyClopsJM䛾ᶵ⬟ヲ⣽ 
㜈್ኚ᭦౛ 
CPU䛾䝻䞊䝗䜰䝧䝺䞊䝆䛾䝖䝸䜺䞊㜈್䜢ኚ᭦䛧䛯䛔ሙྜ 
㜈್ኚ᭦Job ᚟ᖐJob 
䝟䝷䝯䞊䝍䛻௨ୗ䜢ᣦᐃ 
trigger_name 
Processor load is too high on {HOST.NAME} 
trigger_cond 
{localhost:system.cpu.load[percpu,avg1].avg(5m)}>10 
䝟䝷䝯䞊䝍䛾ᣦᐃ䛺䛧 
㜈್ኚ᭦Job䛷ኚ᭦䛥䜜䛯䜒䛾䜢඲䛶ඖ䛻ᡠ䛩
HyClopsJM䛾䝕䝰 
1. 䝆䝵䝤㏣ຍ䛻క䛔⮬ືⓗ䛻Zabbix䛻┘ど䜰䜲䝔䝮Ⓩ㘓 
2. 䝖䝸䜺䞊ኚ᭦䝆䝵䝤⤌䜏㎸䜏䛻䜘䜛ືⓗ䛺㜈್ኚ᭦ 
䝕䝰⎔ቃᵓᡂ 
AWS EC2䜲䞁䝇䝍䞁䝇䛻 Zabbix Server䛸JobScheduler Engine䜢ᑟධ 
EC2 
䝩䝇䝖ྡ: js-server 
Zabbix Server 
Job 
Chain 
Job 
Chain 
䜰䜲䝔䝮⮬ືⓏ㘓 
Job Job 
JobScheduler Engine 
㏣ຍ
HyClopsJM䛾䝕䝰 
1. 䝆䝵䝤㏣ຍ䛻క䛔⮬ືⓗ䛻Zabbix䛻┘ど䜰䜲䝔䝮Ⓩ㘓 
2. 䝖䝸䜺䞊ኚ᭦䝆䝵䝤⤌䜏㎸䜏䛻䜘䜛ືⓗ䛺㜈್ኚ᭦ 
䝕䝰⎔ቃᵓᡂ 
AWS EC2䜲䞁䝇䝍䞁䝇䛻 Zabbix Server䛸JobScheduler Engine䜢ᑟධ 
EC2 
䝩䝇䝖ྡ: js-server 
Zabbix Server 
䝖䝸䜺䞊ኚ᭦ 
Job 
Chain 
䝖䝸䜺䞊ኚ᭦䝆䝵䝤⤌䜏㎸䜏JobChain Run! 
Job 
Chain 
Job Job 
JobScheduler Engine
䜎䛸䜑 
● Zabbix䜔JobScheduler䛿㠀ᖖ䛻㧗ᶵ⬟䛺OSS 
● HyClops JobMonitoring䛿 
○ ྛ䝒䞊䝹䛾ᶵ⬟䜢ά⏝䛧䛴䛴୙㊊䛩䜛㒊ศ䜢⿵䛖䛣䛸䛷 
┦஌ຠᯝ䜢๰ฟ 
○ ௨ୗ䛾ᶵ⬟䜢ᐇ⌧ 
■ 䝆䝵䝤ᐇ⾜᫬㛫䛾┘ど 
■ 䝆䝵䝤ኻᩋ䛾᳨▱ 
■ 䝆䝵䝤ᐇ⾜᫬䛾㈇Ⲵ≧ἣ䛻ᛂ䛨䛯ືⓗ䛺┘どタᐃ 
● ௒ᚋ䛾ணᐃ 
○ OSS໬䜢ᐇ⌧ 
○ ㏣ຍᶵ⬟㛤Ⓨ䜢⾜䛔᭦䛺䜛᭷ຠ䛺㐃ᦠ䜢ᐇ⌧ 
ヨ䛧䛶䜏䛯䛔䛸䛔䛖᪉䛿䛤㐃⤡䜢䟿䟿 
ᒎ♧䝤䞊䝇䛻䛶䝕䝰䜒䛚ぢ䛫䛷䛝䜎䛩䟿
䛚ၥ䛔ྜ䜟䛫䛿䛣䛱䜙䜎䛷 
TISᰴᘧ఍♫ 
OSS䝃䝫䞊䝖䝃䞊䝡䝇ᢸᙜ❆ཱྀ 
oss-sales@ml.tis.co.jp 
䛤Ύ⫈䛒䜚䛜䛸䛖䛤䛦䛔䜎䛧䛯䚹

Zabbixとjob scheduler連携による運用システムoss化の実現

  • 1.
    Zabbix䛸JobScheduler㐃ᦠ 䛻䜘䜛㐠⏝䝅䝇䝔䝮OSS໬䛾ᐇ⌧ ࠥHyClopsJobMonitoring䛾䛤⤂௓ࠥ TIS ᰴᘧ఍♫ ᡓ␎ᢏ⾡䝉䞁䝍䞊 ụ⏣䚷኱㍜ 2014/9/5 OSC2014.Enterprise@Osaka
  • 2.
    ⮬ᕫ⤂௓ ụ⏣䚷኱㍜ TISᰴᘧ఍♫ᡓ␎ᢏ⾡䝉䞁䝍䞊 @ike_dai https://www.facebook.com/ikedai Zabbix,AWS,fluentd,Chef,JobScheduler... ྡ๓ ᡤᒓ Twitter Facebook ⯆࿡ AWS䚸VMware┘ど⏝䝥䝷䜾䜲䞁䛄HyClops for Zabbix䛅㛤Ⓨ ᢏ⾡ホㄽ♫HP : http://gihyo.jp/book/2014/978-4-7741-6288-1 (http://tech-sketch.github.io/hyclops/jp/) Amazon : http://www.amazon.co.jp/dp/4774162884
  • 3.
    TIS OSS䝃䝫䞊䝖䝃䞊䝡䝇 OSS䛾฼⏝䜢䝖䞊䝍䝹䛻䝞䝑䜽䜰䝑䝥 ಖᏲ䝃䝫䞊䝖 䝁䞁䝃䝹䝔䜱䞁䜾 䝬䜲䜾䝺䞊䝅䝵䞁
  • 4.
    TIS OSS䝃䝫䞊䝖䝃䞊䝡䝇 ᑐ㇟OSS アプリケーション 稼働基盤 運用基盤 インフラ基盤 䈜௒ᚋ䝃䝫䞊䝖ᑐ㇟ணᐃ
  • 5.
    TIS OSS䝃䝫䞊䝖䝃䞊䝡䝇 ᑐ㇟OSS アプリケーション 稼働基盤 運用基盤 インフラ基盤 䈜௒ᚋ䝃䝫䞊䝖ᑐ㇟ணᐃ 䈜TIS䛿Zabbix♫䛾ㄆᐃ䝟䞊䝖䝘䞊䛷䛩䚹
  • 6.
    㐠⏝䜢ྲྀ䜚ᕳ䛟≧ἣ䛾ኚ໬ 䝅䝇䝔䝮䛾௬᝿໬䞉䜽䝷䜴䝗໬䛜ຍ㏿ ⟶⌮ᑐ㇟ᶵჾ䛾኱つᶍ໬⟶⌮ᑐ㇟ᶵჾ䛾ືⓗ䛺ኚ໬ ᡭື㐠⏝䛷䜹䝞䞊䛷䛝䜛䝅䝇䝔䝮䛜ῶᑡ “㐠⏝⮬ື໬”䛜┈䚻ồ䜑䜙䜜䜛≧ἣ䛷 䜘䜚Ᏻᐃ䛧䛯䝅䝇䝔䝮䜢ప౯᱁䛷㎿㏿䛻ᥦ౪
  • 7.
    㐠⏝䝅䝇䝔䝮䛻 OSS䜢ྲྀ䜚ධ䜜䜛䛣䛸䛾䝯䝸䝑䝖 Ᏻᐃ ά⏝஦౛➼䜒䜸䞊䝥䞁䛻䛥䜜䜛䛣䛸䛜ከ䛟䚸 Ᏻᐃ㐠⏝䛻ྥ䛡䛯ຠᯝⓗ䛺฼⏝䝜䜴䝝䜴䛾ඹ᭷䛜ྍ⬟ ฼⏝䛾㝿䛾䝷䜲䝉䞁䝇㈝䜢పῶ (≉䛻㐠⏝⟶⌮ᑐ㇟ᶵჾ䛾ᩘ䛻ᑐ䛧䛶ᙉ䜏䛜ฟ䛶䛟䜛) 䜸䞊䝥䞁䛰䛛䜙䛣䛭㛤Ⓨ䜒άⓎ 㐠⏝⮬ື໬䛻ྥ䛡䛯ᵝ䚻䛺ᶵ⬟䜢䝒䞊䝹⮬㌟䛻㏣ຍᐇ⿦ 䝃䞊䝗䝟䞊䝔䜱〇䛸䛧䛶䝋䝸䝳䞊䝅䝵䞁ᥦ౪䛥䜜䜛䛺䛹άⓎ ప౯᱁ ㎿㏿
  • 8.
    䝆䝵䝤⟶⌮䛸┘ど䛾䛒䜚᪉ 䝆䝵䝤⟶⌮䛻ồ䜑䜙䜜䜛䛣䛸䠛 䝆䝵䝤䛜䜚䛺䛟䝇䜿䝆䝳䞊䝹㏻䜚䛻ṇᖖᐇ⾜䛥䜜䜛䛣䛸 ● 䝆䝵䝤ᐇ⾜䛜᫬㛫㏻䜚䛻ᐇ⾜䛥䜜䜛 ● 䝆䝵䝤䛜ṇᖖ⤊஢䛧䛶䛔䜛 ௒ᚋᐇ⾜䛥䜜䜛䝆䝵䝤䝇䜿䝆䝳䞊䝹䛜☜ᐇ䛻ไᚚ䛥䜜䜛䛣䛸 ● ௒ᚋ䛒䜛䝆䝵䝤䜢ᐇ⾜䛩䜛㝿䛻䝃䞊䝞䝸䝋䞊䝇䛜㊊䜚䜛䛛 ● ௒ᚋ䛒䜛䝆䝵䝤䜢ᐇ⾜䛩䜛㝿䛻☜ᐇ䛻ணᐃ㏻䜚᏶஢䛷䛝䜛䛛 “┘ど”䛜㔜せ
  • 9.
  • 10.
    OSS⤫ྜ┘ど䝋䝣䝖䜴䜵䜰Zabbix OSS⤫ྜ┘ど䝒䞊䝹 (GPLver.2) 䝷䝖䝡䜰䛾ZabbixSIA♫䛜㛤Ⓨ 2014/9/5᫬Ⅼ䛾᭱᪂Ᏻᐃ∧2.2.6 (2014/8/27䝸䝸䞊䝇) ┘どᑐ㇟ᶵჾ - Linux - Windows - NWᶵჾ ➼ ┘ど᪉ἲ - Zabbix Agent┘ど - SNMP┘ど - IPMI┘ど - 䜶䞊䝆䜵䞁䝖䝺䝇┘ど ➼ ┘どෆᐜ - 䝸䝋䞊䝇┘ど - Ṛά┘ど - ICMPPing - 䝥䝻䝉䝇 - Web┘ど - 䝻䜾┘ど - HW┘ど - SQL┘ど ➼
  • 11.
    Zabbix䛿┘どᡭἲ䛜㇏ᐩ snmp server snmp server ௵ព䛾᝟ሗ䜢pushᆺ䛷Zabbix䛻Ⓩ㘓 SNMP Trap SNMP ssh server ssh server Zabbix Agent ௬᝿໬HV Cloud ODBC DB JMX Java 䜰䝥䝸 ௬᝿⎔ቃAPI 䜽䝷䜴䝗API Zabbix Sender Zabbix Server SSH᥋⥆䛻䜘䜛 ෆ㒊䝁䝬䞁䝗ᐇ⾜┘ど ZabbixAgent 䛻䜘䜛ෆ㒊┘ど (䝸䝋䞊䝇䜔䝻䜾➼) ODBC䛻䜘䜛DB┘ど Zabbix Java Gateway䛻䜘䜛Java┘ど እ㒊䝏䜵䝑䜽䝇䜽䝸䝥䝖┘どᶵ⬟䛻䜘䜛 API┘ど
  • 12.
    Zabbix䛿⮬ື໬ᶵ⬟䛜඘ᐇ ┘どタᐃ䝟䝍䞊䞁໬ ┘どタᐃ⮬ື໬㐠⏝⮬ື໬ 䝔䞁䝥䝺䞊䝖ᶵ⬟ ZabbixAgent⮬ືⓏ㘓ᶵ⬟ 䝻䞊䝺䝧䝹䝕䜱䝇䜹䝞䝸ᶵ⬟ 䜰䜽䝅䝵䞁ᶵ⬟ ┘どタᐃ䜢䝃䞊䝞䛾⏝㏵ẖ䚸 ✀ูẖ➼䜎䛸䜑䜛ᶵ⬟ Agent㉳ື᫬䛻⮬㌟䛾᝟ሗ䜢 Zabbix Server䛻⮬ື㐃ᦠ ┘どᑐ㇟䝃䞊䝞ෆ䛾᝟ሗ䜢཰㞟䛧䚸 䛭䛾ෆᐜ䛻ྜ䜟䛫䛶⮬ືタᐃ ᮲௳䜢タᐃ䛧䛶䛒䜙䛛䛨䜑Ⓩ㘓䛧 䛯㐠⏝సᴗฎ⌮ᐇ⾜䝁䝬䞁䝗䜢 ⮬ືᐇ⾜ Zabbix APIᶵ⬟ 䝥䝻䜾䝷䝮䛛䜙 Zabbix䛾タᐃ䜔 ┘ど⤖ᯝ᝟ሗྲྀᚓ䜢ᐇ⌧
  • 13.
    ཧ⪃᝟ሗ: 䝻䞊䝺䝧䝹䝕䜱䝇䜹䝞䝸 ZabbixServer 䝕䜱䝇䜹䝞䝸䝹䞊䝹 / /data ᐃᮇⓗ䛻᥈⣴ 䜰䜲䝔䝮䝥䝻䝖䝍䜲䝥 䜰䜲䝔䝮 vfs.fs.size[{#FSNAME},free] { "data":[ 䚷䚷 { 䚷䚷 "{#FSNAME}":"/", 䚷 "{#FSTYPE}":"ext3"}, 䚷 { 䚷 "{#FSNAME}":"/data", 䚷 "{#FSTYPE}":"ext3"}, 䚷 䞉䞉䞉┬␎ } 䜰䜲䝔䝮 vfs.fs.size[/,free] vfs.fs.size[/data,free] ά⏝౛䠖 JobScheduler䛻Ⓩ㘓䛥䜜䛶䛔䜛Job୍ぴ᝟ሗ䛻ᇶ䛵䛔䛶䜰䜲䝔䝮⮬ືⓏ㘓䛸䛛
  • 14.
    ཧ⪃᝟ሗ: ZabbixAPI ■䛷䛝䜛䛣䛸 䚷- Zabbix䛻ᑐ䛩䜛᧯స(タᐃኚ᭦䚸䝩䝇䝖㏣ຍ䚸䜾䝹䞊䝥సᡂ➼)ᐇ᪋ 䚷- Zabbix䛷⟶⌮䛧䛶䛔䜛┘ど⤖ᯝ䛾ྲྀᚓ(history,event,IT service᝟ሗ) ■ API䛾ᙧᘧ: JSON-RPC 䚷- URL: http://<zabbix-server>/zabbix/api_jsonrpc.php 䚷- 䝯䝋䝑䝗: host.create䚸host.update䚸item.get䚸history.get䛺䛹 䚷- 䝟䝷䝯䞊䝍: 䝯䝋䝑䝗ẖ䛻ᚲせ䛺䝟䝷䝯䞊䝍䜢Ώ䛩 host.get䛾౛ 䝺䝇䝫䞁䝇 { "auth": "xxxxxxx", "method": "host.get", "params": { "output": "extend", "filter": {"host":"Zabbix server"} }, "jsonrpc": "2.0", "id": 1 } { "jsonrpc":"2.0", "result":[{ "maintenances":[], "hostid":"10084", "host":"Zabbix server", 䞉䞉䞉␎ }], "id":1 } 䝸䜽䜶䝇䝖䝟䝷䝯䞊䝍
  • 15.
    OSS䝆䝵䝤⟶⌮䝋䝣䝖䜴䜵䜰JobScheduler OSS䝆䝵䝤⟶⌮䝒䞊䝹 (GPLver.2) 䝗䜲䝒 䝧䝹䝸䞁䛾SOS♫䛜㛤Ⓨ 2014/9/5᫬Ⅼ䛾᭱᪂Ᏻᐃ∧1.7.4241 (2014/8/29䝸䝸䞊䝇) ᑐᛂ䝥䝷䝑䝖䝣䜷䞊䝮 - Linux - Windows - Solaris - AIX - HP-UX 䝆䝵䝤ᐇ⾜᪉ἲ - JS༢య䛷䛾䝆䝵䝤ᐇ⾜ - JS Agent䛻䜘䜛 䝸䝰䞊䝖䝃䞊䝞䜈䛾䝆䝵䝤ᐇ⾜ - SSH᥋⥆䛻䜘䜛 䝸䝰䞊䝖䝃䞊䝞䜈䛾䝆䝵䝤ᐇ⾜ ᑐᛂྍ⬟ฎ⌮ - 䝣䝻䞊(JobChain)ᐇ⾜ - 䝇䜿䝆䝳䞊䝹ᐇ⾜ - ከ㔜ᐇ⾜䞉᤼௚ไᚚ - ඛ⾜䞉ᚋ⥆ไᚚ - ඃඛᗘタᐃ - 䝆䝵䝤ᐇ⾜┘ど - 䝯䞊䝹㏻▱ ➼ 䈜 JS = JobScheduler 䈜 JS Agent = JobScheduler Agent
  • 16.
    JobScheduler䛾ᇶᮏᵓᡂ JOC JOE 䈜Job䜶䝕䜱䝍 Job Job Chain JobScheduler Agent JobScheduler Agent SSH Server 䈜Job⟶⌮WebGUI Job Job Chain DB 䈜Jobᐇ⾜ᒚṔ䚸 䚷Job䝇䜿䝆䝳䞊䝸䞁䜾᝟ሗ 䚷➼ Schedule Process Class Schedule Process Class JobScheduler Engine
  • 17.
    JobScheduler䛾Job䛾⪃䛘᪉ Standalone Job Job Chain Order Job Order Job Schedule • Standalone Job – ༢య䛷✌ື䛩䜛䝆䝵䝤䜢ᐃ⩏ • Job Chain – 」ᩘ䝆䝵䝤䜢㡰ᗎ㛵ಀ䜢⪃៖䛧䛶 ⤌䜏ྜ䜟䛫䜛䝆䝵䝤(order job)䜢ᐃ⩏ • Schedule – 䝆䝵䝤䛾✌ືணᐃ䜢ᐃ⩏ • 䝆䝵䝤ᐃ⩏䛿ᵝ䚻䛺グ㏙䛜ྍ⬟ – ShellScript, Windows bat,Java, JavaScript,VBScript, javax.script, Perl JobScheduler䛾≉ᚩ ● 䝆䝵䝤㛵㐃䛾඲䛶䛾ᐃ⩏䛜XML䝣䜯䜲䝹䛷⟶⌮ ● ᵝ䚻䛺᧯స䜢⡆༢䛻⾜䛘䜛䜘䛖䛻䛩䜛ᶵ⬟䛜ᥦ౪䛥䜜䛶䛔䜛 ○ 䝣䜯䜲䝹᧯స䚸䝣䜯䜲䝹㌿㏦䚸DBไᚚ䚸SQL᧯స䚸䝺䝫䞊䝔䜱䞁䜾ᶵ⬟䚷䛺䛹䛺䛹
  • 18.
    JobScheduler༢య䛰䛸䛷䛝䛺䛔䛣䛸 ● 䝆䝵䝤䛾ኻᩋ᫬䛾㏻▱䛿༢⣧䛺䝯䞊䝹㏦ಙ䛾䜏 (ኻᩋ᫬䛾ฎ⌮䜢䝆䝵䝤䛸䛧䛶ᐃ⩏䛩䜜䜀௵ព䛾ฎ⌮䛿ྍ⬟䛰䛜) ● JobScheduler䛻䛿䝴䞊䝄⟶⌮ᶵ⬟䛜䛺䛔 ● 䝆䝵䝤䛾ᐇ⾜⤖ᯝ䛿䝔䜻䝇䝖䛸䛧䛶ಖᏑ䛥䜜䛶䛔䜛䛾䜏 (⾲♧䛿䝔䜻䝇䝖䝧䞊䝇䛾୍ぴ⾲♧) ● ┘どᶵ⬟䛿䛺䛔䛾䛷䝆䝵䝤ᐇ⾜᫬䛾䝃䞊䝞䝸䝋䞊䝇᝟ሗ䛿⟶⌮୙ྍ
  • 19.
  • 20.
    HyClops JobMonitoring ᴫせ ● 䝆䝵䝤ᐇ⾜≧ἣ䜢Zabbix䛻䛶┘ど䛩䜛䝒䞊䝹 ● Zabbix䜔JobScheduler⮬య䛾ᨵኚ䛿䛫䛪䛭䛾䜎䜎฼⏝ྍ⬟ ● Zabbix䛾┘どᶵ⬟䚸JobScheduler䛾䝆䝵䝤ᐇ⾜ᶵ⬟䜢ά⏝ ● JobScheduler฼⏝᫬䛻୙㊊䛩䜛┘どᶵ⬟䜢Zabbix䛷⿵䛖 ● OSS䛸䛧䛶බ㛤ணᐃ (㗦ព㛤Ⓨ୰䛻䛴䛝බ㛤䜎䛷䛧䜀䛧䛚ᚅ䛱䜢)
  • 21.
    HyClops JobMonitoring 3䛴䛾䝯䝸䝑䝖 䛭䛾1. 䝆䝵䝤䛾ኻᩋ䜔㐜ᘏ᝟ሗ䜢Zabbix䛷䜻䝱䝑䝏䜰䝑䝥 TimeLimit! Zabbix ㏻▱䜢Zabbix䛾䜰䜽䝅䝵䞁タᐃ䛻ᇶ䛵䛔䛶ᐇ⾜䛷䛝䜛䛯䜑䚸䜘䜚ᰂ㌾䛺ᑐᛂ䛜ྍ⬟䛻
  • 22.
    HyClops JobMonitoring 3䛴䛾䝯䝸䝑䝖 䛭䛾2. 䝆䝵䝤ᐇ⾜᫬䛾≧ែ䛾᥎⛣䜢Zabbix䛷⟶⌮ Zabbix䛾DB䛻䛶᥎⛣≧ἣ䜢⟶⌮䛷䛝䜛䛾䛷䝆䝵䝤ᐇ⾜䛾ศᯒ䛜ྍ⬟䛻 ྛ䝆䝵䝤䛾ᐇ⾜᫬䛾䝃䞊䝞䝸䝋䞊䝇䛾≧ἣ䛸⣣௜䛡䛶䛾⟶⌮䜒ྍ⬟䛻 1ᖺ๓ Finish! 1᫬㛫 ⌧ᅾ Finish! 3᫬㛫
  • 23.
    HyClops JobMonitoring 3䛴䛾䝯䝸䝑䝖 䛭䛾3. 䝆䝵䝤ᐇ⾜᫬䛾㧗㈇Ⲵ䛻ഛ䛘䛯┘どタᐃኚ᭦ ᖹᖖ᫬䛾CPU㈇Ⲵ Jobᐇ⾜᫬䛾CPU㈇Ⲵ 㜈್ 㜈್ Job䛾ᐇ⾜䛻䜘䜚䛒䜙䛛䛨䜑᝿ᐃ䛥䜜䜛㈇Ⲵ䜈䛾ᑐᛂ䛜ྍ⬟
  • 24.
    HyClopsJM䛾ᶵ⬟ヲ⣽ 䝆䝵䝤ኻᩋ≧ἣ䛾᳨▱ ZabbixHyClops JM JobScheduler Engine ᐇ⾜ Error⤊஢ 䝯䞊䝹䜢 䝣䝑䜽 Zabbix㐃ᦠฎ⌮ mail㏦ಙ 䜘䜚ᰂ㌾䛺㏻▱ 䈜JobScheduler Agentẖ䛻⟶⌮ Zabbix Sender Zabbix䜰䜽䝅䝵䞁タᐃ Zabbix䝖䝸䜺䞊タᐃ Job✌ാ䝇䝔䞊䝍䝇 Jo┘bど✌䜰ാ䜲䝇䝔䝔䝮䞊䝍䝇 ┘ど䜰䜲䝔䝮
  • 25.
    HyClopsJM䛾ᶵ⬟ヲ⣽ 䝆䝵䝤ኻᩋ㏻▱タᐃ౛ ᫬㛫㍈ JobChain1 ኻᩋ JobChain1 ኻᩋ JobChain1 ኻᩋ JobChain1 ኻᩋ Skip! Skip!
  • 26.
    Jobᐃ⩏ HyClopsJM䛾ᶵ⬟ヲ⣽ 䝆䝵䝤✌ാ᫬㛫䛾┘ど Zabbix 䝩䝇䝖 server-01 server-02 䜰䜲䝔䝮 JobA✌ാ᫬㛫 Job1✌ാ᫬㛫 Job3✌ാ᫬㛫 JobScheduler Engine server-01 JobScheduler Agent server-02 JobScheduler Agent HyClops JM ᐃ⩏᝟ሗ䜢䜒䛸䛻 タᐃ⮬ື㐃ᦠ ᐇ⾜䝻䜾䜢ゎᯒ䛧 ✌ാ᫬㛫⮬ືⓏ㘓 ⮬ືⓏ㘓 Job2✌ാ᫬㛫 Job4✌ാ᫬㛫 JobA JobChain1 Job1 Job2 JobChain2 Job3 Job4 䝻䞊䝺䝧䝹䝕䜱䝇䜹䝞䝸 Zabbix Sender
  • 27.
  • 28.
    HyClopsJM䛾ᶵ⬟ヲ⣽ 䝖䝸䜺䞊㜈್䛾⮬ືኚ᭦ Zabbix 㜈್ኚ᭦Job 䝯䜲䞁Job ᚟ᖐJob Job䝔䞁䝥䝺䞊䝖 㜈್ኚ᭦Job 䝔䞁䝥䝺䞊䝖 ᚟ᖐJob 䝔䞁䝥䝺䞊䝖 ᪤Ꮡ䝖䝸䜺䞊 ↓ຠ໬ Jobᐇ⾜୰䛾䜏 ᭷ຠ䛺䝖䝸䜺䞊 ᪂つసᡂ ౛: CPU౑⏝⋡>70%䛺䜙㞀ᐖ ౛: CPU౑⏝⋡>90%䛺䜙㞀ᐖ Zabbix API
  • 29.
    HyClopsJM䛾ᶵ⬟ヲ⣽ 䝖䝸䜺䞊㜈್䛾⮬ືኚ᭦ Zabbix 㜈್ኚ᭦Job 䝯䜲䞁Job ᚟ᖐJob Job䝔䞁䝥䝺䞊䝖 㜈್ኚ᭦Job 䝔䞁䝥䝺䞊䝖 ᚟ᖐJob 䝔䞁䝥䝺䞊䝖 ᪤Ꮡ䝖䝸䜺䞊 ᭷ຠ໬ Jobᐇ⾜୰䛾䜏 ᭷ຠ䛺䝖䝸䜺䞊 ๐㝖 ౛: CPU౑⏝⋡>70%䛺䜙㞀ᐖ ౛: CPU౑⏝⋡>90%䛺䜙㞀ᐖ Zabbix API
  • 30.
    HyClopsJM䛾ᶵ⬟ヲ⣽ 㜈್ኚ᭦౛ CPU䛾䝻䞊䝗䜰䝧䝺䞊䝆䛾䝖䝸䜺䞊㜈್䜢ኚ᭦䛧䛯䛔ሙྜ 㜈್ኚ᭦Job ᚟ᖐJob 䝟䝷䝯䞊䝍䛻௨ୗ䜢ᣦᐃ trigger_name Processor load is too high on {HOST.NAME} trigger_cond {localhost:system.cpu.load[percpu,avg1].avg(5m)}>10 䝟䝷䝯䞊䝍䛾ᣦᐃ䛺䛧 㜈್ኚ᭦Job䛷ኚ᭦䛥䜜䛯䜒䛾䜢඲䛶ඖ䛻ᡠ䛩
  • 31.
    HyClopsJM䛾䝕䝰 1. 䝆䝵䝤㏣ຍ䛻క䛔⮬ືⓗ䛻Zabbix䛻┘ど䜰䜲䝔䝮Ⓩ㘓 2. 䝖䝸䜺䞊ኚ᭦䝆䝵䝤⤌䜏㎸䜏䛻䜘䜛ືⓗ䛺㜈್ኚ᭦ 䝕䝰⎔ቃᵓᡂ AWS EC2䜲䞁䝇䝍䞁䝇䛻 Zabbix Server䛸JobScheduler Engine䜢ᑟධ EC2 䝩䝇䝖ྡ: js-server Zabbix Server Job Chain Job Chain 䜰䜲䝔䝮⮬ືⓏ㘓 Job Job JobScheduler Engine ㏣ຍ
  • 32.
    HyClopsJM䛾䝕䝰 1. 䝆䝵䝤㏣ຍ䛻క䛔⮬ືⓗ䛻Zabbix䛻┘ど䜰䜲䝔䝮Ⓩ㘓 2. 䝖䝸䜺䞊ኚ᭦䝆䝵䝤⤌䜏㎸䜏䛻䜘䜛ືⓗ䛺㜈್ኚ᭦ 䝕䝰⎔ቃᵓᡂ AWS EC2䜲䞁䝇䝍䞁䝇䛻 Zabbix Server䛸JobScheduler Engine䜢ᑟධ EC2 䝩䝇䝖ྡ: js-server Zabbix Server 䝖䝸䜺䞊ኚ᭦ Job Chain 䝖䝸䜺䞊ኚ᭦䝆䝵䝤⤌䜏㎸䜏JobChain Run! Job Chain Job Job JobScheduler Engine
  • 33.
    䜎䛸䜑 ● Zabbix䜔JobScheduler䛿㠀ᖖ䛻㧗ᶵ⬟䛺OSS ● HyClops JobMonitoring䛿 ○ ྛ䝒䞊䝹䛾ᶵ⬟䜢ά⏝䛧䛴䛴୙㊊䛩䜛㒊ศ䜢⿵䛖䛣䛸䛷 ┦஌ຠᯝ䜢๰ฟ ○ ௨ୗ䛾ᶵ⬟䜢ᐇ⌧ ■ 䝆䝵䝤ᐇ⾜᫬㛫䛾┘ど ■ 䝆䝵䝤ኻᩋ䛾᳨▱ ■ 䝆䝵䝤ᐇ⾜᫬䛾㈇Ⲵ≧ἣ䛻ᛂ䛨䛯ືⓗ䛺┘どタᐃ ● ௒ᚋ䛾ணᐃ ○ OSS໬䜢ᐇ⌧ ○ ㏣ຍᶵ⬟㛤Ⓨ䜢⾜䛔᭦䛺䜛᭷ຠ䛺㐃ᦠ䜢ᐇ⌧ ヨ䛧䛶䜏䛯䛔䛸䛔䛖᪉䛿䛤㐃⤡䜢䟿䟿 ᒎ♧䝤䞊䝇䛻䛶䝕䝰䜒䛚ぢ䛫䛷䛝䜎䛩䟿
  • 34.
    䛚ၥ䛔ྜ䜟䛫䛿䛣䛱䜙䜎䛷 TISᰴᘧ఍♫ OSS䝃䝫䞊䝖䝃䞊䝡䝇ᢸᙜ❆ཱྀ oss-sales@ml.tis.co.jp 䛤Ύ⫈䛒䜚䛜䛸䛖䛤䛦䛔䜎䛧䛯䚹