This document provides instructions for setting up and using an agri inventory system called nouka and yaoya. Nouka collects data from servers using commands and sends it to the naya data store, which uses fluentd and MongoDB. It explains the components, data formats, and provides steps to get the required software and set up the system.

1. agri inventory

2. agri inventory
nouka data collector
naya data store
naya
yaoya data convertor
push crawl
output
yaoya
nouka

3. yaoya output example(1)
 $ bin/yaoya --group system_A --name command_uname
{u'return_code': u'0', u'command_line': u'uname -a', u'execute_at':
u'1339942622', u'group_name': u'system_A', u'visible': u'True', u'host_name':
u'bb-mba.local', u'time': datetime.datetime(2012, 6, 17, 14, 17, 3), u'output':
u'Darwin bb-mba.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7
16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386n', u'_id':
ObjectId('4fdde6e0c286630ca0000009'), u'command_name':
u'command_uname'}
{u'return_code': u'0', u'command_line': u'uname -a', u'execute_at':
u'1339943547', u'group_name': u'system_A', u'visible': u'True', u'host_name':
u'bbtest-c6', u'time': datetime.datetime(2012, 6, 17, 14, 32, 28), u'output':
u'Linux bbtest-c6 2.6.32-220.13.1.el6.x86_64 #1 SMP Tue Apr 17 23:56:34 BST
2012 x86_64 x86_64 x86_64 GNU/Linuxn', u'_id':
ObjectId('4fddea7dc286630ca000003c'), u'command_name':
u'command_uname'}

4. yaoya output example(2)
 $ bin/csv --group system_A --name command_uname
"bb-mba.local","Darwin bb-mba.local 10.8.0 Darwin Kernel
Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-
1504.15.3~1/RELEASE_I386 i386"
"bbtest-c6","Linux bbtest-c6 2.6.32-220.13.1.el6.x86_64 #1
SMP Tue Apr 17 23:56:34 BST 2012 x86_64 x86_64 x86_64
GNU/Linux"

5. default commands
 command_ifconfig = ifconfig -a  command_dmidecode = /usr/sbin/dmidecode
 command_hostname = hostname  command_resolv = cat /etc/resolv.conf
 command_uname = uname -a  command_passwd = cat /etc/passwd
 command_df = df -akl  command_group = cat /etc/group
 command_iptables = /sbin/iptables-save  command_shadow = cat /etc/shadow
 command_rpm = rpm -qa  command_proc_cpuinfo = cat /proc/cpuinfo
 command_chkconfig = /sbin/chkconfig --list  command_proc_meminfo = cat /proc/meminfo
 command_ip_addr = /sbin/ip addr show  command_proc_mounts = cat /proc/mounts
 command_ip_route = /sbin/ip route show
To add command, just only
define in nouka.conf!

6. components

7. nouka data collector
 pure python 2.4
 run with python standard library
 POST result to naya with HTTP

8. naya data store
 fluentd / td-agent
 fluent-plugin-http-enhanced
 https://github.com/parolkar/fluent-plugin-http-enhanced
 fluent-plugin-mongo
 mongodb

9. data format
{ *** are same value in same execution
‘group_name’ : ‘<group_name’s value in config file>‘, ***
'host_name' : '<env HOSTNAME>', ***
'command_name' : '<command name in config file. named command_*>',
'command_line' : '<command line(value) in config file>',
'output' : '<output of command line>',
'return_code' : '<return code of command line>',
‘visible’ : ‘True’, ***
'execute_at' : '<execute date as unixtime>’, ***
}

10. getting started

11. project repository
 nouka
 https://bitbucket.org/netmarkjp/nouka
 yaoya
 https://bitbucket.org/netmarkjp/yaoya

12. setup(1/2)
 td-agent
 cat <<EOT >/etc/yum.repos.d/td.repo
[treasuredata]
name=TreasureData
baseurl=http://packages.treasure-
data.com/redhat/$basearch
gpgcheck=0
EOT
 yum -y install td-agent
 /usr/lib64/fluent/ruby/bin/gem install bson_ext
 /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-http-
enhanced

13. setup(2/2)
 mongod
 cat <<EOT >/etc/yum.repos.d/10gen.repo
[10gen]
name=10gen Repository
baseurl=http://downloads-
distro.mongodb.org/repo/redhat/os/x86_64
gpgcheck=0
EOT
 yum -y install mongo-10gen.x86_64 mongo-10gen-server.x86_64
 mercurial
 yum -y install python-devel python-setuptools gcc
 pip install mercurial

14. enjoy development!