Your SlideShare is downloading. ×
0
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
agri inventory - nouka data collector / yaoya data convertor
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

agri inventory - nouka data collector / yaoya data convertor

992

Published on

overview of agri inventory. composed by nouka data collector, yaoya data convertor

overview of agri inventory. composed by nouka data collector, yaoya data convertor

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
992
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. agri inventory
  • 2. agri inventorynouka data collectornaya data store nayayaoya data convertor push crawl output yaoya nouka
  • 3. yaoya output example(1) $ bin/yaoya --group system_A --name command_uname{ureturn_code: u0, ucommand_line: uuname -a, uexecute_at:u1339942622, ugroup_name: usystem_A, uvisible: uTrue, uhost_name:ubb-mba.local, utime: datetime.datetime(2012, 6, 17, 14, 17, 3), uoutput:uDarwin bb-mba.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 716:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386n, u_id:ObjectId(4fdde6e0c286630ca0000009), ucommand_name:ucommand_uname}{ureturn_code: u0, ucommand_line: uuname -a, uexecute_at:u1339943547, ugroup_name: usystem_A, uvisible: uTrue, uhost_name:ubbtest-c6, utime: datetime.datetime(2012, 6, 17, 14, 32, 28), uoutput:uLinux bbtest-c6 2.6.32-220.13.1.el6.x86_64 #1 SMP Tue Apr 17 23:56:34 BST2012 x86_64 x86_64 x86_64 GNU/Linuxn, u_id:ObjectId(4fddea7dc286630ca000003c), ucommand_name:ucommand_uname}
  • 4. yaoya output example(2) $ bin/csv --group system_A --name command_uname"bb-mba.local","Darwin bb-mba.local 10.8.0 Darwin KernelVersion 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386""bbtest-c6","Linux bbtest-c6 2.6.32-220.13.1.el6.x86_64 #1SMP Tue Apr 17 23:56:34 BST 2012 x86_64 x86_64 x86_64GNU/Linux"
  • 5. default commands command_ifconfig = ifconfig -a  command_dmidecode = /usr/sbin/dmidecode command_hostname = hostname  command_resolv = cat /etc/resolv.conf command_uname = uname -a  command_passwd = cat /etc/passwd command_df = df -akl  command_group = cat /etc/group command_iptables = /sbin/iptables-save  command_shadow = cat /etc/shadow command_rpm = rpm -qa  command_proc_cpuinfo = cat /proc/cpuinfo command_chkconfig = /sbin/chkconfig --list  command_proc_meminfo = cat /proc/meminfo command_ip_addr = /sbin/ip addr show  command_proc_mounts = cat /proc/mounts command_ip_route = /sbin/ip route show To add command, just only define in nouka.conf!
  • 6. components
  • 7. nouka data collector pure python 2.4 run with python standard library POST result to naya with HTTP
  • 8. naya data store fluentd / td-agent fluent-plugin-http-enhanced  https://github.com/parolkar/fluent-plugin-http-enhanced fluent-plugin-mongo mongodb
  • 9. data format{ *** are same value in same execution ‘group_name’ : ‘<group_name’s value in config file>‘, *** host_name : <env HOSTNAME>, *** command_name : <command name in config file. named command_*>, command_line : <command line(value) in config file>, output : <output of command line>, return_code : <return code of command line>, ‘visible’ : ‘True’, *** execute_at : <execute date as unixtime>’, ***}
  • 10. getting started
  • 11. project repository nouka  https://bitbucket.org/netmarkjp/nouka yaoya  https://bitbucket.org/netmarkjp/yaoya
  • 12. setup(1/2) td-agent  cat <<EOT >/etc/yum.repos.d/td.repo [treasuredata] name=TreasureData baseurl=http://packages.treasure- data.com/redhat/$basearch gpgcheck=0 EOT  yum -y install td-agent  /usr/lib64/fluent/ruby/bin/gem install bson_ext  /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-http- enhanced
  • 13. setup(2/2) mongod  cat <<EOT >/etc/yum.repos.d/10gen.repo [10gen] name=10gen Repository baseurl=http://downloads- distro.mongodb.org/repo/redhat/os/x86_64 gpgcheck=0 EOT  yum -y install mongo-10gen.x86_64 mongo-10gen-server.x86_64 mercurial  yum -y install python-devel python-setuptools gcc  pip install mercurial
  • 14. enjoy development!

×