SlideShare a Scribd company logo
“grifork”
- fast task runner via ssh -
@key_amb
at the 40th Ginza.rb #ginzarb
Who am I ?
❏ @key_amb
❏ Software Engineer
❏ Serverside, Web
❏ https://github.com/key-amb
❏ https://rubygems.org/profiles/key-amb
Before “grifork” - “fireap”
❏ https://github.com/key-amb/fireap
❏ Consul クラスタ上で動作する S3 非依存の pull 型デプロイ
ツール "fireap" を作った
“fireap” in a nutshell
❏ O(log N) … N = node number
❏ Propagative task runner on Consul Cluster
Then, what about “grifork”
❏ O(log N) … N = node number
❏ Propagative task runner via ssh
❏ https://github.com/key-amb/grifork
❏ Blog:
❏ ssh と rsync だけで Tree Deploy を実現する
"grifork" を作った
How “grifork” works?
Concept of “Tree Deploy”
Two modes to run “grifork”
1. “Standalone mode”
○ Require grifork only on the task-running host
2. “Grifork mode”
○ Require grifork on every node
Standalone mode 1/3
Standalone mode 2/3
Standalone mode 3/3
Grifork mode 1/3
Grifork mode 2/3
Grifork mode 3/3
Quickstart
% gem install grifork
% edit Griforkfile
% grifork
Configure by “Griforkfile”
branches 4
#mode :standalone
log file: 'grifork.log'
hosts ['web1.internal', 'web2.internal', 'db1.internal',
'db2.internal', ...]
local do
rsync '/path/to/myapp/'
end
remote do
rsync_remote '/path/to/myapp/'
end
See examples for more info
Changes in v0.2 ~ v0.5
❏ Released as a RubyGems
❏ https://rubygems.org/gems/grifork
❏ Enable to configure Parallel, ssh and rsync
options by Griforkfile
❏ Enable to configure tasks which runs at first
in the job or at last
Summary
❏ “grifork” is an easy way to speed up
deployment
❏ Enjoy!
Thank you!
Appendix
fireap Resource:
❏ http://www.slideshare.net/YasutakeKiyoshi/fi
reap-fast-task-runner-on-consul
Products for O(1) deployment:
❏ https://github.com/fujiwara/stretcher
❏ https://github.com/sorah/mamiya

More Related Content

What's hot

Rails Plugins 2 Hoe
Rails Plugins 2 HoeRails Plugins 2 Hoe
Rails Plugins 2 Hoe
oscon2007
 

What's hot (20)

Journey to Microservice architecture via Amazon Lambda
Journey to Microservice architecture via Amazon LambdaJourney to Microservice architecture via Amazon Lambda
Journey to Microservice architecture via Amazon Lambda
 
170112
170112170112
170112
 
OSDC 2019 | Virtualisation in Docker, using KVM as Hypervisor by Kososochukwu...
OSDC 2019 | Virtualisation in Docker, using KVM as Hypervisor by Kososochukwu...OSDC 2019 | Virtualisation in Docker, using KVM as Hypervisor by Kososochukwu...
OSDC 2019 | Virtualisation in Docker, using KVM as Hypervisor by Kososochukwu...
 
Continuous deployment with Cloud Foundry, Github and Travis CI | anynines
Continuous deployment with Cloud Foundry, Github and Travis CI | anyninesContinuous deployment with Cloud Foundry, Github and Travis CI | anynines
Continuous deployment with Cloud Foundry, Github and Travis CI | anynines
 
OSDC.no 2015 introduction to node.js workshop
OSDC.no 2015 introduction to node.js workshopOSDC.no 2015 introduction to node.js workshop
OSDC.no 2015 introduction to node.js workshop
 
Return on Ignite 2019: Azure, .NET, A.I. & Data
Return on Ignite 2019: Azure, .NET, A.I. & DataReturn on Ignite 2019: Azure, .NET, A.I. & Data
Return on Ignite 2019: Azure, .NET, A.I. & Data
 
Introduction to Python Asyncio
Introduction to Python AsyncioIntroduction to Python Asyncio
Introduction to Python Asyncio
 
Guillotina: The Asyncio REST Resource API
Guillotina: The Asyncio REST Resource APIGuillotina: The Asyncio REST Resource API
Guillotina: The Asyncio REST Resource API
 
Node.js Internals and V8 / Operating System Interaction
Node.js Internals and V8 / Operating System InteractionNode.js Internals and V8 / Operating System Interaction
Node.js Internals and V8 / Operating System Interaction
 
Macruby intro
Macruby introMacruby intro
Macruby intro
 
Concurrency in ruby
Concurrency in rubyConcurrency in ruby
Concurrency in ruby
 
Demystifying docker networking black magic - Lorenzo Fontana, Kiratech
Demystifying docker networking black magic - Lorenzo Fontana, KiratechDemystifying docker networking black magic - Lorenzo Fontana, Kiratech
Demystifying docker networking black magic - Lorenzo Fontana, Kiratech
 
Event Machine
Event MachineEvent Machine
Event Machine
 
Ender
EnderEnder
Ender
 
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
Distributed locks in Ruby - Correctness vs Efficiency - Knapsack Pro case stu...
 
Pharo, Spec and GTK
Pharo, Spec and GTKPharo, Spec and GTK
Pharo, Spec and GTK
 
Simulation in R and Python
Simulation in R and PythonSimulation in R and Python
Simulation in R and Python
 
Introduction to Node.js
Introduction to Node.jsIntroduction to Node.js
Introduction to Node.js
 
BBC's GraphDB (formerly Owlim) AWS Cloud Migration
BBC's GraphDB (formerly Owlim) AWS Cloud MigrationBBC's GraphDB (formerly Owlim) AWS Cloud Migration
BBC's GraphDB (formerly Owlim) AWS Cloud Migration
 
Rails Plugins 2 Hoe
Rails Plugins 2 HoeRails Plugins 2 Hoe
Rails Plugins 2 Hoe
 

Viewers also liked

How to create/improve OSS product and its community (revised)
How to create/improve OSS product and its community (revised)How to create/improve OSS product and its community (revised)
How to create/improve OSS product and its community (revised)
SATOSHI TAGOMORI
 
Deploying JRuby Web Applications
Deploying JRuby Web ApplicationsDeploying JRuby Web Applications
Deploying JRuby Web Applications
Joe Kutner
 

Viewers also liked (20)

Introduction to poloxy - proxy for alerting
Introduction to poloxy - proxy for alertingIntroduction to poloxy - proxy for alerting
Introduction to poloxy - proxy for alerting
 
"fireap" - fast task runner on consul
"fireap" - fast task runner on consul"fireap" - fast task runner on consul
"fireap" - fast task runner on consul
 
Azkaban
AzkabanAzkaban
Azkaban
 
Jenkins 2.0 Pipeline & Blue Ocean
Jenkins 2.0 Pipeline & Blue OceanJenkins 2.0 Pipeline & Blue Ocean
Jenkins 2.0 Pipeline & Blue Ocean
 
ボトムアップアプローチでのVRサービス立ち上げ / DeNA TechCon 2016
ボトムアップアプローチでのVRサービス立ち上げ / DeNA TechCon 2016ボトムアップアプローチでのVRサービス立ち上げ / DeNA TechCon 2016
ボトムアップアプローチでのVRサービス立ち上げ / DeNA TechCon 2016
 
Data Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby UsageData Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby Usage
 
DigdagはなぜYAMLなのか?
DigdagはなぜYAMLなのか?DigdagはなぜYAMLなのか?
DigdagはなぜYAMLなのか?
 
TLS 1.3 と 0-RTT のこわ〜い話
TLS 1.3 と 0-RTT のこわ〜い話TLS 1.3 と 0-RTT のこわ〜い話
TLS 1.3 と 0-RTT のこわ〜い話
 
Best Practices Consortium
Best Practices ConsortiumBest Practices Consortium
Best Practices Consortium
 
20140416 伊藤
20140416 伊藤20140416 伊藤
20140416 伊藤
 
とある AWS サービスの運用移管〜データストア編〜 #jawsmeguro
とある AWS サービスの運用移管〜データストア編〜 #jawsmeguroとある AWS サービスの運用移管〜データストア編〜 #jawsmeguro
とある AWS サービスの運用移管〜データストア編〜 #jawsmeguro
 
How to create/improve OSS product and its community (revised)
How to create/improve OSS product and its community (revised)How to create/improve OSS product and its community (revised)
How to create/improve OSS product and its community (revised)
 
Data Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby UsageData Analytics Service Company and Its Ruby Usage
Data Analytics Service Company and Its Ruby Usage
 
How to Make Norikra Perfect
How to Make Norikra PerfectHow to Make Norikra Perfect
How to Make Norikra Perfect
 
今更OAuth1.0についてRFC読んで理解してみた
今更OAuth1.0についてRFC読んで理解してみた今更OAuth1.0についてRFC読んで理解してみた
今更OAuth1.0についてRFC読んで理解してみた
 
Running Ruby on Solaris (RubyKaigi 2015, 12/Dec/2015)
Running Ruby on Solaris (RubyKaigi 2015, 12/Dec/2015)Running Ruby on Solaris (RubyKaigi 2015, 12/Dec/2015)
Running Ruby on Solaris (RubyKaigi 2015, 12/Dec/2015)
 
Deploying JRuby Web Applications
Deploying JRuby Web ApplicationsDeploying JRuby Web Applications
Deploying JRuby Web Applications
 
Distributed Logging Architecture in Container Era
Distributed Logging Architecture in Container EraDistributed Logging Architecture in Container Era
Distributed Logging Architecture in Container Era
 
The worst Ruby codes I’ve seen in my life - RubyKaigi 2015
The worst Ruby codes I’ve seen in my life - RubyKaigi 2015The worst Ruby codes I’ve seen in my life - RubyKaigi 2015
The worst Ruby codes I’ve seen in my life - RubyKaigi 2015
 
RubyKaigi2015 making robots-with-mruby
RubyKaigi2015 making robots-with-mrubyRubyKaigi2015 making robots-with-mruby
RubyKaigi2015 making robots-with-mruby
 

Similar to grifork - fast propagative task runner -

Programming Hive Reading #4
Programming Hive Reading #4Programming Hive Reading #4
Programming Hive Reading #4
moai kids
 
9 steps to install and configure postgre sql from source on linux
9 steps to install and configure postgre sql from source on linux9 steps to install and configure postgre sql from source on linux
9 steps to install and configure postgre sql from source on linux
chinkshady
 
Chef on SmartOS
Chef on SmartOSChef on SmartOS
Chef on SmartOS
Eric Saxby
 

Similar to grifork - fast propagative task runner - (20)

Violent python
Violent pythonViolent python
Violent python
 
Middleware as Code with mruby
Middleware as Code with mrubyMiddleware as Code with mruby
Middleware as Code with mruby
 
Cpu analysis with flamegraphs
Cpu analysis with flamegraphsCpu analysis with flamegraphs
Cpu analysis with flamegraphs
 
PWL: One VM to Rule Them All
PWL: One VM to Rule Them AllPWL: One VM to Rule Them All
PWL: One VM to Rule Them All
 
Toolbox of a Ruby Team
Toolbox of a Ruby TeamToolbox of a Ruby Team
Toolbox of a Ruby Team
 
Fuzzing softwares for bugs - OWASP Seasides
Fuzzing softwares for bugs - OWASP SeasidesFuzzing softwares for bugs - OWASP Seasides
Fuzzing softwares for bugs - OWASP Seasides
 
FLISOL 2015 - Criptografia é importante! Aprenda meios simples de proteger ar...
FLISOL 2015 - Criptografia é importante! Aprenda meios simples de proteger ar...FLISOL 2015 - Criptografia é importante! Aprenda meios simples de proteger ar...
FLISOL 2015 - Criptografia é importante! Aprenda meios simples de proteger ar...
 
Programming Hive Reading #4
Programming Hive Reading #4Programming Hive Reading #4
Programming Hive Reading #4
 
Introduction to Spark Streaming & Apache Kafka | Big Data Hadoop Spark Tutori...
Introduction to Spark Streaming & Apache Kafka | Big Data Hadoop Spark Tutori...Introduction to Spark Streaming & Apache Kafka | Big Data Hadoop Spark Tutori...
Introduction to Spark Streaming & Apache Kafka | Big Data Hadoop Spark Tutori...
 
Expert JavaScript Programming
Expert JavaScript ProgrammingExpert JavaScript Programming
Expert JavaScript Programming
 
zebra & openconfigd Introduction
zebra & openconfigd Introductionzebra & openconfigd Introduction
zebra & openconfigd Introduction
 
Fedora Developer's Conference 2014 Talk
Fedora Developer's Conference 2014 TalkFedora Developer's Conference 2014 Talk
Fedora Developer's Conference 2014 Talk
 
Get your teeth into Plack
Get your teeth into PlackGet your teeth into Plack
Get your teeth into Plack
 
9 steps to install and configure postgre sql from source on linux
9 steps to install and configure postgre sql from source on linux9 steps to install and configure postgre sql from source on linux
9 steps to install and configure postgre sql from source on linux
 
Breaking Bad Habits with GitLab CI
Breaking Bad Habits with GitLab CIBreaking Bad Habits with GitLab CI
Breaking Bad Habits with GitLab CI
 
Chef on SmartOS
Chef on SmartOSChef on SmartOS
Chef on SmartOS
 
Kubernetes debug like a pro
Kubernetes debug like a proKubernetes debug like a pro
Kubernetes debug like a pro
 
Groovy there's a docker in my application pipeline
Groovy there's a docker in my application pipelineGroovy there's a docker in my application pipeline
Groovy there's a docker in my application pipeline
 
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
OSMC 2017 | Groovy There is a Docker in my Dashing Pipeline by Kris Buytaert
 
Rugged Driven Development with Gauntlt
Rugged Driven Development with GauntltRugged Driven Development with Gauntlt
Rugged Driven Development with Gauntlt
 

Recently uploaded

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 

Recently uploaded (20)

UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
Exploring UiPath Orchestrator API: updates and limits in 2024 🚀
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
JMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and GrafanaJMeter webinar - integration with InfluxDB and Grafana
JMeter webinar - integration with InfluxDB and Grafana
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualitySoftware Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered Quality
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024IoT Analytics Company Presentation May 2024
IoT Analytics Company Presentation May 2024
 
Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........Bits & Pixels using AI for Good.........
Bits & Pixels using AI for Good.........
 
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...
 
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024
 
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová10 Differences between Sales Cloud and CPQ, Blanka Doktorová
10 Differences between Sales Cloud and CPQ, Blanka Doktorová
 

grifork - fast propagative task runner -