Rubish- A Quixotic Shell
Upcoming SlideShare
Loading in...5
×
 

Rubish- A Quixotic Shell

on

  • 1,077 views

 

Statistics

Views

Total Views
1,077
Views on SlideShare
1,077
Embed Views
0

Actions

Likes
1
Downloads
5
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?
  • Concurrent safety is

Rubish- A Quixotic Shell Rubish- A Quixotic Shell Presentation Transcript

  • Rubish: A Quixotic Shell Howard Yeh
  • Quix.ot.ic • kwik-’sä-tik • foolishly impractical especially in the pursuit of ideals ; especially : marked by rash lofty romantic ideas or extravagantly chivalrous action.
  • Why Do You Care? • Closes the gap between script and shell • Object-Oriented Meta-shell (frobbable) • Extensible – Namespace – Local extensions (with singleton objects) – Abstraction as codified memory • Ruby you know and love
  • Symbiosis With Ruby • Meta access • Data type • Plausibly concise syntax • Object#instance_eval(&block) • Rubish has no metasyntax • Rubish uses no monkey patches – Not strictly true, but…
  • Introduction to Rubish abandon bash all ye who enters…
  • Overview • Executable – Command, Pipe – Sed, Awk – Batch • Job Control – Concurrency; Exception Mechanism • Context – Dynamically scoped IO; Workspace
  • Command Rubish Bash foo foo foo :abc foo -abc foo “a b c” foo a b c foo.q “a b c” foo “a b c” foo “a”, “b”, “c” foo [quot;aquot;,quot;bquot;,quot;cquot;] foo [[quot;aquot;,quot;bquot;],quot;cquot;,quot;dquot;] Handling weird filenames rsh> touch(quot;a bquot;,quot;c dquot;).q rsh> wc(ls.map).q
  • Pipe • Build pipe with factory rsh> p { cmd1; cmd2; cmd3 } • Build pipe from array of commands rsh> @cmds = [cmd1,cmd2,cmd3] rsh> p(@cmds)
  • IO Redirection • Methods defined on the Executable class – Executable#{i,o,err} – Common API to all subclasses – So far only supports stdin, stdout, stderr • With File • With Ruby IO object • With a block that receives a pipe – Good for building further abstractions
  • IO to File rsh> cat.o(quot;outputquot;) a b c ^D rsh> wc(quot;outputquot;).first.to_i 3
  • IO with Block rsh> @c = cat.i {|pipe| pipe.puts 1,2,3 } rsh> @c 1 2 3 # ask the command to write to a pipe instead rsh> @c.o { |pipe| ... }
  • IO Abstractions • Enumerating methods – Common API to all Executable subclasses – Executable#{each,map} – Executable#{head(n=1),tail(n=1),first,last} • Implemented with the Rubish IO architecture. – OOP for the win!
  • Executable#each! def each! self.o do |pipe| pipe.each_line do |line| line.chomp! yield(line) end end job = self.exec! return job end
  • Processing with Ruby # array of listed file names ls.map # last file as string ls.last # == ls.tail(1).first # extension name of last 10 files ls.tail(10) {|f| File.extname(f) } # ditto with pipe p { …}.map {|line| … }
  • Awk & Sed
  • Sed “One-Liner” • Courtesy J. Zawinsky (Unix Hater Handbook) # find *.el files that didn't have corresponding *.elc files # only two processes per file. sh> find . -name ’*.el’ -print | sed ’s/^/FOO=/’| sed ’s/$/; if * ! -f ${FOO}c ]; then echo $FOO ; fi/’ | sh • Rubish Sed rsh> find(quot;. -name '*.el'quot;).sed { p if !File.exist?(line + quot;cquot;) } rsh> find(quot;. -name '*.el'quot;).sed { p if !File.exist?(line + quot;cquot;) }.map {|f| … }
  • Awk & Sed • Unix Powertools are for Powerfools • Hard to predict complexity. • Hard to get right. • Hard to remember. • Hard to extend. • Hard to generalize. – “weird chars” problem
  • Rubish Sed & Awk • Doesn’t aim for full generality – Since we are embedded in Ruby anyway • Captures common usage patterns – Common-Lisp loopesque helpers • Be explicit – Sed does not print by default • Fits well – Both are subclasses of Executable
  • grep with context (Sed) # grep -A1 -B1 sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h
  • grep with context (Rubish) # rubish sed sed { if line =~ /regexp/ p quot;====quot; # special case if the first line matches p prev if respond_to?(:prev) p p peek(1) end hold(:prev,1,line) # dynamically creates a method “prev” }
  • Streamer • Awk and Sed are subclasses of Streamer • Streamer#{peek(n=1),skip(n=1)} – No more sedistic register shufflings • Streamer#{max,min,count,collect,…} – Common usage patterns • Streamer#{quit,stop} • Streamer#{each,map,head,tail,…} – Inherited methods from Executable
  • grep with context (Rubish) rsh> cat.i {|p| p.puts((1..11).to_a)}.sed { ... } ==== 1 2 ==== 9 10 11 ==== 10 11
  • grep with context (Rubish) • Easy to generalize # simulates ‘grep regexp -Aa -Bb’ sed { if line =~ regexp p quot;====quot; # special case if the first line matches p prev if respond_to?(:prev) p p peek(a) end hold(:prev,b,line) }
  • Aggregator • Streamer#{max,min,count,collect,…} – Capture common usage patterns – Inspired by Common Lisp’s Loop Facility – Method signature: helper(name,value,key) • Aggregation partitioned by key – nil is the special global key under which everything is aggregated.
  • Aggregator # longest and shortest filename lengths rsh> ls.awk { max(:mx,line) min(:mn,line) }.end {[mx,mn]} [25, 3]
  • Aggregator rsh> ls.awk { f=a[0]; collect(:fn,f,File.extname(f))}.end { fn} {quot;quot;=> [quot;aquot;,...,quot;utilquot;], quot;.gzquot;=>[quot;ruby-termios-0.9.5.tar.gzquot;], quot;.ymlquot;=>[quot;VERSION.ymlquot;], quot;.shquot;=>[quot;test.shquot;], quot;.outputquot;=>[quot;awk.outputquot;], quot;.5quot;=>[quot;ruby-termios-0.9.5quot;], quot;.textilequot;=>[quot;README.textilequot;], quot;.rbquot;=>[quot;address.rbquot;, quot;foo.rbquot;, quot;my.rbquot;], quot;.barquot;=>[quot;foo.barquot;] nil=>[quot;aquot;, ...,quot;VERSION.ymlquot;]} # fn(“.rb”) contains the array of *.rb
  • Addressed Patterns .sed(3) { … } # triggered for line 3 .sed(3,9) { … } # lines 3 to 9 inclusive .sed(3,:eof) { … } # lines 3 to end of file .sed(/a/) { … } # triggered for matching lines .sed(/a/,/b/) # triggered for lines between # ditto for awk .awk(/a/,/b/)
  • Rubish Concurrency
  • Concurrency • Coarse Grained – For independent tasks • No interaction between tasks – No deadlock – No shared memory (by abstinence) – But is safety by policy (read: no safety) • Ruby Green Thread – You could always fork… I guess?
  • Background Jobs • Executable#{exec!,each!,map!} – #exec! returns an instance of Job – #each! invokes the iterator in a background thread – #map!(acc) << into an (hopefully thread safe) accumulator. • Job#{wait,stop} – #wait returns the result of a job after it completes – #stop signal a job to terminate, then wait • For Command,Pipe,Sed,Awk, and more.
  • Background Jobs # returns immediately rsh> @job = slowcat(3).exec! rsh> jobs # == [@job] # slowcat(3) takes 3 seconds to complete rsh> waitall # after 3 seconds => @job.wait rsh> @acc = [] # should use a thread-safe acc rsh> ls.map!(@acc) rsh> ls.map!(@acc)
  • Exception Handling • Job#wait would raise on abnormal completion – exitstatus != 0 • Exception avoids error checking cruft – It’s the 21st century! • A little suprising – grep quot;notfound *quot; – wc a-directory
  • Rubish in Context
  • Context • Encapsulates IOs – Dynamically scoped • Encapsulates Bindings – Lexically scoped – Namespace management – Local extensibility • A context defines the meaning of a closure with free bindings.
  • Context-Sensitive Block • Object#instance_eval(&block) obj.instance_eval { binding1 binding2 } • Local extensibility obj.extend(module).instance_eval { … }
  • Context IO with { cmd1.exec cmd2.exec with { cmd3 }.o(quot;output3quot;).exec }.o(quot;output1-2quot;).exec
  • Context Extension # ad hoc, local extensions with(derive({def foo; ...; end})) { ... # outer foo with(derive({def foo; ...; end})) { ... #inner foo } ... # outer foo }
  • Context Extension with Modules # prebaked (modules from ‘load’) with(derive(mod1,mod2)) { … }
  • Batch • A batch job is a contextualized block executed in a thread. • Schematically : @job = Thread.new { context.eval { … }} @job.wait • Similar to subshell, but in the same process • A Batch is also an Executable!
  • Batch # first extend context # then carry out a block within a batch thread batch(derive(...)) { ... } # all Executable methods apply @job = batch(derive(...)) { ... }.exec! batch(derive(...)) { ... }.map batch(derive(...)) { ... }.awk …
  • Structured Concurrency # Concurrent Jobs arranged in a tree: batch { exec! cmd1, cmd2 batch { exec! cmd3, cmd4 batch { exec! cmd5 } }.exec # we’ll wait till this batch completes … }.exec!
  • Conclusion
  • • OOP is great! – Inheritance makes code confusing – Polymorphism is powerful – Excellent namespace management • Design by Symbiosis • Singleton objects for local extensibility • Object#instance_eval(&block) • Metaprogramming – Make things frobbable – I don’t miss lisp that much…
  • Thank You • http://github.com/hayeah/rubish/tree/master – Need lots more work. – Generalize Rubish for remote scripting. • Looking for interesting projects – I like weird languages. • hayeah@gmail.com