Rubish- A Quixotic Shell

867 views

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
867
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?FoobarVerySurprisingDon’;t you think??Why, but why, would I want to say that?
  • Concurrent safety is
  • Rubish- A Quixotic Shell

    1. 1. Rubish: A Quixotic Shell Howard Yeh
    2. 2. Quix.ot.ic • kwik-’sä-tik • foolishly impractical especially in the pursuit of ideals ; especially : marked by rash lofty romantic ideas or extravagantly chivalrous action.
    3. 3. Why Do You Care? • Closes the gap between script and shell • Object-Oriented Meta-shell (frobbable) • Extensible – Namespace – Local extensions (with singleton objects) – Abstraction as codified memory • Ruby you know and love
    4. 4. Symbiosis With Ruby • Meta access • Data type • Plausibly concise syntax • Object#instance_eval(&block) • Rubish has no metasyntax • Rubish uses no monkey patches – Not strictly true, but…
    5. 5. Introduction to Rubish abandon bash all ye who enters…
    6. 6. Overview • Executable – Command, Pipe – Sed, Awk – Batch • Job Control – Concurrency; Exception Mechanism • Context – Dynamically scoped IO; Workspace
    7. 7. Command Rubish Bash foo foo foo :abc foo -abc foo “a b c” foo a b c foo.q “a b c” foo “a b c” foo “a”, “b”, “c” foo [quot;aquot;,quot;bquot;,quot;cquot;] foo [[quot;aquot;,quot;bquot;],quot;cquot;,quot;dquot;] Handling weird filenames rsh> touch(quot;a bquot;,quot;c dquot;).q rsh> wc(ls.map).q
    8. 8. Pipe • Build pipe with factory rsh> p { cmd1; cmd2; cmd3 } • Build pipe from array of commands rsh> @cmds = [cmd1,cmd2,cmd3] rsh> p(@cmds)
    9. 9. IO Redirection • Methods defined on the Executable class – Executable#{i,o,err} – Common API to all subclasses – So far only supports stdin, stdout, stderr • With File • With Ruby IO object • With a block that receives a pipe – Good for building further abstractions
    10. 10. IO to File rsh> cat.o(quot;outputquot;) a b c ^D rsh> wc(quot;outputquot;).first.to_i 3
    11. 11. IO with Block rsh> @c = cat.i {|pipe| pipe.puts 1,2,3 } rsh> @c 1 2 3 # ask the command to write to a pipe instead rsh> @c.o { |pipe| ... }
    12. 12. IO Abstractions • Enumerating methods – Common API to all Executable subclasses – Executable#{each,map} – Executable#{head(n=1),tail(n=1),first,last} • Implemented with the Rubish IO architecture. – OOP for the win!
    13. 13. Executable#each! def each! self.o do |pipe| pipe.each_line do |line| line.chomp! yield(line) end end job = self.exec! return job end
    14. 14. Processing with Ruby # array of listed file names ls.map # last file as string ls.last # == ls.tail(1).first # extension name of last 10 files ls.tail(10) {|f| File.extname(f) } # ditto with pipe p { …}.map {|line| … }
    15. 15. Awk & Sed
    16. 16. Sed “One-Liner” • Courtesy J. Zawinsky (Unix Hater Handbook) # find *.el files that didn't have corresponding *.elc files # only two processes per file. sh> find . -name ’*.el’ -print | sed ’s/^/FOO=/’| sed ’s/$/; if * ! -f ${FOO}c ]; then echo $FOO ; fi/’ | sh • Rubish Sed rsh> find(quot;. -name '*.el'quot;).sed { p if !File.exist?(line + quot;cquot;) } rsh> find(quot;. -name '*.el'quot;).sed { p if !File.exist?(line + quot;cquot;) }.map {|f| … }
    17. 17. Awk & Sed • Unix Powertools are for Powerfools • Hard to predict complexity. • Hard to get right. • Hard to remember. • Hard to extend. • Hard to generalize. – “weird chars” problem
    18. 18. Rubish Sed & Awk • Doesn’t aim for full generality – Since we are embedded in Ruby anyway • Captures common usage patterns – Common-Lisp loopesque helpers • Be explicit – Sed does not print by default • Fits well – Both are subclasses of Executable
    19. 19. grep with context (Sed) # grep -A1 -B1 sed -n -e '/regexp/{=;x;1!p;g;$!N;p;D;}' -e h
    20. 20. grep with context (Rubish) # rubish sed sed { if line =~ /regexp/ p quot;====quot; # special case if the first line matches p prev if respond_to?(:prev) p p peek(1) end hold(:prev,1,line) # dynamically creates a method “prev” }
    21. 21. Streamer • Awk and Sed are subclasses of Streamer • Streamer#{peek(n=1),skip(n=1)} – No more sedistic register shufflings • Streamer#{max,min,count,collect,…} – Common usage patterns • Streamer#{quit,stop} • Streamer#{each,map,head,tail,…} – Inherited methods from Executable
    22. 22. grep with context (Rubish) rsh> cat.i {|p| p.puts((1..11).to_a)}.sed { ... } ==== 1 2 ==== 9 10 11 ==== 10 11
    23. 23. grep with context (Rubish) • Easy to generalize # simulates ‘grep regexp -Aa -Bb’ sed { if line =~ regexp p quot;====quot; # special case if the first line matches p prev if respond_to?(:prev) p p peek(a) end hold(:prev,b,line) }
    24. 24. Aggregator • Streamer#{max,min,count,collect,…} – Capture common usage patterns – Inspired by Common Lisp’s Loop Facility – Method signature: helper(name,value,key) • Aggregation partitioned by key – nil is the special global key under which everything is aggregated.
    25. 25. Aggregator # longest and shortest filename lengths rsh> ls.awk { max(:mx,line) min(:mn,line) }.end {[mx,mn]} [25, 3]
    26. 26. Aggregator rsh> ls.awk { f=a[0]; collect(:fn,f,File.extname(f))}.end { fn} {quot;quot;=> [quot;aquot;,...,quot;utilquot;], quot;.gzquot;=>[quot;ruby-termios-0.9.5.tar.gzquot;], quot;.ymlquot;=>[quot;VERSION.ymlquot;], quot;.shquot;=>[quot;test.shquot;], quot;.outputquot;=>[quot;awk.outputquot;], quot;.5quot;=>[quot;ruby-termios-0.9.5quot;], quot;.textilequot;=>[quot;README.textilequot;], quot;.rbquot;=>[quot;address.rbquot;, quot;foo.rbquot;, quot;my.rbquot;], quot;.barquot;=>[quot;foo.barquot;] nil=>[quot;aquot;, ...,quot;VERSION.ymlquot;]} # fn(“.rb”) contains the array of *.rb
    27. 27. Addressed Patterns .sed(3) { … } # triggered for line 3 .sed(3,9) { … } # lines 3 to 9 inclusive .sed(3,:eof) { … } # lines 3 to end of file .sed(/a/) { … } # triggered for matching lines .sed(/a/,/b/) # triggered for lines between # ditto for awk .awk(/a/,/b/)
    28. 28. Rubish Concurrency
    29. 29. Concurrency • Coarse Grained – For independent tasks • No interaction between tasks – No deadlock – No shared memory (by abstinence) – But is safety by policy (read: no safety) • Ruby Green Thread – You could always fork… I guess?
    30. 30. Background Jobs • Executable#{exec!,each!,map!} – #exec! returns an instance of Job – #each! invokes the iterator in a background thread – #map!(acc) << into an (hopefully thread safe) accumulator. • Job#{wait,stop} – #wait returns the result of a job after it completes – #stop signal a job to terminate, then wait • For Command,Pipe,Sed,Awk, and more.
    31. 31. Background Jobs # returns immediately rsh> @job = slowcat(3).exec! rsh> jobs # == [@job] # slowcat(3) takes 3 seconds to complete rsh> waitall # after 3 seconds => @job.wait rsh> @acc = [] # should use a thread-safe acc rsh> ls.map!(@acc) rsh> ls.map!(@acc)
    32. 32. Exception Handling • Job#wait would raise on abnormal completion – exitstatus != 0 • Exception avoids error checking cruft – It’s the 21st century! • A little suprising – grep quot;notfound *quot; – wc a-directory
    33. 33. Rubish in Context
    34. 34. Context • Encapsulates IOs – Dynamically scoped • Encapsulates Bindings – Lexically scoped – Namespace management – Local extensibility • A context defines the meaning of a closure with free bindings.
    35. 35. Context-Sensitive Block • Object#instance_eval(&block) obj.instance_eval { binding1 binding2 } • Local extensibility obj.extend(module).instance_eval { … }
    36. 36. Context IO with { cmd1.exec cmd2.exec with { cmd3 }.o(quot;output3quot;).exec }.o(quot;output1-2quot;).exec
    37. 37. Context Extension # ad hoc, local extensions with(derive({def foo; ...; end})) { ... # outer foo with(derive({def foo; ...; end})) { ... #inner foo } ... # outer foo }
    38. 38. Context Extension with Modules # prebaked (modules from ‘load’) with(derive(mod1,mod2)) { … }
    39. 39. Batch • A batch job is a contextualized block executed in a thread. • Schematically : @job = Thread.new { context.eval { … }} @job.wait • Similar to subshell, but in the same process • A Batch is also an Executable!
    40. 40. Batch # first extend context # then carry out a block within a batch thread batch(derive(...)) { ... } # all Executable methods apply @job = batch(derive(...)) { ... }.exec! batch(derive(...)) { ... }.map batch(derive(...)) { ... }.awk …
    41. 41. Structured Concurrency # Concurrent Jobs arranged in a tree: batch { exec! cmd1, cmd2 batch { exec! cmd3, cmd4 batch { exec! cmd5 } }.exec # we’ll wait till this batch completes … }.exec!
    42. 42. Conclusion
    43. 43. • OOP is great! – Inheritance makes code confusing – Polymorphism is powerful – Excellent namespace management • Design by Symbiosis • Singleton objects for local extensibility • Object#instance_eval(&block) • Metaprogramming – Make things frobbable – I don’t miss lisp that much…
    44. 44. Thank You • http://github.com/hayeah/rubish/tree/master – Need lots more work. – Generalize Rubish for remote scripting. • Looking for interesting projects – I like weird languages. • hayeah@gmail.com

    ×