Automation

John Ashmead
Automation - John Ashmead
Friday, December 27, 13

It is a profoundly erroneous
truism, repeated ...
Cemetery Permissions
• Family owned cemetery with elaborate rules
• Too elaborate to do by hand
• But came in logical grou...
Video transmission system
• SQL Server 6.0: very buggy
• VC++: less buggy than SQL Server 6.0
• Informix: stable but only ...
Web Portfolio System
• Needed lots of simple stored procedures
• Changed a lot
• So, wrote code generator to build stored
...
Assets Under Mgmt
• Errors of order $10,000,000
• 7 dimensions of analysis
• Best interface: Active Reports
• But, took 1 ...
actrpt_bld.pl
#!/opt/perl/perl-5.6.1/perl -w
# actrpt_bld.pl -- build xml, cs files for active reports, using stuff in
#
e...
XML (RPX)
<?xml version="1.0" encoding="UTF-8"?>
<!-- 2005-03-24 15:22:03: /usr/local/bin/actrpt_bld.pl -d gim2 lu_manager...
C#
// arLuManagers.cs
// 2005-03-24 15:22:03:
using
using
using
using
using
using

/usr/local/bin/actrpt_bld.pl -d gim2 lu...
Report builder

Automation - John Ashmead
Friday, December 27, 13

http://ashmeadsoftwareandconsulting.com
What is Automation?
Automation - John Ashmead
Friday, December 27, 13

http://ashmeadsoftwareandconsulting.com
Levels of automation
• Robots
• Objects
• Frameworks
• Macros
• Scripts
• One-liners
Automation - John Ashmead
Friday, Dec...
Key principles
• Get it working manually first
• Code generator should chunk the same way a
human does

• Output should be ...
Languages
• Perl
• C (with Lex & Yacc)
• Shell
• M4
• CPP
Automation - John Ashmead
Friday, December 27, 13

http://ashmea...
Perl
• Personal favorite
• The “Swiss Army Knife” of programming
languages

• Lots & lots & lots of modules on CPAN
• Synt...
Benefits
• Correctness
• Speed of construction
• Enhanced functionality
• Maintainability (this one goes both ways)
• Chall...
Correctness
• Code that doesn’t exist can’t break
• Resulting code can be more detailed than is
practical by hand

• When ...
Speed of construction
• 20,000 lines/
minute

• Reduced testing,
once initial tests
are done

• Increased

Automation - Jo...
Enhanced functionality
• Can do things that may not be possible with
existing tools

• Or, may be possible but impractical...
Maintainability
• Less “real code” means fewer places to look
for bugs

• Generated code tends to break
spectacularly or n...
Intellectually challenging
• Higher level of
abstraction

• Better fit to
problem
domain

• Eqn, pic, other
“little
languag...
Gotchas
• Can be harder to maintain
• Can acquire a “sinister life of its own”
• Automation is not a substitute for judgme...
Maintenance issues:
need extra attention to:
• Defensive
• Self-checking
• Transparent
• Readable
output

Automation - Joh...
Automation
can get out
of control

Automation - John Ashmead
Friday, December 27, 13

http://ashmeadsoftwareandconsulting....
User:
[Do I] still have my user id in the future or
[am] I phased out?
Developer:
No, unfortunately the human race will be...
Automation not a
substitute for judgment
• That video transmission system failed

because it turned out to have no clear
o...
Other forms of
automation
• Automated testing
• Toy languages
• Intelligent software
• Frameworks
• Insert startling work ...
Different Styles
• Cheap & cheerful
• Diverse & experimental
• Clean & elegant
Automation - John Ashmead
Friday, December ...
CMM

Automation - John Ashmead
Friday, December 27, 13

http://ashmeadsoftwareandconsulting.com
Automation is
Nature’s way
• 30 repetitions equals one habit
• Time-honored traditions can appear with
astonishing speed

...
Take-aways
• Automation something to always keep an
eye out for

• Offers significant benefits
• Requires careful attention ...
Conclusion
• Civilization

advances by
extending the
number of
important
operations we
can perform
without thinking
about ...
Zen & the Art of
Debugging
• www.zenandtheartofdebugging.com

Automation - John Ashmead
Friday, December 27, 13

http://as...
Upcoming SlideShare
Loading in …5
×

Automation

641 views

Published on

"It is a profoundly erroneous truism, repeated by all copybooks and by eminent people making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations we can perform without thinking about them. Operations of thought are like cavalry charges in a battle - they are strictly limited in number, they require fresh horses, and must only be made at decisive moments."
-- Alfred North Whitehead,

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
641
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
5
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Automation

  1. 1. Automation John Ashmead Automation - John Ashmead Friday, December 27, 13 It is a profoundly erroneous truism, repeated by all copybooks and by eminent people making speeches, that we should cultivate the habit of thinking of what we are doing. The precise opposite is the case. Civilization advances by extending the number of important operations we can perform without thinking about them. Operations of thought are like cavalry charges in a battle - they are strictly limited in number, they require fresh horses, and must only be made at decisive moments. -- Alfred North Whitehead, 1911 http://ashmeadsoftwareandconsulting.com
  2. 2. Cemetery Permissions • Family owned cemetery with elaborate rules • Too elaborate to do by hand • But came in logical groups • So, wrote some scripts to “fan-out” permissions from master identities Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  3. 3. Video transmission system • SQL Server 6.0: very buggy • VC++: less buggy than SQL Server 6.0 • Informix: stable but only on server • So, sent stored procedures definitions from SQL Server to Informix • Then, used Informix to build VC++ • Which then called SQL Server 6.0 Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  4. 4. Web Portfolio System • Needed lots of simple stored procedures • Changed a lot • So, wrote code generator to build stored procedures: 20,000 lines of code/minute • 60% good as is, 30% needed tweaks, 10% worthless • Turned 8 week project -> 2 weeks Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  5. 5. Assets Under Mgmt • Errors of order $10,000,000 • 7 dimensions of analysis • Best interface: Active Reports • But, took 1 hour + per report (& boring) • So, wrote code generators to build C#, XML • 6500 lines of code, 21 reports in 10 hours Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  6. 6. actrpt_bld.pl #!/opt/perl/perl-5.6.1/perl -w # actrpt_bld.pl -- build xml, cs files for active reports, using stuff in # ete_procs # ete_proc_inputs # ete_proc_outputs # Weak spot: # typing is done implicitly # KNOWN WEAK SPOTS: # mapping of Informix types to C# types, # selection of conversion function from user parameter to C# type, # treatment of parameters which have been left null by user use strict; use use use use use Getopt::Long; Time::localtime; DBI; FileHandle; XML::Writer; sub actrpt_bld(); Automation - John Ashmead Friday, December 27, 13 # build active report files (CS, XML) for one proc http://ashmeadsoftwareandconsulting.com
  7. 7. XML (RPX) <?xml version="1.0" encoding="UTF-8"?> <!-- 2005-03-24 15:22:03: /usr/local/bin/actrpt_bld.pl -d gim2 lu_managers --> <ActiveReportLayout Version="3" PrintWidth="14400" DocumentName="ActiveReports Document" ScriptLang="C#" CodeFile="arLuManagers.cs"> <StyleSheet> <Style Name="Normal" Value="font-family: Arial; font-size: 10pt; font-weight: normal; vertical-align: bottom; text-align: left; color: rgb(0, 0, 0); "></Style> <Style Name="Heading1" Value="font-family: Arial; font-size: 16pt; font-weight: bold; vertical-align: bottom; text-align: left; color: rgb(0, 0, 0); "></Style> <Style Name="Heading2" Value="font-family: Arial; font-size: 14pt; font-weight: bold; vertical-align: bottom; text-align: left; color: rgb(0, 0, 0); "></Style> <Style Name="Heading3" Value="font-family: Arial; font-size: 12pt; font-weight: bold; vertical-align: bottom; text-align: left; color: rgb(0, 0, 0); "></Style> </StyleSheet> <Sections> <Section Type="ReportHeader" Name="ReportHeader" Height="300" BackColor="16777215" CanShrink="0"> <Control Type="AR.Label" Name="lbl_v_managers_manager" MOD="4" Left="0" Top="0" Width="1800" Height="300" Text="v_managers_manager" CanGrow="0" Caption="Manager Code" Style="white-space: nowrap; fontweight: bold"></Control> Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  8. 8. C# // arLuManagers.cs // 2005-03-24 15:22:03: using using using using using using /usr/local/bin/actrpt_bld.pl -d gim2 lu_managers System; System.Data; IBM.Data.Informix; DataDynamics.ActiveReports; DataDynamics.ActiveReports.DataSources; DataDynamics.ActiveReports.Document; namespace ETE.Reports { public class arLuManagers : ar1838Base { public arLuManagers() { InitializeReport(); this._default_path = @"ims-kop-fs01Departments"; // force into landscape, more appropriate to Excel extracts this.PageSettings.Orientation = DataDynamics.ActiveReports.Document.PageOrientation.Landscape; this.PageSettings.Margins.Left = (float)0.5; this.PageSettings.Margins.Right = (float)0.5; this.PageSettings.Margins.Top = (float)0.5; this.PageSettings.Margins.Bottom = (float)0.5; } Automation private void arLuManagers_DataInitialize(object sender, System.EventArgs eArgs) { // gather parameters - John Ashmead FP = new frmParameters(); http://ashmeadsoftwareandconsulting.com frmParameters Friday, December 27, 13
  9. 9. Report builder Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  10. 10. What is Automation? Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  11. 11. Levels of automation • Robots • Objects • Frameworks • Macros • Scripts • One-liners Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  12. 12. Key principles • Get it working manually first • Code generator should chunk the same way a human does • Output should be as readable as possible (i.e. include indentation & comments) • Don’t expect to automate everything Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  13. 13. Languages • Perl • C (with Lex & Yacc) • Shell • M4 • CPP Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  14. 14. Perl • Personal favorite • The “Swiss Army Knife” of programming languages • Lots & lots & lots of modules on CPAN • Syntax a derivative of Old High Martian Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  15. 15. Benefits • Correctness • Speed of construction • Enhanced functionality • Maintainability (this one goes both ways) • Challenging Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  16. 16. Correctness • Code that doesn’t exist can’t break • Resulting code can be more detailed than is practical by hand • When grunt work is done by machine, there is more time to focus on tricky bits • People are bad at getting the details right Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  17. 17. Speed of construction • 20,000 lines/ minute • Reduced testing, once initial tests are done • Increased Automation - John Ashmead Friday, December 27, 13 willingness to refactor, since rebuilds are easier http://ashmeadsoftwareandconsulting.com
  18. 18. Enhanced functionality • Can do things that may not be possible with existing tools • Or, may be possible but impractical • Or, that it didn’t even occur to you to build Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  19. 19. Maintainability • Less “real code” means fewer places to look for bugs • Generated code tends to break spectacularly or not at all • Of course, the code generator itself will be harder to understand, since it works at a higher level of abstraction Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  20. 20. Intellectually challenging • Higher level of abstraction • Better fit to problem domain • Eqn, pic, other “little languages” Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  21. 21. Gotchas • Can be harder to maintain • Can acquire a “sinister life of its own” • Automation is not a substitute for judgment Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  22. 22. Maintenance issues: need extra attention to: • Defensive • Self-checking • Transparent • Readable output Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  23. 23. Automation can get out of control Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  24. 24. User: [Do I] still have my user id in the future or [am] I phased out? Developer: No, unfortunately the human race will be extinct by that time, having been replaced by some cyber organisms that evolved upward from current-day cell phones. Once the cell phones realized they didn’t need people to have meaningful conversations with their fellow phones, it was pretty much obsolescence-ville for the human race. A cell-phone hosted AI programmed by Sony but then hacked by a Japanese teenage girl in 2009 will be the starting point for all this, tho it takes a full 23 years for the end to finally arrive. But it’s ok: everything interesting which the human race had to say had already been recorded by the cells. Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  25. 25. Automation not a substitute for judgment • That video transmission system failed because it turned out to have no clear objective • The quality of the code became irrelevant • Automating a mess produces an automated mess Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  26. 26. Other forms of automation • Automated testing • Toy languages • Intelligent software • Frameworks • Insert startling work of heart-breaking genius here… Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  27. 27. Different Styles • Cheap & cheerful • Diverse & experimental • Clean & elegant Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  28. 28. CMM Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  29. 29. Automation is Nature’s way • 30 repetitions equals one habit • Time-honored traditions can appear with astonishing speed • Penalty for failure to automate is death Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  30. 30. Take-aways • Automation something to always keep an eye out for • Offers significant benefits • Requires careful attention to execution Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  31. 31. Conclusion • Civilization advances by extending the number of important operations we can perform without thinking about them Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com
  32. 32. Zen & the Art of Debugging • www.zenandtheartofdebugging.com Automation - John Ashmead Friday, December 27, 13 http://ashmeadsoftwareandconsulting.com

×