Copyright 2015 by
XTRAN, LLC
Using Automation to Improve
Software Services
Stephen F. Heffner
President
XTRAN, LLC
http://WWW.XTRAN-LLC.com
Copyright 2015 by XTRAN, LLC;
reproduction prohibited without
permission
Copyright 2015 by
XTRAN, LLC
Software services automation
Build configuration – automating the software build process
Source code control – controlling software versions
Model-driven development – code generated from a model, or
executable model
Testing automation frameworks
BUT – what about existing code? Alternatives –
“Point solution” tools for code analysis, transformation, or
translation
Rule-driven expert system, to automate all activities involving
existing code – a very different approach
Copyright 2015 by
XTRAN, LLC
Software services activities
on existing code
Code analysis – pull information out of the code
Code re-engineering – apply transformations to the
code
Code translation – transform code's functionality to a
different language – usually to same or higher level
(decompilation)
Code generation – special case of code translation –
usually higher-level to lower-level (compilation)
Copyright 2015 by
XTRAN, LLC
Analysis – pull information from the code
Code “demographics”
Module/function cross-reference (both directions)
Function calling tree (both directions)
Include/COPY dependency tree (both directions)
Global symbol cross-reference (both directions)
Statements per function
Code quality measures – overall / per module / per function
Cyclomatic complexity
Halstead's volume
“goto”s & exits
Knots – “goto” crossings
Dead code
Cloned code
Adherence to coding standards and conventions – specific to shop
Measurement is critical to monitoring quality; should be automated
Ad hoc analyses – answer questions specific to the code
Frequency of occurrence is critical to automation strategy
Copyright 2015 by
XTRAN, LLC
Re-engineering – apply code transformations
Code quality improvement
Structure to eliminate goto 's – impose if/else, do, while, and for
Eliminate cloned code by refactoring
Remove dead code
“Decompile” lower-level constructs to higher level
Impose coding standards and conventions
Operating system or 3rd-party API shift
Change function calls
Remap data structures
Change preprocessor constructs
Ad hoc transformations specific to code being changed
Copyright 2015 by
XTRAN, LLC
Translation – move code's functionality to a
different language
Obsolete/fading/dying language to modern language
Support for compiler / run library ending, or getting expensive
Modern development tools not available for old language
Developer productivity higher with modern language
Hard to find programmers with old language experience
Reasons for lower-level language no longer valid (e. g. speed)
Proprietary language to portable language
Rationalize disparate languages to common language
Historical legacy
Merger or acquisition
Application portfolio rationalization
Copyright 2015 by
XTRAN, LLC
What do we mean by “meta”?
Information about information
Old concept (metaphysics), recently applied to EA & IT
One level of abstraction up from “real life”
Example – meta-data in XML
Example – meta-models in modeling
A different way of approaching software engineering automation
Meta-meta – information about meta – one abstraction level up from
meta, 2 levels up from “real life”
Meta's abstraction requires conceptual & semantic ontologies
In my view, meta is the future of EA and IT, including software
services
Copyright 2015 by
XTRAN, LLC
Implications of meta for enterprise & IT
architecture
Abstraction allows horizontal integration across
domains
Abstraction allows vertical integration via meta-
tools & their rules
Abstraction reduces complexity
Abstraction reduces rate of change; the more
abstract, the less frequently it changes
Copyright 2015 by
XTRAN, LLC
Meta is all about leverage
No tools
ResultEffort
Point solution tools
ResultEffort
Meta-tool
ResultEffort
Copyright 2015 by
XTRAN, LLC
What is a “point solution” software
services tool?
Vendor has identified well-defined user “points
of pain”
Tool addresses those “pain points”
Limited flexibility may exist for the user, but –
Tool can't automate other software services
tasks
Input and output formats are usually tool-specific
Copyright 2015 by
XTRAN, LLC
What is a software engineering
meta-tool?
Not specialized for a specific “point solution”
Rule-driven tool for creating tools rapidly, hence a “meta-tool”
Expert system – expertise domain is computer languages, data,
and text
Rules language (“meta-code”) – general-purpose, but enriched with
software engineering automation primitives – a hybrid GPL/DSL
Rules language must be learnable by senior developers in
reasonable time frame, not just by mad scientists
Objectives: Very high leverage on effort expended, & expertise
reuse
Puts software engineering automation under user's control, not tool
vendor's
Copyright 2015 by
XTRAN, LLC
Anatomy of a meta-tool
User-User-
developeddeveloped
rulesrules
StandardStandard
rulesrules
Code toCode to
processprocess
AnalysisAnalysis
resultsresults
TranslatedTranslated
codecode
XMLXML
AnyAny
formatformat
Re-Re-
engineeredengineered
codecode
UserUser
PointPoint
SolutionSolution
OtherOther
toolstools
OtherOther
toolstools
Copyright 2015 by
XTRAN, LLC
Point solutions vs. a meta-tool
Software
engineering
task
Software
engineering
task
Software
engineering
task
Point
solution
Point
solution
Point
solution
Point
solution
Software
engineering
task
Software
engineering
task
Software
engineering
task
Meta-
tool
Point
solution
Rules Rules Rules
Training 1 Training 2 Training 3 Training 4
Training
ISV
Out-
sourcer
End
user
Copyright 2015 by
XTRAN, LLC
Language processing tool chains
“Point-solution” tool chain – with misfits and coverage gaps
Computer languagesComputer languages VendorVendor
AA
VendorVendor
CC
VendorVendor
BB
????
?? ??
User creates tool chain – no misfits or coverage gaps – on their schedule
– to their specifications – and they own it
Computer languagesComputer languages Meta-Meta-
tooltool
Meta-Meta-
tooltool
Meta-Meta-
tooltool
Meta-Meta-
tooltool
UserUserUserUser UserUser
Copyright 2015 by
XTRAN, LLC
A meta-tool's architecture
Symbolic Internal Representation (IR) of language
content – a meta-model of languages and their content
An instance of IR is a model of the original language
content
IR must be complete (no information lost), for recreating
re-engineered or translated code
Language parsers, to convert text form of code to IR
Rules language, to manipulate IR as its data
IR must be very rich, to facilitate manipulation
Language renderers, to convert IR back to text form
(including styling issues)
Copyright 2015 by
XTRAN, LLC
Pattern matching & replacement –
imposing if / else
if (<expression>)
{
<statements-1>
goto <label>;
}
<statements-2>
<label>:
<statement>
Match pattern
Replacement pattern
if (<expression>)
<statements-1>
else
<statements-2>
<statement>
blue => C
<red> => meta-code
Copyright 2015 by
XTRAN, LLC
if (<expression>)
{
<statements-1>
goto <label>;
}
<statements-2>
<label>:
<statement>
Match pattern
if (<expression>)
<statements-1>
else
<statements-2>
<statement>
Replacement pattern
Imposing if / else – match & replace
if ( i < 4 )
{
for (k = 0; k < 10; ++k)
++j;
goto lbl1 ;
}
if (j < 5)
k = 4;
lbl1 :
k = 3;
<expression>
<statements-1>
<label>
<statements-2>
<label>
<statement>
MatchReplace
if ( i < 4 )
{
for (k = 0; k < 10; ++k)
++j;
}
else
{
if (j < 5)
k = 4;
}
k = 3;
<expression>
<expression>
<statements-1>
<statements-2>
<statement>
Copyright 2015 by
XTRAN, LLC
The future of software services
Automation is just beginning to affect software services
As it does, the number of developers will dramatically
decrease
Average level of the survivors' talent and experience will
just as dramatically increase
They will be creating/enhancing systems using automation,
or creating/enhancing the automation tools
They will be scarce, will work "virtual" from anywhere, and
will earn globally-competitive compensation
This will eliminate labor cost arbitrage, rendering "off-
shoring" obsolete
Copyright 2015 by
XTRAN, LLC
Summary
Software engineering meta-tool – an alternative approach to software services
automation
Rule-driven expert system for manipulating computer languages and text
Capture code analysis, re-engineering, translation expertise in the form of rules,
for automated deployment
Provides leverage needed to quickly create software engineering tools to
automate the analysis,, re-engineering,, translation,, and generation of any
computer languagecomputer language
Resulting tools must be easy enough to create for ad hoc use,, but fast enough in
execution for production use
Meta-tool's rules languagerules language mustmust provideprovide sophisticated manipulation of any
computer language / data / text, including pattern matching / replacement and
“tree surgery”
Applications of such a meta-tool are limited only by the user's imaginationuser's imagination
Copyright 2015 by
XTRAN, LLC
Contact Information
Stephen F. Heffner, President
XTRAN, LLC
2820 N. Pinal Avenue, Suite 12
Casa Grande, Arizona 85122-7927
USA
http://WWW.XTRAN-LLC.com
+1(480)626-5503
info@XTRAN-LLC.com

Using Automation to Improve Software Services

  • 1.
    Copyright 2015 by XTRAN,LLC Using Automation to Improve Software Services Stephen F. Heffner President XTRAN, LLC http://WWW.XTRAN-LLC.com Copyright 2015 by XTRAN, LLC; reproduction prohibited without permission
  • 2.
    Copyright 2015 by XTRAN,LLC Software services automation Build configuration – automating the software build process Source code control – controlling software versions Model-driven development – code generated from a model, or executable model Testing automation frameworks BUT – what about existing code? Alternatives – “Point solution” tools for code analysis, transformation, or translation Rule-driven expert system, to automate all activities involving existing code – a very different approach
  • 3.
    Copyright 2015 by XTRAN,LLC Software services activities on existing code Code analysis – pull information out of the code Code re-engineering – apply transformations to the code Code translation – transform code's functionality to a different language – usually to same or higher level (decompilation) Code generation – special case of code translation – usually higher-level to lower-level (compilation)
  • 4.
    Copyright 2015 by XTRAN,LLC Analysis – pull information from the code Code “demographics” Module/function cross-reference (both directions) Function calling tree (both directions) Include/COPY dependency tree (both directions) Global symbol cross-reference (both directions) Statements per function Code quality measures – overall / per module / per function Cyclomatic complexity Halstead's volume “goto”s & exits Knots – “goto” crossings Dead code Cloned code Adherence to coding standards and conventions – specific to shop Measurement is critical to monitoring quality; should be automated Ad hoc analyses – answer questions specific to the code Frequency of occurrence is critical to automation strategy
  • 5.
    Copyright 2015 by XTRAN,LLC Re-engineering – apply code transformations Code quality improvement Structure to eliminate goto 's – impose if/else, do, while, and for Eliminate cloned code by refactoring Remove dead code “Decompile” lower-level constructs to higher level Impose coding standards and conventions Operating system or 3rd-party API shift Change function calls Remap data structures Change preprocessor constructs Ad hoc transformations specific to code being changed
  • 6.
    Copyright 2015 by XTRAN,LLC Translation – move code's functionality to a different language Obsolete/fading/dying language to modern language Support for compiler / run library ending, or getting expensive Modern development tools not available for old language Developer productivity higher with modern language Hard to find programmers with old language experience Reasons for lower-level language no longer valid (e. g. speed) Proprietary language to portable language Rationalize disparate languages to common language Historical legacy Merger or acquisition Application portfolio rationalization
  • 7.
    Copyright 2015 by XTRAN,LLC What do we mean by “meta”? Information about information Old concept (metaphysics), recently applied to EA & IT One level of abstraction up from “real life” Example – meta-data in XML Example – meta-models in modeling A different way of approaching software engineering automation Meta-meta – information about meta – one abstraction level up from meta, 2 levels up from “real life” Meta's abstraction requires conceptual & semantic ontologies In my view, meta is the future of EA and IT, including software services
  • 8.
    Copyright 2015 by XTRAN,LLC Implications of meta for enterprise & IT architecture Abstraction allows horizontal integration across domains Abstraction allows vertical integration via meta- tools & their rules Abstraction reduces complexity Abstraction reduces rate of change; the more abstract, the less frequently it changes
  • 9.
    Copyright 2015 by XTRAN,LLC Meta is all about leverage No tools ResultEffort Point solution tools ResultEffort Meta-tool ResultEffort
  • 10.
    Copyright 2015 by XTRAN,LLC What is a “point solution” software services tool? Vendor has identified well-defined user “points of pain” Tool addresses those “pain points” Limited flexibility may exist for the user, but – Tool can't automate other software services tasks Input and output formats are usually tool-specific
  • 11.
    Copyright 2015 by XTRAN,LLC What is a software engineering meta-tool? Not specialized for a specific “point solution” Rule-driven tool for creating tools rapidly, hence a “meta-tool” Expert system – expertise domain is computer languages, data, and text Rules language (“meta-code”) – general-purpose, but enriched with software engineering automation primitives – a hybrid GPL/DSL Rules language must be learnable by senior developers in reasonable time frame, not just by mad scientists Objectives: Very high leverage on effort expended, & expertise reuse Puts software engineering automation under user's control, not tool vendor's
  • 12.
    Copyright 2015 by XTRAN,LLC Anatomy of a meta-tool User-User- developeddeveloped rulesrules StandardStandard rulesrules Code toCode to processprocess AnalysisAnalysis resultsresults TranslatedTranslated codecode XMLXML AnyAny formatformat Re-Re- engineeredengineered codecode UserUser PointPoint SolutionSolution OtherOther toolstools OtherOther toolstools
  • 13.
    Copyright 2015 by XTRAN,LLC Point solutions vs. a meta-tool Software engineering task Software engineering task Software engineering task Point solution Point solution Point solution Point solution Software engineering task Software engineering task Software engineering task Meta- tool Point solution Rules Rules Rules Training 1 Training 2 Training 3 Training 4 Training ISV Out- sourcer End user
  • 14.
    Copyright 2015 by XTRAN,LLC Language processing tool chains “Point-solution” tool chain – with misfits and coverage gaps Computer languagesComputer languages VendorVendor AA VendorVendor CC VendorVendor BB ???? ?? ?? User creates tool chain – no misfits or coverage gaps – on their schedule – to their specifications – and they own it Computer languagesComputer languages Meta-Meta- tooltool Meta-Meta- tooltool Meta-Meta- tooltool Meta-Meta- tooltool UserUserUserUser UserUser
  • 15.
    Copyright 2015 by XTRAN,LLC A meta-tool's architecture Symbolic Internal Representation (IR) of language content – a meta-model of languages and their content An instance of IR is a model of the original language content IR must be complete (no information lost), for recreating re-engineered or translated code Language parsers, to convert text form of code to IR Rules language, to manipulate IR as its data IR must be very rich, to facilitate manipulation Language renderers, to convert IR back to text form (including styling issues)
  • 16.
    Copyright 2015 by XTRAN,LLC Pattern matching & replacement – imposing if / else if (<expression>) { <statements-1> goto <label>; } <statements-2> <label>: <statement> Match pattern Replacement pattern if (<expression>) <statements-1> else <statements-2> <statement> blue => C <red> => meta-code
  • 17.
    Copyright 2015 by XTRAN,LLC if (<expression>) { <statements-1> goto <label>; } <statements-2> <label>: <statement> Match pattern if (<expression>) <statements-1> else <statements-2> <statement> Replacement pattern Imposing if / else – match & replace if ( i < 4 ) { for (k = 0; k < 10; ++k) ++j; goto lbl1 ; } if (j < 5) k = 4; lbl1 : k = 3; <expression> <statements-1> <label> <statements-2> <label> <statement> MatchReplace if ( i < 4 ) { for (k = 0; k < 10; ++k) ++j; } else { if (j < 5) k = 4; } k = 3; <expression> <expression> <statements-1> <statements-2> <statement>
  • 18.
    Copyright 2015 by XTRAN,LLC The future of software services Automation is just beginning to affect software services As it does, the number of developers will dramatically decrease Average level of the survivors' talent and experience will just as dramatically increase They will be creating/enhancing systems using automation, or creating/enhancing the automation tools They will be scarce, will work "virtual" from anywhere, and will earn globally-competitive compensation This will eliminate labor cost arbitrage, rendering "off- shoring" obsolete
  • 19.
    Copyright 2015 by XTRAN,LLC Summary Software engineering meta-tool – an alternative approach to software services automation Rule-driven expert system for manipulating computer languages and text Capture code analysis, re-engineering, translation expertise in the form of rules, for automated deployment Provides leverage needed to quickly create software engineering tools to automate the analysis,, re-engineering,, translation,, and generation of any computer languagecomputer language Resulting tools must be easy enough to create for ad hoc use,, but fast enough in execution for production use Meta-tool's rules languagerules language mustmust provideprovide sophisticated manipulation of any computer language / data / text, including pattern matching / replacement and “tree surgery” Applications of such a meta-tool are limited only by the user's imaginationuser's imagination
  • 20.
    Copyright 2015 by XTRAN,LLC Contact Information Stephen F. Heffner, President XTRAN, LLC 2820 N. Pinal Avenue, Suite 12 Casa Grande, Arizona 85122-7927 USA http://WWW.XTRAN-LLC.com +1(480)626-5503 info@XTRAN-LLC.com