Cover Page           Towards a New       Paradigm to Resolve        the Software Crisis Author: Jeffrey G. Long (jefflong@...
Towards A New Paradigm to Resolve the Software CrisisVarious kinds of metrics have been used to indicate the scope and mag...
Towards A NewParadigm to Resolve the    Software Crisis    S f      Ci i         Jeff Long        Feb 5, 2003        jeffl...
PProposed A       d Agenda             d Current paradigm and consequences C      t     di      d thereof (15 minutes) Pro...
P diParadigm A way of looking at a subject        f l ki     t     bj t An example, pattern, archetype, or model A set of ...
CCurrent S ft      t Software P di                 Paradigm All applications are d fi d i t         li ti        defined i...
Algorithms are implemented as softwareEverything else, such as inventoryquantities or employee names is                   ...
IImmediate C    di t Consequences Every application requires a l t of E         li ti         i     lot f software, often ...
There is a l t of “maintenance” of theTh    i    lot f “ i t        ” f thsoftware    The software has to be changed as th...
The hTh changes t software th             to ft    themselves                            lintroduce new bugs    Extensive ...
AdditiAdditional A         l Assumptions                 ti Users know what they want and can communicate that to programm...
RResults t D t    lt to Date Increasing application backlog (maybe use open source software) 2/6 development projects canc...
High turnover of Chi f I fHi h t          f Chief Information                               tiOfficers (average tenure: 18...
LLessons L     d?        Learned? Subject experts cannot communicate all requirements to programmers     their expertise t...
A AltAn Alternative P di          ti Paradigm Remove most rules from software Represent rules in canonical form, as data i...
IImmediate C    di t Consequences Software size is reduced by 2+ orders of magnitude     simpler to create, manage, unders...
“Corporate” knowledge is externalized and is Corporatein a form anyone can see and understand    Knowledge is actionable b...
Programmers do not need to know orunderstand all rules, just enough todetermine the classes of rules and theproper animati...
RResults t D t    lt to Date  Commercial system: 7x growth with  $1K/month software expenseOther prototypes in language, b...
A l i of R lAnalysis f Rules Statement of rules and device for St t      t f l        dd i f executing them can be differe...
“If” values specify conditions under        l          if    diti      dwhich each rule is examined; these arecalled “f t ...
Factors become primary keys in thi dF t     b          i      k     i third-normal form RDBMSAlternate keys can be specifi...
One i fO informal, “molecular” rule (             l “ l     l ” l (e.g. ththreestrikes and you’re out in baseball) may bet...
Tens of thousands of rules can begrouped into a small number of classesbased on their syntax and semanticsThese classes ca...
I l     t tiImplementation Separation of rules i t classes d fi S       ti    f l into l           defines tables in the R...
The R l fTh Ruleform H            Hypothesis                th i Complex system structures are created by not-     necessa...
The C RE HypothesisTh CoRE H    th iWe can create “Competency Rule Engines”, or CoREs,  consisting of <50 ruleforms, that ...
Ultra-Structure is an Example ofNotational Engineering Many problems i science, government, th M         bl     in i      ...
R fReferences Long, J.,      Denning, D., Ultra-Structure: Long J and Denning D “Ultra-Structure: A design theory for comp...
Upcoming SlideShare
Loading in …5
×

Towards a new paradigm to resolve the software crisis

845 views

Published on

February 5, 2003: “Towards a New Paradigm to Resolve the Software Crisis”. Talk given at University of North Carolina, Chapel Hill.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
845
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Towards a new paradigm to resolve the software crisis

  1. 1. Cover Page   Towards a New  Paradigm to Resolve  the Software Crisis Author: Jeffrey G. Long (jefflong@aol.com Date: February 5, 20033 Forum: Talk presented at the University of North Carolina, Chapel Hill.  Contents Page 1: Proposal Pages 2‐28: Slides (but no text) for presentation   License This work is licensed under the Creative Commons Attribution‐NonCommercial 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by‐nc/3.0/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA.  Uploaded June 27, 2011 
  2. 2. Towards A New Paradigm to Resolve the Software CrisisVarious kinds of metrics have been used to indicate the scope and magnitude of thesoftware crisis. Some of these pertain to the high percentage of uninstalled systems;others to the increasing length of the applications backlog in most shops; others to thehigh lifetime cost of developing and maintaining software, whether custom-made orpackaged; and others to the opportunity costs.All of these metrics indicate that the software business continues to be in a state of crisiswhich persists in spite of technical advances such as database management systems;structured code; improved and more formal development practices; the use of packagedsoftware, code generators, computer-aided software engineering tools; component-based, object-oriented, multi-tier software architectures; and colossal budgets. Theproblem becomes worse as society expects more from computers and becomes moredependent upon them. User and organizational frustrations with this problem havecaused CIOs to be replaced regularly, and, increasingly, have resulted in a strong trendto simply outsource all MIS computer and software operations -- and blame.This talk presents a new paradigm that has been developed and tested since 1985. It isbased on an analysis of the nature of rules, their underlying common structure, and howrules are and ought to be represented. The approach is called “Ultra-Structure”.Rules are typically hard-coded into software applications, and the maintenance of theserules as they change is the primary cause of subsequent software changes. Oneunfortunate consequence of this standard approach is that subject experts mustsomehow communicate the rules they wish to see automated to programmers whotypically are not experts in the subject matter of the application; much is lost in thetranslation. Another unfortunate consequence is that the software becomes large, oftenmillions of lines of code, such that no one involved in the project can really comprehendor manage it. There have been of course numerous initiatives directed at solving theseproblems, but the solutions have been ineffectual and the problems they address areactually secondary and symptomatic rather than primary.The basic premise of this new approach is that we can resolve all of the major difficultieswith software development, and generate significant new kinds of benefits for users, byremoving most rules and all knowledge of the world from software and insteadrepresenting them the same way we represent data, i.e. as tables in a relationaldatabase.This approach combines key features of the normally disparate areas of managementinformation systems, expert systems, and simulations, borrowing the strengths of eachand largely eliminating the known problems of each. The approach can be applied toany kind of rule-based system, whether the application is intended to support business,scientific, or other application areas. It has been applied thus far to business and naturallanguage applications, and an early prototype of its applications to biology has beendeveloped by Dr. Gidding’s laboratory.
  3. 3. Towards A NewParadigm to Resolve the Software Crisis S f Ci i Jeff Long Feb 5, 2003 jefflong@aol.com
  4. 4. PProposed A d Agenda d Current paradigm and consequences C t di d thereof (15 minutes) Proposed alternative paradigm and consequences thereof (15 minutes) Technical analysis of rules and their implementation ( minutes) p (15 ) General Q&A (15 minutes) 2/5/03 Copyright 2003 Jeff Long 2
  5. 5. P diParadigm A way of looking at a subject f l ki t bj t An example, pattern, archetype, or model A set of unconscious assumptions we have about a subject 2/5/03 Copyright 2003 Jeff Long 3
  6. 6. CCurrent S ft t Software P di Paradigm All applications are d fi d i t li ti defined in terms of f algorithms and data Algorithms are the rules which are used to manipulate data The model for this is the abacus 2/5/03 Copyright 2003 Jeff Long 4
  7. 7. Algorithms are implemented as softwareEverything else, such as inventoryquantities or employee names is names,implemented as dataThus most rules (e.g. work processes,decision trees) are implemented assoftware2/5/03 Copyright 2003 Jeff Long 5
  8. 8. IImmediate C di t Consequences Every application requires a l t of E li ti i lot f software, often >1 million lines No one can really understand or manage this level of complexity There are numerous bugs in any software Th b i ft system The ft Th software has to be changed as b h t b h d bugs are found and eliminated 2/5/03 Copyright 2003 Jeff Long 6
  9. 9. There is a l t of “maintenance” of theTh i lot f “ i t ” f thsoftware The software has to be changed as the rules change Rules h R l change as users b become more sophisticated Rules h R l change d t external pressures as due to t l well2/5/03 Copyright 2003 Jeff Long 7
  10. 10. The hTh changes t software th to ft themselves lintroduce new bugs Extensive testing is required in order to have any comfort level Extensive documentation i necessary, b t E t i d t ti is but often skipped; maintenance programmers don t don’t know why system was so designed2/5/03 Copyright 2003 Jeff Long 8
  11. 11. AdditiAdditional A l Assumptions ti Users know what they want and can communicate that to programmers Programmers who have experience in an industry can understand user requirements User requirements can be documented in a form that is less complex than the actual working system Software can be designed the same way as any other complex technology 2/5/03 Copyright 2003 Jeff Long 9
  12. 12. RResults t D t lt to Date Increasing application backlog (maybe use open source software) 2/6 development projects cancelled; additional 3/6 considered failures True value of very good designers and programmers increasingly recognized Bugs have greater and greater consequences; viruses don’t help 2/5/03 Copyright 2003 Jeff Long 10
  13. 13. High turnover of Chi f I fHi h t f Chief Information tiOfficers (average tenure: 18 months)Increasing use of packages This typically forces changes in work processes, or else requires expensive customization (>2x package price)Increasing use of outsourcing uponsurrender (35-45% by 2005)2/5/03 Copyright 2003 Jeff Long 11
  14. 14. LLessons L d? Learned? Subject experts cannot communicate all requirements to programmers their expertise took many y p y years to acquire q their own understanding will evolve Subject experts must see working prototypes, not paper representations t t ti Subject experts must be able to directly and continuously update rules as needed “Corporate” knowledge must be externalized 2/5/03 Copyright 2003 Jeff Long 12
  15. 15. A AltAn Alternative P di ti Paradigm Remove most rules from software Represent rules in canonical form, as data in a small set of tables f Externalize “corporate” knowledge in relational k l i l knowledgebase l d b Let subject experts manage data 2/5/03 Copyright 2003 Jeff Long 13
  16. 16. IImmediate C di t Consequences Software size is reduced by 2+ orders of magnitude simpler to create, manage, understand, test, p , g , , , document, and teach remaining software has no knowledge of the world; it provides basic control logic that knows what tables to check in what order, how to resolve conflicts, etc. Software development team is very small and S ft d l tt i ll d manageable 2/5/03 Copyright 2003 Jeff Long 14
  17. 17. “Corporate” knowledge is externalized and is Corporatein a form anyone can see and understand Knowledge is actionable by the computer: reasoning, error checking, d i i support i h ki decisionSubject experts can enter, change, andotherwise manage rules directly without directly,programmer assistance Data no longer merely defines certain details of the overall system; it defines the identity of the system2/5/03 Copyright 2003 Jeff Long 15
  18. 18. Programmers do not need to know orunderstand all rules, just enough todetermine the classes of rules and theproper animation proceduresSerious prototyping becomes feasible;communications with users improvesTesting & QA can be far more rigorousDocumentation can be more complete2/5/03 Copyright 2003 Jeff Long 16
  19. 19. RResults t D t lt to Date Commercial system: 7x growth with $1K/month software expenseOther prototypes in language, biology, legal, p yp g g , gy, g , games, and artificial life seem to confirm basic claims and expectationsCloser to SEI Vision: “Th right software,Cl Vi i “The i h f delivered defect free, on time and on cost, every time ” time. 2/5/03 Copyright 2003 Jeff Long 17
  20. 20. A l i of R lAnalysis f Rules Statement of rules and device for St t t f l dd i f executing them can be different; need not be software for both tb ft f b th Rules can be reformulated into a canonical form of “If a and b and c... then consider x and y and z” 2/5/03 Copyright 2003 Jeff Long 18
  21. 21. “If” values specify conditions under l if diti dwhich each rule is examined; these arecalled “f t ” in Ult St t ll d “factors” i Ultra-Structure Theory Th(UST)“Then consider” values specifyadditional criteria that must beconsidered; these are called“considerations” in UST2/5/03 Copyright 2003 Jeff Long 19
  22. 22. Factors become primary keys in thi dF t b i k i third-normal form RDBMSAlternate keys can be specified if usefulControl logic ( g (“animation p procedures”))reads relevant rules, including rulesabout selecting rules, and carries out g ,specified actions2/5/03 Copyright 2003 Jeff Long 20
  23. 23. One i fO informal, “molecular” rule ( l “ l l ” l (e.g. ththreestrikes and you’re out in baseball) may betranslated as many atomic rulesBasic process is to define what exists (existential rules) define relations between these (network & authorization rules) define processes (protocol & meta-protocol rules)2/5/03 Copyright 2003 Jeff Long 21
  24. 24. Tens of thousands of rules can begrouped into a small number of classesbased on their syntax and semanticsThese classes can be managed easilyby the tools of a RDBMSDesign can proceed by iterativeprototype; small prototypes can easilyevolve to necessary level of complexity2/5/03 Copyright 2003 Jeff Long 22
  25. 25. I l t tiImplementation Separation of rules i t classes d fi S ti f l into l defines tables in the RDBMS; there is no practical limit on number of rows in a table Referential integrity and field edits ensure that rules maintain integrity Queries, report writers and forms make access to rules easy for authorized users y 2/5/03 Copyright 2003 Jeff Long 23
  26. 26. The R l fTh Ruleform H Hypothesis th i Complex system structures are created by not- necessarily complex processes; and these p processes are created by the animation of y competency rules. Competency rules can be grouped into a small number of classes whose form is prescribed by "ruleforms". While the p y competency rules of a system change over time, the ruleforms remain constant. A well-designed collection of ruleforms can anticipate all logically p g y possible competency rules that might apply to the system, and constitutes the deep structure of the system. y 2/5/03 Copyright 2003 Jeff Long 24
  27. 27. The C RE HypothesisTh CoRE H th iWe can create “Competency Rule Engines”, or CoREs, consisting of <50 ruleforms, that are sufficient to represent all rules found among systems sharing p g y g broad family resemblances, e.g. all corporations. Their definitive deep structure will be permanent, unchanging, and robust for all members of the family, g g, y, whose differences in manifest structures and behaviors will be represented entirely as differences in competency rules. The animation p p y procedures for each engine will be relatively simple compared to current applications, requiring less than 100,000 lines of code in a third generation language. g g g 2/5/03 Copyright 2003 Jeff Long 25
  28. 28. Ultra-Structure is an Example ofNotational Engineering Many problems i science, government, th M bl in i t the arts, and engineering are caused by the way we represent them These problems cannot be resolved with more computing power or more money They require a new abstraction which can be the basis of a notational revolution 2/5/03 Copyright 2003 Jeff Long 26
  29. 29. R fReferences Long, J., Denning, D., Ultra-Structure: Long J and Denning D “Ultra-Structure: A design theory for complex systems and processes.” In Communications of the ACM (January 1995) Long, J., “A new notation for representing business and other g, , p g rules.” In Long, J. (guest editor), Semiotica Special Issue on Notational Engineering, Volume 125-1/3 (1999) Long, J., “How could the notation be the limitation?” In Long, J. (guest editor), S i ti S ( t dit ) Semiotica Special Ii l Issue on N t ti Notational l Engineering, Volume 125-1/3 (1999) Long, J., "Automated Identification of Sensitive Information in Documents Using Ultra-Structure". In Proceedings of the 20th Ultra-Structure Annual ASEM Conference, American Society for Engineering Management (October 1999) 2/5/03 Copyright 2003 Jeff Long 27

×