Paul Compton, UNSW, RAISE'13

1. Incremental Software
Engineering
Paul Compton
School of Computer Science and Engineering
The University of New South Wales, Australia
Knowledge

2. Can Software Engineering be reduced to
Knowledge Engineering?
Knowledge Engineering ??!!**

3. The problem
• Requirements are never complete
Why?
• McCarthy’s qualification problem (1977)
– You can never anticipate every situation a robot will
have to deal with
• Domain knowledge is never complete
– An expert can’t recall every situation related to the
context.
• Is knowledge acquisition asymptotic?
– Smaller and smaller improvements
– At greater and greater cost

4. 96% accurate
99.7% accurate

5. Neural Net?
Support Vector Machine?
Decision tree learners?
iLog, Drools?
Etc?
Building a classifier

7. 7 rules added8 rules added
Ripple Down Rules

8. Ripple-Down Rules Features
• Controlled inference
• Cornerstone cases
– Any set of cases correctly handled by previously
rules which may fire a new rule being added
– Generally, cases for which rules were added
• Monitoring
– 85.4% auto-validated
Standard production rules If [condition] then [action-list]
Linked production rules If [condition] then [case action-list]
[inference action]
else [inference action]
• 7.5 million cases, 6 months, 12 labs, 185 KBs
• 85.4% auto-validated (labs range 48% to 96%)
• Rules added for 0.3% of the 14.6% manually monitored

9. General Biochemistry
0.06
0.13
0.25
0.50
1.00
2.00
4.00
8.00
16.00
32.00
64.00
128.00
256.00
512.00
0 500 1,000 1,500 2,000 2,500 3,000 3,500
meintoaddanddebugarule(mins)
medianme(mins)
rule number
2003 2004 2005 2006 2007 2008 2010 2011
253 different comments
7.3M reports processed
28hrs 8hrs 10hrs 9hrs 12hrs 5hrs 6hrs
Data from Pacific Knowledge Systems

10. 0.06
0.13
0.25
0.50
1.00
2.00
4.00
8.00
16.00
32.00
64.00
128.00
256.00
512.00
0 2,000 4,000 6,000 8,000 10,000 12,000 14,000
Timetoaddarule(mins)
medianmeover50rules(mins)
Rule number
Data from Pacific Knowledge Systems

11. 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 12000 13000
Percetofrulesaddedwithvariousnumbersofcondions
rules added rounded up to the next 1000
> 10 condi ons
6 to 10 condi ons
4 or 5 condi ons
up to 3 condi ons
Data from Pacific Knowledge Systems

12. 0.06
0.13
0.25
0.50
1.00
2.00
4.00
8.00
16.00
32.00
64.00
128.00
256.00
512.00
0 2,000 4,000 6,000 8,000 10,000 12,000 14,000
Timetoaddarule(mins)medianmeover50rules(mins)
averageover50rules(mins)avg.noofcondionsperrulerules
Rule number
Data from Pacific Knowledge Systems

13. Comparison Data
Rule-building
techniques
Time to add rules number of rules Source of the
data
Various standard
techniques
Zacharias - 2008
38 mins per rule
To
385 mins per rule
126,016
(estimate)
64 survey
respondents
Ripple Down Rules
Compton et al - 2011
78 secs per rule 57,626 Logs from 17 labs

14. Other applications
Pathology applications
Pacific Knowledge Systems
• Report interpretation
• Auditing data entry
• Real-time alerts
• Risk management
• Data cleansing
• Information extraction
– Part of speech
– Named entity
– Relations

15. Re-engineering legacy systems
RDR System
Legacy System
input output
compare
rules added when necessary
Erudine Pty. Ltd.

16. What sort of Software Engineering is
Possible with Ripple-Down Rules?

17. output
General RDR
2 3 4 41
input
If A & B THEN X , (RULE 1, ELSE RULE 4)
If A & B THEN X
UNLESS C THEN nul (rule 4, ELSE rule 1)
If C THEN Y (rule 1, exit)
5
If . . . . . THEN . . (rule 1, exit)
If . . . . . THEN . . (rule 1, rule 5)
No rule can change an assertion
made by any other rule
Linked production rules If [condition] then [case action-list]
[inference action]
else [inference action]

18. General RDR
output
2 3 4 41
input
5
Another
knowledge-based
system
If A & B Then X
Some other
program
A
other program

19. Summary
• The unexpected will always arise.
– (but everything was unexpected once)
• Incremental knowledge engineering works
• Is incremental software engineering possible?
• Can incremental knowledge engineering by used in
software engineering?
THANK YOU