A	
  Hybrid	
  Diagnosis	
  Approach	
  Combining	
  
Black-­‐Box	
  and	
  White-­‐Box	
  Reasoning	
  	
  
Mingmin	
  Ch...
Outline	
  
•  The	
  Taxonomy	
  Alignment	
  Problem	
  
T1	
  +	
  T2	
  +	
  A	
  =>	
  T3	
  	
  	
  	
  	
  	
  	
  ...
Meet	
  Prof.	
  Nico	
  Franz:	
  	
  
	
  Curator	
  of	
  Insects	
  @	
  ASU	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
 ...
What	
  Nico	
  et	
  al.	
  do	
  for	
  a	
  living	
  …	
  	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  Rule...
 	
  What	
  Nico	
  does	
  for	
  a	
  living	
  (cont’d):	
  
The	
  Indoors	
  Part	
  
•  First:	
  go	
  fun	
  plac...
•  Given:
–  Taxonomies T1 , T2
•  incl. constraints (coverage, disjointness)
–  Set of articulations (alignment) A
•  Fin...
 	
  TAP:	
  Possible	
  Outcomes	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
isa
isa
Input	
  Alignment	
  
Hybrid	
  Diagn...
 	
  TAP:	
  Possible	
  Outcomes	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
isa
isa
Input	
  Alignment	
  
{A1, A2, A3, A4...
 TAP:	
  Possible	
  Outcomes	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
isa
isa
Input	
  Alignment	
  
{A1, A2, A3, A4}
{A...
 TAP:	
  Possible	
  Outcomes	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
isa
isa
Input	
  Alignment	
  
{A1, A2, A3, A4}
{A...
 TAP:	
  Possible	
  Outcomes	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
isa
isa
Input	
  Alignment	
  
{A1, A2, A3, A4}
{A...
•  FO	
  reasoning	
  about	
  
taxonomies	
  (MFOL)	
  
•  Earlier:	
  CleanTax	
  	
  
–  Prover9/Mace4	
  
•  Now:	
  E...
Real-­‐world	
  examples:	
  Turn	
  this	
  …	
  	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  P...
… into this!
(Perellescus Alignment Result)
•  T3 := T1 and T2 are “merged”
–  Blue dashed: overlaps è resolve via “zoom-...
 	
  Possible	
  Outcome:	
  Inconsistency!	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
isa
isa
Input	
  Alignment	
  
{A1, ...
Example	
  Instance	
  	
  
(from	
  synthefc	
  benchmark	
  suite)	
  
•  Here:	
  N	
  =	
  10	
  taxa	
  in	
  T1,	
  ...
Visualizing	
  Diagnoses:	
  Scalability	
  
Issues	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  ...
Beaer	
  Idea:	
  Just	
  show	
  MIS,	
  MCS	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  Prague...
..	
  but	
  4	
  MCS	
  and	
  
1	
  MIC	
  tell	
  it	
  all!	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  Rul...
Visualizing	
  Diagnoses	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  Prague	
   20	
  
Example	
...
Black-Box Inconsistency Analysis
(Diagnostic Lattice)
•  Then:
–  Repair: find & revise minimal inconsistent subsets (Min-...
•  Black-­‐box	
  Analysis	
  (Hinng	
  Set	
  algo.)	
  yields	
  a	
  Diagnosis	
  (lance)	
  
–  for	
  n=4	
  arfculaf...
Improving	
  Diagnosis	
  
•  Reiter’s	
  “black-­‐box”	
  (model-­‐based)	
  diagnosis	
  
helps	
  debug	
  the	
  arfcu...
Key	
  Idea:	
  exploit	
  white-­‐box	
  info	
  
•  We	
  use	
  Answer	
  Set	
  Programming	
  (ASP)	
  to	
  solve	
 ...
The	
  Provenance	
  “Trick”	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  Prague	
   25	
  
Hybrid	
  Provenance	
  
A3:	
  	
  c	
  <	
  f	
  
Black-­‐box	
  Provenance	
  
1.a
1.b
isa
1.c
isa
2.d
=
2.e<
<
2.f<
is...
Hybrid	
  Provenance	
  
A3:	
  	
  c	
  <	
  f	
  
Black-­‐box	
  Provenance	
  
r7: d = e ∪ f
a = e ∪ f
A1: a = d
r3: a ...
The	
  Hybrid	
  Approach	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  Prague	
   28	
  
Hybrid	
  Approach	
  
Hybrid	
  Diagnosis	
  in	
  Euler	
  	
  @	
  	
  RuleML2014,	
  Prague	
   29	
  
What	
  ar5cula...
Benchmark	
  
Results	
  	
  
•  White-­‐box	
  <	
  Hybrid	
  <	
  Black-­‐box	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  (r...
Benchmark	
  
DLV	
  
•  White-­‐box	
  <	
  Hybrid	
  <	
  Black-­‐box	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  (runfmes)	...
Benchmark	
  
Clingo	
  
•  White-­‐box	
  <	
  Hybrid	
  <	
  Black-­‐box	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  (runfme...
Conclusions	
  
•  ASP	
  rules	
  can	
  be	
  used	
  to	
  efficiently	
  solve	
  real-­‐
world	
  taxonomy	
  reasoning...
Upcoming SlideShare
Loading in …5
×

Bertram's talk on Hybrid (Black-box + White-box) Diagnosis at RuleML'14 in Prague

1,823 views

Published on

Your logical axioms might be inconsistent?
Use this approach to speed up your model-based diagnosis!

Presented at RuleML'14 in August in Prague (co-located with ECAI 2014).

Published in: Data & Analytics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,823
On SlideShare
0
From Embeds
0
Number of Embeds
1,065
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Bertram's talk on Hybrid (Black-box + White-box) Diagnosis at RuleML'14 in Prague

  1. 1. A  Hybrid  Diagnosis  Approach  Combining   Black-­‐Box  and  White-­‐Box  Reasoning     Mingmin  Chen1                                  Shizhuo  Yu1           Nico  Franz2              Shawn  Bowers3              Bertram  Ludäscher1  4       1  Department  of  Computer  Science  ,  University  of  California,  Davis   2  School  of  Life  Sciences,  Arizona  State  University   3  Department  of  Computer  Science,  Gonzaga  University   4  GSLIS  &  NCSA,  University  of  Illinois  at  Urbana-­‐Champaign  
  2. 2. Outline   •  The  Taxonomy  Alignment  Problem   T1  +  T2  +  A  =>  T3                      (ambiguous  ..  unique  ..  inconsistent)   •  Model-­‐based  Diagnosis  [Reiter’87]   – Black-­‐box   •  Hybrid  Approach   – Black-­‐box  &  White-­‐box  combined   •  Benchmark  Results   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   2  
  3. 3. Meet  Prof.  Nico  Franz:      Curator  of  Insects  @  ASU   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   3  
  4. 4. What  Nico  et  al.  do  for  a  living  …     Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   4  
  5. 5.    What  Nico  does  for  a  living  (cont’d):   The  Indoors  Part   •  First:  go  fun  places,  find  new  bugs,  study  them  …   –  “Bugs-­‐R-­‐Us”  (see  taxonbytes.org)   •  Now:  Compare,  align  and  revise  taxonomies,  based  on   careful  observafon,  “character”  data,  experfse  …       •  Formally:   –  Input:  T1  +  T2  (taxonomies)  +  A  (expert  ar.cula.ons)     –  Output:  revised,  “merged”  taxonomy  (-­‐ies)  T3   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   5  
  6. 6. •  Given: –  Taxonomies T1 , T2 •  incl. constraints (coverage, disjointness) –  Set of articulations (alignment) A •  Find: –  Combined (“merged”) taxonomy T3 (= T1 + T2 + A) •  Is it a taxonomy? Or a DAG? –  Optional: •  Final alignment (should be minimal) 6   Taxonomy Alignment Problem (TAP) T1   T2   A   T3  
  7. 7.    TAP:  Possible  Outcomes   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   7  
  8. 8.    TAP:  Possible  Outcomes   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   {A1, A2, A3, A4} {A1, A2, A3} {A1, A2, A4} {A1, A3, A4} {A2, A3, A4} {A1, A2} {A1, A3} {A2, A3} {A1, A4} {A2, A4} {A3, A4} {A1} {A2} {A3} {A4} { } Inconsistent!       è  Diagnosis  (Reiter)     =  Black-­‐Box    Provenance   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   8  
  9. 9.  TAP:  Possible  Outcomes   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   {A1, A2, A3, A4} {A1, A2, A3} {A1, A2, A4} {A1, A3, A4} {A2, A3, A4} {A1, A2} {A1, A3} {A2, A3} {A1, A4} {A2, A4} {A3, A4} {A1} {A2} {A3} {A4} { } Inconsistent!       è  Diagnosis  (Reiter)     =  Black-­‐Box    Provenance   1.b2.e 1.c 1.a 2.d 2.f Ambiguous!     è  Mul5ple   Possible     Worlds   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   9  
  10. 10.  TAP:  Possible  Outcomes   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   {A1, A2, A3, A4} {A1, A2, A3} {A1, A2, A4} {A1, A3, A4} {A2, A3, A4} {A1, A2} {A1, A3} {A2, A3} {A1, A4} {A2, A4} {A3, A4} {A1} {A2} {A3} {A4} { } Inconsistent!       è  Diagnosis  (Reiter)     =  Black-­‐Box    Provenance   1.b2.e 1.c 1.a 2.d 2.f Ambiguous!     è  Mul5ple   Possible     Worlds   1.c2.f 1.b 1.a 2.d 2.e Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   10  
  11. 11.  TAP:  Possible  Outcomes   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   {A1, A2, A3, A4} {A1, A2, A3} {A1, A2, A4} {A1, A3, A4} {A2, A3, A4} {A1, A2} {A1, A3} {A2, A3} {A1, A4} {A2, A4} {A3, A4} {A1} {A2} {A3} {A4} { } Inconsistent!       è  Diagnosis  (Reiter)     =  Black-­‐Box    Provenance   1.b2.e 1.c 1.a 2.d 2.f Ambiguous!     è  Mul5ple   Possible     Worlds   1.c2.f 1.b 1.a 2.d 2.e 1.b 1.a 2.e 2.d 1.c 2.f Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   11  
  12. 12. •  FO  reasoning  about   taxonomies  (MFOL)   •  Earlier:  CleanTax     –  Prover9/Mace4   •  Now:  Euler   –  ASP  Reasoners  (DLV,   Clingo)   –  Specialized  reasoners   (PyRCC)   –  …     –  X  =  ASP,  RCC,  …     Euler/X  Toolkit  and  Workflow     Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   12  
  13. 13. Real-­‐world  examples:  Turn  this  …     Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   13  
  14. 14. … into this! (Perellescus Alignment Result) •  T3 := T1 and T2 are “merged” –  Blue dashed: overlaps è resolve via “zoom-in view” 14   1.16 1.14 2.40 2.44 2.47 1.11 2.38 2.35 1.20 1.23 2.52 2.53 2.54 1.17 2.41 1.22 2.46 1.25 2.48 1.12 2.36 1.26 2.49 1.13 2.37 1.18 2.42 1.19 2.43 1.15 2.39 1.21 2.45 1.12L 2.36L 1.27 2.50 1.24 2.51 Nodes Taxonomy 1 5 Taxonomy 2 8 MERGED Taxa 13 Edges Overlaps 10 Input 24 INFERRED 5
  15. 15.    Possible  Outcome:  Inconsistency!   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   {A1, A2, A3, A4} {A1, A2, A3} {A1, A2, A4} {A1, A3, A4} {A2, A3, A4} {A1, A2} {A1, A3} {A2, A3} {A1, A4} {A2, A4} {A3, A4} {A1} {A2} {A3} {A4} { } Inconsistent!       è  Diagnosis  (Reiter)     =  Black-­‐Box    Provenance   •  Need  to  debug  the   input  arfculafons  è   (black-­‐box)  diagnosis!   •  Focus:   –  How  do  we  efficiently   compute  the  diagnosfc   lance?   •  Also:   –  How  to  visualize..     Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   15  
  16. 16. Example  Instance     (from  synthefc  benchmark  suite)   •  Here:  N  =  10  taxa  in  T1,  T2   •  Euler/X  finds:   inconsistent!   •  è  diagnosc  la]ce  of  210   =  1024  nodes   è Find  minimal  inconsistent   subset  (MIS)     è maximal  consistent  subset   (MCS)  ..     è show  to  user!   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   16  
  17. 17. Visualizing  Diagnoses:  Scalability   Issues   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   17   N  =  10  arfculafons  è  210  =  1024  node  diagnosfc  lance  (…  ouch!)            …  but  only  one  MIS  …          
  18. 18. Beaer  Idea:  Just  show  MIS,  MCS   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   18   N  =  4  arfculafons  è  24  =  16  node  diagnosfc  lance,   but  3  MCS  and  2  MIS  are  enough!  
  19. 19. ..  but  4  MCS  and   1  MIC  tell  it  all!   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   19   1024  node  lance   Visualizing  Diagnoses:  Focusing  on   MIS  (and  MCS)  
  20. 20. Visualizing  Diagnoses   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   20   Example  from   paper:  N=12  è   4096  nodes     ..  but  7  MCS  and     5  MIC  tell  it  all!  
  21. 21. Black-Box Inconsistency Analysis (Diagnostic Lattice) •  Then: –  Repair: find & revise minimal inconsistent subsets (Min-Incons) –  Expand: find maximal consistent subsets (Max-Cons) & revise outs What happens if you can’t have all (here: 4) articulations together? 21  
  22. 22. •  Black-­‐box  Analysis  (Hinng  Set  algo.)  yields  a  Diagnosis  (lance)   –  for  n=4  arfculafons,  there  are  168  possible  diagnoses   –  depending  on  expected  “red/green  areas”  è  explore  space  differently   •  |arfculafons|  =  n  è        |possible  diagnoses|  =  |monotonic  Boolean  funcfons|        =  Dedekind  Number  (n):    2,  3,  6,  20,  168,  7581,  7828354,  ...       Inconsistency Analysis (Diagnostic Lattice) •  The Min-Incons (MIS) and Max-Cons (MCS) sets determine all others è  Repair MIS and/or Expand MCS 22
  23. 23. Improving  Diagnosis   •  Reiter’s  “black-­‐box”  (model-­‐based)  diagnosis   helps  debug  the  arfculafons   •  Limited  scalability  (inherent  complexity)   •  But  every  bit  helps:   – Hinng  Set  Algorithm  (“logarithmic  extracfon”)   •  Our  idea:   – Exploit  “white-­‐box”  reasoning  informafon   è  RULES  to  the  rescue   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   23  
  24. 24. Key  Idea:  exploit  white-­‐box  info   •  We  use  Answer  Set  Programming  (ASP)  to  solve   Taxonomy  Alignment  Problem  (TAP)   •  Inconsistency  =  “False”  is  derived  in  the  head:   False  :-­‐      <denial  of  integrity  constraint>     •  Apply  provenance  trick  from  databases  J     –  What  arfculafons  contribute  to  a  derivafon  of   “False”  ?     –  Eliminate  those  that  don’t!   è  an  example  of  reusing  inferences  across  separate   black-­‐box  tests!   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   24  
  25. 25. The  Provenance  “Trick”   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   25  
  26. 26. Hybrid  Provenance   A3:    c  <  f   Black-­‐box  Provenance   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   26  
  27. 27. Hybrid  Provenance   A3:    c  <  f   Black-­‐box  Provenance   r7: d = e ∪ f a = e ∪ f A1: a = d r3: a = b ∪ c f < c r4: b ∩ c = ∅ r8: e ∩ f = ∅ A2: b < e A1+A2  +  …  =>  f  <  c   White-­‐box     Provenance   1.a 1.b isa 1.c isa 2.d = 2.e< < 2.f< isa isa Input  Alignment   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   27  
  28. 28. The  Hybrid  Approach   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   28  
  29. 29. Hybrid  Approach   Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   29   What  ar5cula5ons   contribute  to  some   inconsistency?     Good  old  black-­‐box   (HST)  
  30. 30. Benchmark   Results     •  White-­‐box  <  Hybrid  <  Black-­‐box                    (runfmes)   •  Note:  white-­‐box  does  not  give  you  a  diagnosis   •  Potassco  <  DLV       Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   30  
  31. 31. Benchmark   DLV   •  White-­‐box  <  Hybrid  <  Black-­‐box                    (runfmes)   •  Potassco  <  DLV       Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   31  
  32. 32. Benchmark   Clingo   •  White-­‐box  <  Hybrid  <  Black-­‐box                    (runfmes)   •  Potassco  <  DLV       Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   32  
  33. 33. Conclusions   •  ASP  rules  can  be  used  to  efficiently  solve  real-­‐ world  taxonomy  reasoning  problems   •  Reiter’s  diagnosis  useful  to  debug  inconsistent   alignments   •  Adding  a  “white-­‐box”  provenance  approach   speeds  up  state-­‐of-­‐the-­‐art  HST  algorithm  by   eliminang  independent  arculaons   •  Future  work:   –  Further  improvements,  including  parallelism:   •  Trade-­‐off  with  sharing  inferences  across  parallel  instances     Hybrid  Diagnosis  in  Euler    @    RuleML2014,  Prague   33  

×