The document describes a framework for explaining the output of function circuits. It introduces the concept of functions as basic processing units that take inputs and produce outputs. Functions can be composed into circuits where the output of one function is the input of another. The framework represents circuits using designation graphs that trace the propagation of values between function inputs and outputs. It defines derivation operators that take a designation graph and reconstruct the derivation of an output value from input values through the functions in the circuit. This allows explaining how a circuit produced a particular output from its inputs.

1. Sylvain Hallé & Hugo Tremblay
Sylvain Hallé
Hugo Tremblay
Université du Québec à Chicoutimi
CANADA
A Generic Explainability Framework
for Function Circuits
CRSNG
NSERC

2. Sylvain Hallé & Hugo Tremblay
+
Functions
Basic processing unit: function

3. Sylvain Hallé & Hugo Tremblay
+
Functions
Basic processing unit: function
input pin(s)
output pin(s)

4. Sylvain Hallé & Hugo Tremblay
+
Functions
Basic processing unit: function
Given input arguments, a function produces output
values

5. Sylvain Hallé & Hugo Tremblay
+
Functions
Basic processing unit: function
Given input arguments, a function produces output
values
3
2
5

6. Sylvain Hallé & Hugo Tremblay
A function can be "anything"; it can be
parameterized by another function
Functions
α
2

7. Sylvain Hallé & Hugo Tremblay
A function can be "anything"; it can be
parameterized by another function
Functions
α
2
apply on each element square

8. Sylvain Hallé & Hugo Tremblay
A function can be "anything"; it can be
parameterized by another function
Functions
α
2
[4,2,5] [16,4,25]

9. Sylvain Hallé & Hugo Tremblay
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

10. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

11. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
2
6
3
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

12. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
2
6
3
6
8
3
18
6
6
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

13. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
2
6
3
6
8
3
18
6
6
6
33
8
18
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

14. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
2
6
3
6
8
3
18
6
6
6
33
8
18
⊥
⊤
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

15. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
2
6
3
6
8
3
18
6
6
6
33
8
18
⊥
⊤
⊤
⊥
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

16. Sylvain Hallé & Hugo Tremblay
Outputs of "upstream" functions become inputs of
"downstream" functions
2
6
3
6
8
3
18
6
6
6
33
8
18
⊥
⊤
⊤
⊥
⊤
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

17. Sylvain Hallé & Hugo Tremblay
A circuit can be encapsulated into a function
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

18. Sylvain Hallé & Hugo Tremblay
A circuit can be encapsulated into a function
x
y
z
t
(x + y > y × z) ∨ (y > 3)
>
+
×
>
>
3
Function circuits
Functions can be composed to form circuits

19. Sylvain Hallé & Hugo Tremblay
α
>
3
Parameterized functions result in multiple sub-
evaluations of a circuit
All together now

20. Sylvain Hallé & Hugo Tremblay
α
>
3
Parameterized functions result in multiple sub-
evaluations of a circuit
All together now
[4,2,5]

21. Sylvain Hallé & Hugo Tremblay
α
>
3
Parameterized functions result in multiple sub-
evaluations of a circuit
All together now
[4,2,5]
>
3
4
44
3
T
T3
T[ ]

22. Sylvain Hallé & Hugo Tremblay
α
>
3
Parameterized functions result in multiple sub-
evaluations of a circuit
All together now
[4,2,5]
>
3
4
44
3
T
T3
T[ ]
>
3
2
2
3
T
T
3
T

23. Sylvain Hallé & Hugo Tremblay
α
>
3
Parameterized functions result in multiple sub-
evaluations of a circuit
All together now
[4,2,5]
>
3
4
44
3
T
T3
T[ ]
>
3
2
2
3
T
T
3
T
>
3
5
5
3
T
T3
T

24. Sylvain Hallé & Hugo Tremblay
Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...

25. Sylvain Hallé & Hugo Tremblay
Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...
a relation R

26. Sylvain Hallé & Hugo Tremblay
Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...
a relation R
a part of R
⊆

27. Sylvain Hallé & Hugo Tremblay
Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...
a relation R
a part of R
⊆
a tuple t
a=v
a=v
a=v
a=v
a=v
a=v
a=v

28. Sylvain Hallé & Hugo Tremblay
Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...
a relation R
a part of R
⊆
a tuple t
a=v
a=v
a=v
a=v
a=v
a=v
a=v
a part of t
⊆

29. Sylvain Hallé & Hugo Tremblay
Part of
Most provenance models implicitly involve the
concept of "part of"
Straightforward in the relational world...
a relation R
a part of R
⊆
a tuple t
a=v
a=v
a=v
a=v
a=v
a=v
a=v
a part of t
⊆
What if the universe of discourse is not only made
of sets?

30. Sylvain Hallé & Hugo Tremblay
Our proposed model is based on an abstract
topology of objects
Topology of objects
𝖀 is a set of objects
⊑ is a partial order between objects ("part of")
□ is the only object that is part of all others
We suppose that ⊑ follows the intuition for common
objects:
scalars have no parts but themselves
a part of a set is a subset or a single element
a part of a string (list) is a sub-string (-list)

31. Sylvain Hallé & Hugo Tremblay
A designator is any function that takes an object
and returns a part of this object. Examples:
Designator
[i] the i-th item of a list
[i : j] the substring between characters i and j
the contents of some file
the i-th input pin of a function
the i-th output pin of a function
↑i
↓i

32. Sylvain Hallé & Hugo Tremblay
Designators can be composed:
↑2 [3] [1:5]◦ ◦
A designator is any function that takes an object
and returns a part of this object. Examples:
Designator
[i] the i-th item of a list
[i : j] the substring between characters i and j
the contents of some file
the i-th input pin of a function
the i-th output pin of a function
↑i
↓i

33. Sylvain Hallé & Hugo Tremblay
Designators can be composed:
↑2 [3] [1:5]◦ ◦
"The first five characters of the third
item of the second argument of the
function"
A designator is any function that takes an object
and returns a part of this object. Examples:
Designator
[i] the i-th item of a list
[i : j] the substring between characters i and j
the contents of some file
the i-th input pin of a function
the i-th output pin of a function
↑i
↓i

34. Sylvain Hallé & Hugo Tremblay
Designation graph
∧
↓ , f1
< <
↑ , f1< <
∧
[1],↑ , f2< < [1],↑ , f3< <
A designation graph is an and-or directed acyclic
graph (DAG) where:
the endpoints are made of a designator d and a
function f
the head of d points to an input or an output of f
It expresses an "and-or"
relationship between
function inputs and
function outputs in a
circuit.

35. Sylvain Hallé & Hugo Tremblay
Given a function f with m inputs and n outputs, a
derivation operator is a function ∆f:
Derivation
∆f : 𝕯 × 𝖀m
× 𝖀n
→ 𝕿

36. Sylvain Hallé & Hugo Tremblay
Given a function f with m inputs and n outputs, a
derivation operator is a function ∆f:
Derivation
∆f : 𝕯 × 𝖀m
× 𝖀n
→ 𝕿
designator function
input
function
output
designation
graph

37. Sylvain Hallé & Hugo Tremblay
Given a function f with m inputs and n outputs, a
derivation operator is a function ∆f:
Derivation
∆f : 𝕯 × 𝖀m
× 𝖀n
→ 𝕿
The graph must be such that:
its root is the designator d given to ∆f
if d's head points to an input of f, the leaves
point to an output of f (and vice versa)
designator function
input
function
output
designation
graph

38. Sylvain Hallé & Hugo Tremblay
Given a circuit an input/output
pair and a designator, one can
reconstruct the derivation for the
circuit by connecting the ends of
the derivation graph for each
individual function.
Derivation for circuits
>
+
×
>
>
3

39. Sylvain Hallé & Hugo Tremblay
Given a circuit an input/output
pair and a designator, one can
reconstruct the derivation for the
circuit by connecting the ends of
the derivation graph for each
individual function.
Derivation for circuits
>
+
×
>
>
3
⊤
2
6
3

40. Sylvain Hallé & Hugo Tremblay
Given a circuit an input/output
pair and a designator, one can
reconstruct the derivation for the
circuit by connecting the ends of
the derivation graph for each
individual function.
Derivation for circuits
>
+
×
>
>
3
⊤
2
6
3
∧
↓ ,1
< <
>
↑ ,1< <
>
∧
↓ ,1
< < >
∧
↑ ,1< < > ↑ ,2< < >
∧
↓ ,2
< <
∧
↓ ,2
< < 3

41. Sylvain Hallé & Hugo Tremblay
Given a circuit an input/output
pair and a designator, one can
reconstruct the derivation for the
circuit by connecting the ends of
the derivation graph for each
individual function.
Derivation for circuits
>
+
×
>
>
3
⊤
2
6
3
∧
↓ ,1
< <
>
↑ ,1< <
>
∧
↓ ,1
< < >
∧
↑ ,1< < > ↑ ,2< < >
∧
↓ ,2
< <
∧
↓ ,2
< < 3
6 3

42. Sylvain Hallé & Hugo Tremblay
Given a circuit an input/output
pair and a designator, one can
reconstruct the derivation for the
circuit by connecting the ends of
the derivation graph for each
individual function.
Derivation for circuits
>
+
×
>
>
3
⊤
2
6
3
∧
↓ ,1
< <
>
↑ ,1< <
>
∧
↓ ,1
< < >
∧
↑ ,1< < > ↑ ,2< < >
∧
↓ ,2
< <
∧
↓ ,2
< < 3
6 3

43. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

44. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
split file
into lines

45. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
on each
line...

46. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
split on
commas

47. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
take
second
element

48. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
convert to
number

49. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
on each
window of
3 values...

50. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
take the
average

51. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
on each
average...

52. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
check that is
is greater than 3

53. Sylvain Hallé & Hugo Tremblay
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example
assert they
are all so

54. Sylvain Hallé & Hugo Tremblay
⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

55. Sylvain Hallé & Hugo Tremblay
⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
∧
↓ , G1< <
∧ ∧∧
[4:5], [3] ,< < [5:5], [4] ,< < [6:8], [5] ,< < [8:8], [7] ,< <[ : ], [6] ,< <12 12
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

56. Sylvain Hallé & Hugo Tremblay
⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
∧
↓ , G1< <
∧ ∧∧
[4:5], [3] ,< < [5:5], [4] ,< < [6:8], [5] ,< < [8:8], [7] ,< <[ : ], [6] ,< <12 12
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

57. Sylvain Hallé & Hugo Tremblay
⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
∧
↓ , G1< <
∧ ∧∧
[4:5], [3] ,< < [5:5], [4] ,< < [6:8], [5] ,< < [8:8], [7] ,< <[ : ], [6] ,< <12 12
∧
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

58. Sylvain Hallé & Hugo Tremblay
⊤the,2,penny
fool,7,lane
on,18,come
the,2,together
hill,-80,i
strawberry,7,am
fields,1,the
forever,10,walrus
∧
↓ , G1< <
∧ ∧∧
[4:5], [3] ,< < [5:5], [4] ,< < [6:8], [5] ,< < [8:8], [7] ,< <[ : ], [6] ,< <12 12
∧
/, [2] #
α W
3
x
Gα
>
3
1 2
A
C
3 4
B
5
a b c
a
b
A bigger example

59. Sylvain Hallé & Hugo Tremblay
Observations
Pros
generic computation model (circuits)
not limited to tuples and relations
modular (define ∆f separately for each f)
support for alternate lineage ("or" nodes)
fine granularity (parts of objects)
same algorithm, different lineage depending on
definition of ∆f for each function f
Cons
who defines ∆f ? according to what requirements ?
circuits: a way to program, or a model to reason
about a program ?

60. Sylvain Hallé & Hugo Tremblay
Future work
Provide formal definitions of ∆ that correspond
to various (existing) provenance relationships
Implement and test a proof of concept
(under way: https://github.com/liflab/petitpoucet)
Retrofit this framework into existing software
(BeepBeep, Cornipickle)
Study circuits according to the designation
graphs they induce
Thank you!