1+

=3
f(x) = ax2 + bx + c

1+

=3
f(x) = ax2 + bx + c

1+

=3
1+

=3
f(x) = ax2 + bx + c

1+

=3
Why do we
need to know
this, man?

f(x) = ax2 + bx + c

1+

=3
Why do we
need to know
this, man?

Details
Big Picture
Why do we
need to know
this, man?

Details
Big Picture

Details
Big Picture

Details
Big Picture

Details
Big Picture

Details
Big Picture

Cool!
Details
Communities of Practice
culture
culture
culture
culture
culture
culture
Epistemic Frame
Games are cultures
R

eflection-in-action
T

thinking that reshapes
what we are doing
while we are doing it
thinking that reshapes
what we are doing
while we are doing it

K

ways of
knowing
D

ways of
doing

thinking that reshapes
what we are doing
while we are doing it

ways of
knowing
ways of
doing

P

practicum

ways of
knowing
A

action

practicum

R

reflectionon-action
action

practicum

reflectionon-action
action
practicum

reflectionon-action
practicum

R

eflection-in-action
practicum
100%
90%
80%
70%

60%
50%
40%
30%
20%
10%
0%
Year 1

Year 2

Year 3
Women like
teamwork!
d = + 0.28
d = + 0.28
d = + 0.91
d = + 0.91
We value what we can
measure
—Don Norman
We^value what we can
measure
Skills

Knowledge

Identity

Values

Epistemology
Skills

Knowledge

Identity

Values

Epistemology
Epistemic Network Analysis
(ENA)
B

A
B

A
B

C
A

D
E

B

C

F
D

A

©2005, 2008 David Williamson Shaffer
E
B

A

C

D

F
B

C

D

A

F

E

©2005, 2008 David Williamson Shaffer
B
F
D C

A
E
A

B
C

A

B

©2005, 2008 David Williamson Shaffer
C
F

A

B

©2005, 2008 David Williamson Shaffer
C
F
D
A

B

©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F
D
Epistemology of Design

B

E
©2005, 2008 David Williamson Shaffer
C
Skill of Data Analysis

D
Epistemology of Design

B

E
©2005, 2008 David Williamson Shaffer
C
Skill of Data Analysis

D

Valuing the Client
Epistemology of Design

E
©2005, 2008 David Williamson Shaffer
Identity as Engineer
Skill of Data Analysis

Knowledge of Materials

Valuing the Client
Epistemology of Design

Skill of Compromising
©2005, 2008 David Williamson Shaffer
Identity as Engineer
Skill of Data Analysis

Knowledge of Materials

Valuing the Client
Epistemology of Design

Skill of Compromising
©2005, 2008 David Williamson Shaffer
Identity as Engineer
Skill of Data Analysis

Knowledge of Materials

Valuing the Client
Epistemology of Design

Skill of Compromising
©2005, 2008 David Williamson Shaffer
Identity as Engineer
Experienced
Engineers

Skill of Data Analysis

Knowledge of Materials

Valuing the Client
Epistemology of Design

Skill of Compromising
©2005, 2008 David Williamson Shaffer
Identity as Engineer
Experienced
Engineers

Skill of Data Analysis
First-year
Students

Knowledge of Materials

Valuing the Client
Epistemology of Design

Skill of Compromising
©2005, 2008 David Williamson Shaffer
Chat Discourse
Utterance
1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Epistemology of Design

1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Value of Client
1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Skill of Data Analysis
1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza
B
F
D

A

E

C

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0
Stanza
E6
V4

S5
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

K1

I2

K3
Stanza
E6
V4

S5
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

K1

I2

K3
E6
V4

S5
K1

I2

K3
E6
V4

S5
K1

I2

K3
d = + 0.28
Collaboration
Collaboration

Design
Collaboration

Data

Design
Collaboration

Data

Design
Design

Data
Collaboration
Design

Data
Collaboration
Design

Data
Collaboration
Design

Data
Collaboration
Civics?

Design

Data
Collaboration
G
T

ames to
each
G
C

ames are
ultures
G
C

ames can create
ultures of thinking
epistemicgames.org
epistemicgames.org

@epistemicgames
Epistemic Games
Chat Discourse
Utterance
1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Epistemology of Design

1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Value of Client
1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Skill of Data Analysis
1
2
3
4
5
6
7
8

E6 V4 S5
1 0 0
0 0 0
0 0 0
1 0 1
1 0 0
0 0 0
0 0 0
0 0 0

I2 K1K2K3
0 1 0 0
0 1 1 0
0 0 0 0
0 1 0 0
0 0 0 0
0 0 0 1
0 1 0 0
1 0 0 1
Semantic Coding
AI Complete Problem
Structural Functional Linguistic Codes
Skills

Knowledge

Identity

Values

Epistemology
Structural Functional Linguistic Codes

Domain codes

Skills

Knowledge

Identity

Values

Epistemology
Domain codes

Structural Functional Linguistic Codes
Domain codes

Structural Functional Linguistic Codes

Conjunctive Coding
Domain codes

Structural Functional Linguistic Codes
Structural Functional Linguistic Codes

Domain codes

Value of Surfactants
Structural Functional Linguistic Codes

Domain codes

Value of Surfactants
Domain codes

Structural Functional Linguistic Codes

Engineering Identity
Domain codes

Structural Functional Linguistic Codes

Engineering Identity
Human Agreement with AutoCoder

E and AC

E/Data
E/Design
E/Clients
E/Consultants
V/Clients
V/Consultants
S/Data
S/Design
S/Professionalism
S/Collaboration
I/Engineer
I/Intern
K/Nanotech
K/Surfactants
K/Attributes
K/Data
K/Design
K/Clients
K/Materials
K/Processes

N1 and N2 E1 and E2

0.521463
0.64044
0.746846
0.764168
0.538768
0.493692
0.76369
0.801079
0.645465
0.694157
0.479239
0.921389
1
0.95827
0.981197
0.63988
0.728741
1
0.637786
0.364711

0.321308
0.479389
0.796759
0.566491
0.663687
0.663311
0.615417
0.466942
0.456452
0.666465
0.824869
0.816862
0.796759
0.965993
0.900837
0.784384
0.771417
0.945658
0.916048
0.811285

0.697609
0.54518
0.746846
0.853813
0.663311
0.663311
0.940922
0.728728
0.72107
0.888954
0.560584
0.853813
1
0.919605
0.990646
0.869369
0.774949
1
0.984033
0.95827
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza
Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Stanza

Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Codes by Utterance
E6 V4 S5 I2 K1K2K3
1 1 0 0 0 1 0 0
2 0 0 0 0 1 1 0
3 0 0 0 0 0 0 0
4 1 0 1 0 1 0 0
5 1 0 0 0 0 0 0
6 0 0 0 0 0 0 1
7 0 0 0 0 1 0 0
8 0 0 0 1 0 0 1
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Adjacency Matrix for the Stanza
E6 V4 S5 I2 K1 K2 K3
E6 0 0 1 0 1 0 0
V4 0 0 0 0 0 0 0
S5 1 0 0 0 1 0 0
I2
0 0 0 0 0 0 0
K1 1 0 1 0 0 0 0
K2 0 0 0 0 0 0 0
K3 0 0 0 0 0 0 0
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Adjacency Matrix for the Stanza
E6 V4 S5 I2 K1 K2 K3
E6 0 0 1 0 1 0 0
V4 0 0 0 0 0 0 0
S5 1 0 0 0 1 0 0
I2
0 0 0 0 0 0 0
K1 1 0 1 0 0 0 0
K2 0 0 0 0 0 0 0
K3 0 0 0 0 0 0 0
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Adjacency Matrix for the Stanza
E6 V4 S5 I2 K1 K2 K3
E6 0 0 1 0 1 0 0
V4 0 0 0 0 0 0 0
S5 1 0 0 0 1 0 0
I2
0 0 0 0 0 0 0
K1 1 0 1 0 0 0 0
K2 0 0 0 0 0 0 0
K3 0 0 0 0 0 0 0
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Adjacency Matrix for the Stanza
E6 V4 S5 I2 K1 K2 K3
E6 0 0 1 0 1 0 0
V4 0 0 0 0 0 0 0
S5 1 0 0 0 1 0 0
I2
0 0 0 0 0 0 0
K1 1 0 1 0 0 0 0
K2 0 0 0 0 0 0 0
K3 0 0 0 0 0 0 0
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Adjacency Matrix for the Stanza
E6 V4 S5 I2 K1 K2 K3
E6 0 0 1 0 1 0 0
V4 0 0 0 0 0 0 0
S5 1 0 0 0 1 0 0
I2
0 0 0 0 0 0 0
K1 1 0 1 0 0 0 0
K2 0 0 0 0 0 0 0
K3 0 0 0 0 0 0 0
Codes by Stanza
E6 V4 S5 I2 K1K2K3
1 0 1 0 1 0 0

Adjacency Matrix for the Stanza
E6 V4 S5 I2 K1 K2 K3
E6 0 0 1 0 1 0 0
V4 0 0 0 0 0 0 0
S5 1 0 0 0 1 0 0
I2
0 0 0 0 0 0 0
K1 1 0 1 0 0 0 0
K2 0 0 0 0 0 0 0
K3 0 0 0 0 0 0 0
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
C
F

Cumulative Adjacency Matrix

D
A

B

E
C
F

Cumulative Adjacency Matrix

D
A

B

E
C
F

Cumulative Adjacency Matrix

D
A

B

E
C
F

Cumulative Adjacency Matrix

D
A

B

E
C
F

Cumulative Adjacency Matrix

D
A

B

E
Cumulative Adjacency Matrix
C
F
D
A

B

E
©2005, 2008 David Williamson Shaffer
Frontiers of Engineering Education
Frontiers of Engineering Education
Frontiers of Engineering Education
Frontiers of Engineering Education
Frontiers of Engineering Education

Frontiers of Engineering Education

Editor's Notes

  • #119 As we saw earlier, in Nephrotex, students communicate with their team members and their design advisor through an internal chat program.
  • #120 We segment the chat data by utterance, everytime a person types something and hits send. And we code each utterance for epistemic engineering frame elements.
  • #121 Every utterance was represented as a row in a matrix and each column is a code. The coding scheme was based on professional elements in the engineering epistemic frame using previous ethnographic studies and ABET criteria. For example,
  • #122 This first code represents epistemology of design– every time a student is justifying a design decision.
  • #123 This code represents valuing the client’s needs
  • #124 And this code represents the skill of data analysis. There were actually 21 codes in this analysis but for the sake of explaining the ENA method I have selected a few as examples.
  • #125 This utterance is coded for E6 and K1
  • #126 This utterance is coded for K1 and K2
  • #127 And this utterance is coded for E6, S5, and K1
  • #128 Now we can segment the data further and decide that this set of utterances is a stanza. We make the argument that everything that occurs with in this collection of utterances is linked. The discourse is all about one topic. And then, we collapse over these stanzas and do a binary sum.
  • #129 Now we can segment the data further and decide that this set of utterances is a stanza. We make the argument that everything that occurs with in this collection of utterances is linked. The discourse is all about one topic. And then, we collapse over these stanzas and do a binary sum.
  • #130 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #131 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #132 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #133 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #134 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #135 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #136 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #137 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #190 As we saw earlier, in Nephrotex, students communicate with their team members and their design advisor through an internal chat program.
  • #191 We segment the chat data by utterance, everytime a person types something and hits send. And we code each utterance for epistemic engineering frame elements.
  • #192 Every utterance was represented as a row in a matrix and each column is a code. The coding scheme was based on professional elements in the engineering epistemic frame using previous ethnographic studies and ABET criteria. For example,
  • #193 This first code represents epistemology of design– every time a student is justifying a design decision.
  • #194 This code represents valuing the client’s needs
  • #195 And this code represents the skill of data analysis. There were actually 21 codes in this analysis but for the sake of explaining the ENA method I have selected a few as examples.
  • #209 So, let’s take a closer look at the data.
  • #210 This utterance is coded for E6 and K1
  • #211 This utterance is coded for K1 and K2
  • #212 And this utterance is coded for E6, S5, and K1
  • #213 Now we can segment the data further and decide that this set of utterances is a stanza. We make the argument that everything that occurs with in this collection of utterances is linked. The discourse is all about one topic. And then, we collapse over these stanzas and do a binary sum.
  • #214 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #215 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #216 So, if it occurs within the stanza, then it gets a 1 and if not then a 0.
  • #217 Let’s take this stanza as a example for the next step of the analysis. It was coded for three codes. We want to measure how players are making connections with elements and so we look at when the codes are occurring together.
  • #218 The completed adjacency matrix for that stanza
  • #219 The completed adjacency matrix for that stanza
  • #220 The completed adjacency matrix for that stanza
  • #221 The completed adjacency matrix for that stanza
  • #222 The completed adjacency matrix for that stanza
  • #223 The completed adjacency matrix for that stanza
  • #224 The completed adjacency matrix for that stanza
  • #225 The completed adjacency matrix for that stanza
  • #226 The completed adjacency matrix for that stanza
  • #227 The completed adjacency matrix for that stanza
  • #229 The completed adjacency matrix for that stanza
  • #230 The completed adjacency matrix for that stanza
  • #231 The completed adjacency matrix for that stanza
  • #232 The completed adjacency matrix for that stanza
  • #233 The completed adjacency matrix for that stanza
  • #235 The completed adjacency matrix for that stanza