IMPACCT Created & Adapted by Spitzberg (2005) Interactive Package for the Assessment of Communication & Critical Thinknig

Online Package for Assessment of Communication and Critical Thinking (Online PACCT) What: Menu-driven, feedback-generating questionnaire When: Proof of concept this Spring Where: Available on SDSU, and eventually elsewhere Who: College freshmen, and potentially H.S. seniors Why: (A) Institutional Advantages (B) Student Advantages

Online Package for Assessment of Communication and Critical Thinking (Online PACCT)

What: Menu-driven, feedback-generating questionnaire

When: Proof of concept this Spring

Where: Available on SDSU, and eventually elsewhere

Who: College freshmen, and potentially H.S. seniors

Why:

(A) Institutional Advantages

(B) Student Advantages

ASSESSING THE ASSESSMENT: Face/Ecological/Representational Construct (convergent + discriminant) validity: + with + (e.g., skills measures) - with - (e.g., anxiety measures) 0 with 0 (e.g., U.S. ethnicity) 360° Assessment (i.e., network consensus) Delta (gain/loss) Scores Predictive validity (e.g., speech grades)

ASSESSING THE ASSESSMENT:

Face/Ecological/Representational

Construct (convergent + discriminant) validity:

+ with + (e.g., skills measures)

- with - (e.g., anxiety measures)

0 with 0 (e.g., U.S. ethnicity)

360° Assessment (i.e., network consensus)

Delta (gain/loss) Scores

Predictive validity (e.g., speech grades)

CRITICAL THINKING ASSESSMENTS: ANALYSIS SYNTHESIS DEDUCTION INDUCTION CREATIVE THINKING

CRITICAL THINKING ASSESSMENTS:

ANALYSIS

SYNTHESIS

DEDUCTION

INDUCTION

CREATIVE THINKING

1. How many animals of each species did Moses take upon the ark with him? Answer: A. 0 The story goes that NOAH was on the ark, not MOSES. Our minds internalize cultural narratives and assumptions. We hear bits of such narratives and “fill in” the rest, even when it is not there. Therefore, language often evokes “realities” in our minds that are not real or accurate. 0 1 2 infinite

0

1

2

infinite

2. Count the “F’s” in the following: “ FINISHED FILES ARE THE RE- SULT OF YEARS OF SCIENTIF- IC STUDY COMBINED WITH THE EXPERIENCE OF YEARS.” F INISHED F ILES ARE THE RE- SULT O F YEARS O F SCIENTI F - IC STUDY COMBINED WITH THE EXPERIENCE O F YEARS. Answer: D. 6 Because we often “speak” the words in our minds as we read them, we don’t “hear” the “F’s” because they don’t “sound” like “F’s” 3 4 5 6

3

4

5

6

3. The 9-dot problem: With the mouse and line-drawing tool, connect all 9 dots with each other with 4 and only 4 completely straight lines. To solve this problem, you literally have to think “outside of the box.” We tend to think in terms of familiar objects, such as “boxes” or rooms, project the properties of a container onto these 9 dots, and then act as if there are “sides” or walls to the box that constrain our lines. 3 1 2 4

4. How many squares are in the figure below? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 People often miss the correct answer on this problem due to lack of motivation to find all the boxes. Once a large number has been found, it often “seems good enough.” Answer: 40

5. Given the following two propositions , what can be deduced? - As communication anxiety increases, speech errors increase. -As speech errors increase, speaker credibility decreases. -Therefore: As speech errors increase, communication anxiety decreases As speaker credibility decreases, speech errors decrease As communication anxiety increases, speaker credibility decreases As speaker effectiveness decreases, likelihood of persuading a listener decreases General semantics seeks a more “scientific” approach to seeing the world. Understanding how deduction works is a first step. In deductive reasoning, 1. If A, then B; 2. If B, then C; therefore, 3. If A, then C. A B B C

As speech errors increase, communication anxiety decreases

As speaker credibility decreases, speech errors decrease

As communication anxiety increases, speaker credibility decreases

As speaker effectiveness decreases, likelihood of persuading a listener decreases

6. Which of the following diagrams best represents the following proposition: As critical thinking ability increases, G.P.A. increases? In pursuing a more scientific understanding of the world, it is important to recognize both the types of relations things have to one another, but how our language about these relationships represents the “territory” of those relationships. The form here is: As X increases, Y increases, or X is positively related to Y. This is better reflected on the next slide. A B C D Hi Critical Thinking Ability Lo 1.0 4.0 G.P.A. Hi Critical Thinking Ability Lo 1.0 4.0 G.P.A. Hi Critical Thinking Ability Lo 1.0 4.0 G.P.A. Hi Critical Thinking Ability Lo 1.0 4.0 G.P.A.

Hi Critical Thinking Ability Lo 1.0 4.0 G.P.A. Someone who was low in critical thinking ability (CTA) would also tend to be lower in G.P.A. Someone who was moderate in CTA would also tend to be moderate in G.P.A. Someone who was higher in CTA would also tend to be higher in G.P.A. The line therefore represents graphically what the proposition stated verbally—As critical thinking ability increases, G.P.A. increases. In General Semantics terms, both the verbal and visual maps better represent the territory. 6. continued…

7. What is the maximum number of times, without covering any square more than once, that the figure below can fill in the spaces of the puzzle? Spatial rotation helps to figure out the solution in the same way that jigsaw puzzles often operate by shape of the “map” piece rather than knowledge of the “territory” being reproduced. Answer: 5 1 2 3 4 5

[Illustrative] Lessons: Our senses are fallible—”seeing” should definitely NOT always be believing: With a ruler and calculator, you could solve some of these problems, but we tend not to scientifically “test” our everyday experiences with such care . Our common sense (i.e., inference) is fallible: For example, 9 dots look like squares we have encountered throughout our lives, and because most squares represent boundaries or walls, we infer that these 9 dots are “limits” in some sense to our lines .

Our senses are fallible—”seeing” should definitely NOT always be believing: With a ruler and calculator, you could solve some of these problems, but we tend not to scientifically “test” our everyday experiences with such care .

Our common sense (i.e., inference) is fallible: For example, 9 dots look like squares we have encountered throughout our lives, and because most squares represent boundaries or walls, we infer that these 9 dots are “limits” in some sense to our lines .

SELF- (& OTHER) REPORT ASSESSMENTS: INTERPERSONAL COMMUNICATION COMPETENCE SMALL GROUP DECISION-MAKING & LEADERSHIP COMPETENCE COMPUTER-MEDIATED COMMUNICATION COMPETENCE PUBLIC SPEAKING COMPETENCE

SELF- (& OTHER) REPORT ASSESSMENTS:

INTERPERSONAL COMMUNICATION COMPETENCE

SMALL GROUP DECISION-MAKING & LEADERSHIP COMPETENCE

COMPUTER-MEDIATED COMMUNICATION COMPETENCE

PUBLIC SPEAKING COMPETENCE

SCALING: People differ quite a bit in terms of how skilled they are at communicating. For the following statements, we would like you to estimate, compared to typical conversationalists you encounter, how skilled you are … COMPARED TO TYPICAL CONVERSATIONALISTS YOU ENCOUNTER, I AM… 1 = EXTREMELY BELOW AVERAGE SKILL in my … 2 = MODERATELY BELOW AVERAGE SKILL in my… 3 = SLIGHTLY BELOW AVERAGE SKILL in my… 4 = AVERAGE SKILL in my… 5 = SLIGHTLY ABOVE AVERAGE SKILL in my… 6 = MODERATELY ABOVE AVERAGE SKILL in my… 7 = EXTREMELY ABOVE AVERAGE SKILL in my…

INTERPERSONAL COMMUNICATION COMPETENCE: Sample Items: Speaking rate (i.e., neither too slow nor too fast) Use of eye contact Use of gestures to emphasize what was being said Use of humor and/or stories Expression of personal opinions (i.e., neither too passive or aggressive)

INTERPERSONAL COMMUNICATION COMPETENCE:

Sample Items:

Speaking rate (i.e., neither too slow nor too fast)

Use of eye contact

Use of gestures to emphasize what was being said

Use of humor and/or stories

Expression of personal opinions (i.e., neither too passive or aggressive)

COMPUTER-MEDIATED COMMUNICATION COMPETENCE: SKILLS: COORDINATION __19. I know when and how to close down a topic of conversation in CMC dialogues. ATTENTIVENESS __23. I ask questions of the other person in my CMC. EXPRESSIVENESS __27. I am very articulate and vivid in my CMC messages. COMPOSURE __31. I display a lot of certainty in the way I write my CMC messages.

GROUP & LEADERSHIP COMPETENCE: When communicating as a member of a group, … I am able to help the group define the problem, or what it needs to accomplish. I am able to help the group analyze the causes and effects of the problem to be solved. I am able to come up with ways of evaluating what solutions would work and which would not. I am able to come up with creative ideas for potential solutions or alternatives.

GROUP & LEADERSHIP COMPETENCE:

When communicating as a member of a group, …

I am able to help the group define the problem, or what it needs to accomplish.

I am able to help the group analyze the causes and effects of the problem to be solved.

I am able to come up with ways of evaluating what solutions would work and which would not.

I am able to come up with creative ideas for potential solutions or alternatives.

PUBLIC SPEAKING COMPETENCE: MOTIVATION : Speaking in public makes me nervous. I can confidently get up and speak spontaneously to a group of people. I am anxious speaking to an audience.

PUBLIC SPEAKING COMPETENCE:

MOTIVATION :

Speaking in public makes me nervous.

I can confidently get up and speak spontaneously to a group of people.

I am anxious speaking to an audience.

PUBLIC SPEAKING COMPETENCE: SKILLS : When giving a speech to an audience, … I use some creative way of introducing the speech to get their attention. I provide a clear statement of purpose. I am able to come up with a logical way of organizing my ideas in a speech. I am able to develop sound arguments for my point(s) of view. I am able to back up my arguments with solid evidence in the speech.

PUBLIC SPEAKING COMPETENCE:

SKILLS : When giving a speech to an audience, …

I use some creative way of introducing the speech to get their attention.

I provide a clear statement of purpose.

I am able to come up with a logical way of organizing my ideas in a speech.

I am able to develop sound arguments for my point(s) of view.

I am able to back up my arguments with solid evidence in the speech.

SYSTEM CAPABILITIES: Online Access Delta Scores Random Item Selection Participatory Animation Random Item Interspersing Data Management Measurement Add-Ons Personal Profiles 360° Assessment

SYSTEM CAPABILITIES:

Online Access

Delta Scores

Random Item Selection

Participatory Animation

Random Item Interspersing

Data Management

Measurement Add-Ons

Personal Profiles

360° Assessment

EXEMPLARY PROFILES: Students will be able to download and print at least 3 types of assessment profiles of how their self assessments compare to… POPULATION : all students who have responded NETWORK : acquaintances’ views of the student CHANGE : their time-1 vs. time-2 assessments FOR EXAMPLE…

EXEMPLARY PROFILES:

Students will be able to download and print at least 3 types of assessment profiles of how their self assessments compare to…

POPULATION : all students who have responded

NETWORK : acquaintances’ views of the student

CHANGE : their time-1 vs. time-2 assessments

NETWORK-BASED EXAMPLE PROFILE: Reason- ing 100% 0% = Your self-rating = Averaged network rating % = Percentile of S population Interpersonal Skills CMC Skills Group Skills Public Speaking Skills -4% -9% -6% -12% +3% % Diff.

POPULATION-BASED EXAMPLE PROFILE: 100% 0% Interpersonal Skills CMC Skills Group Skills +1% -3% -9% -6% +7% % Diff. = Your self-rating = Averaged course self-rating % = Percentile of S population Reason- ing Public Speaking Skills

CHANGE EXAMPLE PROFILE: 100% 0% Interpersonal Skills CMC Skills Group Skills +4% +3% +1 +3% +10% % Diff. = Your Time 1 self-rating = Your Time 2 self-rating % = Percentile of S population Reason- ing Public Speaking Skills

For information, contact: Brian H. Spitzberg, Ph.D. Spitz @ mail.sdsu.edu @