The document discusses the 'Evaluate' project, a European initiative assessing the impact of telecollaborative learning on student-teachers' competencies. It explores teacher interventions in virtual exchanges to enhance intercultural and digital-pedagogical skills, alongside showcasing critical incidents that emerged during student collaborations. The findings indicate that active teacher participation can influence the quality of student interactions and belief systems concerning communication and collaboration.

1. The Influence of Teacher Intervention
on Quality of Interaction in a
Telecollaboration
Robert O’Dowd Shannon Sauro Elana Spector-Cohen
University of León Malmö University Tel Aviv University
robert.odowd@unileon.es shannon.sauro@mau.se espector@post.tau.ac.il

2. EVALUATE: Evaluating and Upscaling Telecollaborative Teacher Education (EVALUATE)
What are the aims of EVALUATE?
•This European Policy Experimentation will
1) assess the impact of Virtual Exchange on student-teachers involved in various European
countries
2) work with Public Authorities to upscale its use in Initial Teacher Education.
Research Questions:
Will telecollaboration have a positive impact on future teachers’
1) digital-pedagogical competence?
2) intercultural competence?
3) foreign language competence?
EVALUATE is the largest study on telecollaborative learning to be carried out to date. Our project
was one of 35 Virtual Exchanges involving transnational teams of student teachers using common
task sequences and modes of communication.
www.evaluateproject.eu
Context:

3. Liddicoat and Scarino (2013):
[Referring to a wide range telecollaborative studies:]
“In each instance discussed above, interaction using a social technology
has not necessarily resulted in intercultural learning… The tasks involved
students in exchanges across cultures…but the intercultural learning was
supposed to happen as an automatic result of communication or
engagement with others.”
“The problem is that exposure to interaction of itself does not necessarily
equate with intercultural learning . . . To be able to contribute to learning,
the interaction must first become available in some way for students to
reflect on and interpret.” (p. 112).
Literature Review: Role of teacher in Virtual Exchange

4. Cunningham and Vyatkina (2012): teachers transcribe and code
extracts of students’ telecollaborative videoconferences and
then review these transcripts with their students in class.
Kern (2014): “La salle de retrospection” - French and American
student partners review and reflect on recordings of their
telecollaborative videoconferences with their teachers
Ware (2013): Teaching good online communicative strategies
Examples of Pedagogical Interventions in Telecollaboration:

5. Features of ‘successful online communication’ (Ware, 2013, p. 13)
CATEGORY EXAMPLE from current study
Emotive words and phrases “I think it’s a great idea to get things done!”
Personal forms of address “I don’t think is usual one-to-one classrooms in
Israel, Hanne.”
Topic development “Yes, the final summary should be in evaluate I
guess.”
Question posing “Would you say new technologies are used in the
secondary education system in your country?”
Emoticons :-)
Personal information “I have already tried fanfiction with my students
and it worked very well, and I will continue to
teach fandom.”
Display of alignment “I totally agree, let’s do it ASAP then.”

6. 1) Does the active intervention of the teacher in
telecollaboration influence the quality of interaction?
1) Does this mentoring have a persistent influence on their
beliefs and interactions during the virtual exchange?
Our Research Questions

7. Israel
● 10 students in
International MA
TESOL program at
Tel Aviv University
(2 are Israeli)
● Training to teach
English in diverse
settings around the
globe
Methods: The Three Partners
Spain
● 20 students in MA
programme in
Secondary School
Education at the
University of León
● Training to teach
English in Spanish
secondary schools
Sweden
● 9 students in
BA/MA program at
Malmö University
● Training to teach
English at the
upper secondary
level in Sweden

8. Task 1.1 - Design a short multimodal presentation about
yourself
Task 1.2 - Create a group name and identity
Task 1.3 - Negotiate group expectations
Task 2 - Compare and contrast innovative technologies
Task 3 - Collaborate on task design
Methods: The Tasks

9. Classes divided into Treatment Groups (1-3) and Control Groups (4-6)
In Class Intervention (Beginning of Task 2)
• In class teacher guided-discussion (approx 30 minutes) with members of the
Treatment Groups (Groups 1,2,3)
• Eliciting solutions to 2 scenarios from prior exchanges
• Explicit teaching of 6 linguistic and interactional features of successful interactions
(based on Ware, 2013) with examples from prior exchanges.
Email Follow-Up (Later during Task 2)
• Individual emails to students
• What strategies have they deliberately used?
• What have they observed?
Unplanned Intervention (During Task 3)
• 2 “Critical Incidents”
• Individual emails or phone calls to students and among the 3 partner teachers
• In class discussion of incidents in 2 of the partner classes (up to 45 minutes)
Methods: The Teacher Intervention

10. The Student Interactions (RQ1: Quality of Interaction)
• (Whatsapp, Moodle & Google Docs) for Treatment Groups
• Tasks 1.2 & 1.3
• Task 2
• (Whatsapp, Moodle & Google Docs) for Control Groups
• Task 2
The Student Products (RQ2: Beliefs and Persistence)
• Completed reflection portfolios for Control and Treatment Groups
• Email correspondence and phone calls with Treatment Groups (Task 3)
regarding 2 critical incidents:
• The Whatsapp Incident (Group 1)
• The Google Docs Incident (Group 3)*
Methods: The Data

11. Change from Task 1 to 2
(Treatment Groups Only)
Treatment vs. Control Groups
(Task 2 Only)
Results RQ1

12. The Google Docs Incident
• Some students (Spanish class) felt
their suggestions were being ignored
by another student (Israeli class) and
directly chastised the student.
• That student and peer (Israeli class)
explained that their suggestion built
on that of their peers’ (Spanish class)
and asked for understanding.
• An additional linguistic
misunderstanding meant the request
for understanding was interpreted as
criticism (Spanish class).
The Critical Incident
Communication exclusively
through linear color-coded
posts on doc

13. Anne: (Israel): Hey guys! Today’s date is 12.11.17, I have an idea of what I’d like to do for task 3. I
think it would be great to incorporate the Flipped Classroom…
The Critical Incident
“Spanish Group” (Spain): We are in class right now, we are being explained what we are supposed to
do for next week. We also have some ideas about it, we will share them with you as soon as I get
home,...
Ying (Israel): ...I agree to talk about flipped classroom as Anne said, but we can see what you have
first and decide on that. Looking forward to hearing from you!
Anne:(Israel): Hello All! I’d like to propose that Ying and I work on the flipped classroom approach...
Marta (Spain): Hello Anne! I actually think we should take everyone’s point of view into account and
you have not even had a look at our proposal, I am sorry but I consider you should not work on it yet
until it has been “approved” by the rest of the members. This is a group activity, and this implies that
unilateral decisions MUST NOT be made, and in case you do it on your own, it will not be taken into
account, at least from my side. I hope you understand.

14. Ying (Israel): I agree that we can do the three lessons, as your Spanish group proposed. I also think within
one lesson, whoever in charge of it can change flexibly to meet the teaching needs. So based on what the
three lessons are, what Anne proposed is totally reasonable for the lesson...Lastly but not least, I would
prefer everyone assumes that others DO care about the project. I think we all want to do it well, and based
on that, communicate without forming groups or saying that others don’t care or see their opinions.
The Critical Incident
Anne (Israel): First and foremost If I waited for everyone’s suggestions this project would not have taken
any direction at all! If anyone took a look at my comment from 3 days ago they would see that their
(spanish group ) proposal was based off of my suggestion. Secondly, I picked that task for my partner
and I because we had already started BRAINSTORMING ideas off of each other 2 days ago and felt like
that it would be a load we could take on, since we are already in the middle of our semester and want to
turn in quality work, we understand that some of you have just started and may not be aware of what we
are trying to accomplish. Lastly my partner and I will do whatever we need to do to complete this task as
a team but I WILL NOT BE DISRESPECTED OR BELITTLED for stating an idea when no one had even
commented on where we should go with task 3! So, I thank you all very much for your suggestions and
comments and wish you all a great semester, please keep Ying and I updated on how we should submit
our part of task 3.

15. Marta (Spain): Hello Ying and Anne, I did not mean to be rude at all, politeness is always one of my points, and I
am sorry if you got the message wrong. I honestly do not consider mine as a harmful or disrespectful message at
all and I would like to highlight 2 points 1- All of us are engaged in the project that is why we have felt so
disappointed after being totally felt out because no comments had been made about our own proposal. 2. The
reason to the post was because of the lack of interest on our proposal; you just showed up and gave us your
ideas, waiting for us to agree or not and say our opinions about it, the problem is that you did not take us into
account. No further explanation is required, it was a very simple thing, I am sorry for the misunderstanding
anyway.
Ying (Israel): Hi Marta, I am sorry you felt left out or your plan isn’t responded to. I thought in both Anne and your
posts, it was clear that we agreed to your three lessons template. I am sorry if that wasn’t clear. And I can
imagine how excited you are when you come up with a good plan. I also want to share that joy by understanding
more about the plan. Do you make the lesson background set in Spain? With Refegees students in this lesson
plan from somewhere?
Marta: Hi Anne and Ying!! I would like to leave all this misunderstanding behind and go on, we are humans and
this communication breakdowns are very proper from this type of projects...
The Critical Incident

16. Internal group power relations:
Student from Sweden: “...the Spanish
students gave a different suggestion and
posted it under the name "the spanish
group"....However, I think that using that
name is problematic from a couple of
different reasons. 1. It sort of hints that they
are a group inside of our entire group. 2. It's
not really a discussion, they always agree
with each other and they are 4 persons
which is a majority.”
Linguistic & Intercultural
misunderstandings:
...Lastly but not least, I would prefer
everyone assumes that others DO care
about the project.
Teacher from Spain: “Rosario and others
mistakenly understood this to mean "At least we
in Israel care about the project, not like you" and
this really made them feel annoyed. When I
explained what this sentence really meant
something like "You're not the only ones that
care about the project, you know?' then her
expression and her attitude immediately
changed.”
The Critical Incident - Reasons for communication breakdown
Lack of awareness of Netiquette:
Teacher from Israel: “Not everyone was even aware of this, for e.g. that there is netiquette or
that caps are often seen as screaming.”

17. Does our mentoring have a persistent influence on their beliefs and interactions during the
virtual exchange?
Spanish final portfolios:
Rosario (final portfolio, treatment group 3): Here, I have learnt that sometimes one can
misunderstand what the other says and lose one’s nerves. The fact is that this is quite
common, but now I know that you must read very carefully or even more than twice to avoid
these misunderstandings.
Marta: (final portfolio, treatment group 3): What I have learnt from this project is basically related to
the cultural as well as the communication aspects. Although it was quite logical for me to find certain
type of problems due to the means of the via communication we chose to carry out this activity, I
never thought things could go out of control so much because of a little misunderstanding, so
regarding communication, I learnt that the choice of the main via of communication is crucial in
order to succeed in this aspect as well as the grammatical care , the very precise use of
grammar to convey certain type of information is essential in order to make myself fully
understood and avoid any kind of misunderstanding.
Results RQ 2

18. Israeli final portfolios:
Anne - Israel (critical reflection, treatment group 3): The entire group including myself started to
take on a we vs they mentality which left the one student from Sweden fending for himself,
something that I am truly sorry for. I realize now that I became too emotional about wanting my
ideas heard and forgetting the fact that I am dealing with people who are similar but also different
from me ... If I had to do it all over again, I would use the google doc more effectively by
using comments and the chat box to speak with group members on a more personal note,
and be more understanding of others as well.
Ying - Israel (critical reflection treatment group 3): I have never realised that typed messages
could cause misunderstanding, especially when involving non-specific context, or cross-
cultural factors. We cannot assume that people … deal with problems with similar
approaches. … I have known that the world is bigger than I thought it is... But even if people are
speaking the same language, the meaning may vary across culture. So more patience will be
needed.
More Results 2

19. Tools of communication may mediate the effectiveness of interactional
and linguistic strategies.
“Mentoring” in telecollaboration has to be more than simply “teaching
communicative strategies”. It is also…
• ‘Teaching on the fly’ - reacting and exploiting teachable moments as
they emerge...
• Frequent check-ins and ‘debriefing’
• Increasing sensitivity to how language and culture interact in/with digital
environments - langua-technocultural competence (Sauro & Chapelle,
2017)
• The repertoire of intersecting linguistic, cultural, and digital skills and
practices needed to interact in a range of digital spaces
• In other words, what language or behaviors used with which technology best
achieves what ends with which interlocutors
What conclusions can we draw from our study?

20. “Digital natives” Successful online academic collaborators
1. Provide constant guidance in interpreting and participating in online
interactions - language and cultural issues.
2. Make students aware of culturally situated digital practices (e.g.
acceptable response times, acceptable levels of participation, sharing
of photos etc.) and cultural approaches to conflict, etc.
3. Raise awareness of what tools are effective for different types of
communicative tasks and the specific features of each tool (e.g.,
Google docs - editing, suggesting, comments, chat).
4. Use critical incidents / communication breakdown as opportunities
for learning and encourage students to do the same.
Guidelines for teachers for mentoring Telecollaborative Exchange -

21. Cunningham, D. J., & Vyatkina, N. (2012). Telecollaboration for professional purposes:
Towards developing a formal register in the foreign language classroom.
Canadian Modern Language Review La Revue Canadienne Des Langues Vivantes,
68, 422–450.
Kern, R. (2014). Technology as Pharmakon: The promise and perils of the internet for
foreign language education. Modern Language Journal, 98, 340–357.
Liddicoat, A., & Scarino, A. (2013). Intercultural language teaching and learning. West
Sussex, GB: Wiley-Blackwell.
Sauro, S., & Chapelle, C. A. (2017). Toward langua-technocultural competence. In C. A. Chapelle & S. Sauro
(Eds.), The handbook of technology and second language teaching and learning (pp. 459-472). Oxford: Wiley-
Blackwell.
Ware, P. (2013). Teaching comments: Intercultural communication skills in the digital age. Intercultural
Education, 24(4), 315-326.
Evaluating and Upscaling Telecollaborative Teacher Education (EVALUATE) (582934-EPP-1-2016-2-ES-
EPPKA3-PI-POLICY) is funded by Erasmus+ Key Action 3 (EACEA No 34/2015). The views reflected in this
presentation are the authors’ alone and the commission cannot be held responsible for any use which may be
made of the information contained therein.
References