Presentation by Maksimenkova Olga, Neznanov Alexey, Radchenko Irina, which presents a paper about a data expedition as a formative assessment tool to data-driven courses, ICL-2018.

1. Collaborative Learning in
Data Science Education:
a Data Expedition as
a Formative Assessment Tool
ICL-2018, Kos
Presenter:
Maksimenkova Olga
junior research
fellow IL ISSA,
Faculty of Computer
Science, National
Research University
Higher School of
Economics, Russia
© Maksimenkova O., Neznanov A., Radchenko I. 1

2. Our Working
Group
• From left to right
• Radchenko Irina
• ITMO University, Saint-Petersburg,
Russia
• Maksimenkova Olga
• NRU HSE, Moscow, Russia
• Neznanov Alexey
• NRU HSE, Moscow, Russia

3. Data Science
“Data Science is an interdisciplinary field
that combines machine learning, statistics,
advanced analysis, and programming. It is
a new form of art that draws out hidden
insights and puts data to work in the
cognitive era” (IBM Analitycs)
© Maksimenkova O., Neznanov A., Radchenko I. 3
This image license: CC BY-SA

4. A Data Scientist
Should
© Maksimenkova O., Neznanov A., Radchenko I. 4
be of an
analytical and
exploratory
mindset
have a good
understanding of
how to do data-
based
(quantitative)
research
possess
statistical skills
and be
comfortable
handling diverse
data
be clear,
effective
communicators
with the ability
to interact across
multiple
business levels
have a thorough
understanding of
their business
context

5. Principles of
Teaching Data
Science
© Maksimenkova O., Neznanov A., Radchenko I. 5
This picture license: CC BY-SA-NC
organize the course around a set of
diverse case studies
integrate computing into every
aspect of the course
teach abstraction, but minimize
reliance on mathematic notation
structure course activities to realistically
mimic a data scientist’s experience
demonstrate the importance of critical
thinking/skepticism through examples

6. What is a Data
Expedition?
• “Data expedition (DE) is a learning by
doing computer-supported collaborative
learning technique, which is applied to
Data Science Education”
• Creation a data-driven media
artefacts
• 2-3 weeks
• Collaboration in small (2-3 persons)
teams
© Maksimenkova O., Neznanov A., Radchenko I. 6
This picture license: CC BY

7. A matter of evaluation:
How to evaluate a data
expedition?
• What to evaluate?
• The processes of digital media
artefact creation
• The quality of data citation
• Correctness and reproducibility
• The complexity, interactivity, and
design of final artefacts
© Maksimenkova O., Neznanov A., Radchenko I. 7
This picture license: CC BY-NC-ND

8. Data Expedition Evaluation Criteria
© Maksimenkova O., Neznanov A., Radchenko I. 8
Data expedition dairy (experiment
journal) for research data expeditions
Using scripts written in one of the
programming languages
Using special software for data
preprocessing or analysis
Programming code
Data tiding
Data analysis
Data visualization
This picture license: CC BY

9. Different Learning Environment
• Integration, communication, and management tools
• Rich collaborative environments (Microsoft Office 365, Google G Suite)
• Project management tools (Trello, Microsoft Teams, MeisterTask)
• Online communication tools (Microsoft Skype, Telegram)
• Search and external discussions tools
• Universal search engines (Google Search, Microsoft Bing)
• Specialized search engines (Wolfram Alpha)
• Social Media Services (Facebook, Twitter, Instagram)
• QA services (Data Science Stack Exchange, Public Lab, Quora)
• Data processing tools
• Universal cloud data storages (DropBox, Microsoft OneDrive, Google Drive, Yandex Disk)
• Code repositories (GitHub, BitBucket, GitLab)
• Data analysis and visualization tools (R Studio, Jupyter Notebook, Orange, Microsoft Excel,
Microsoft Machine Learning Studio, Tableau, OpenRefine, Infogram, Plotly)
• Open data platforms (CKAN, Zenodo)
• Specialized data extraction tools (Import.IO, Kimono, Scrapy, ABBYY FineReader)
• Data validation tools (CSVLint, Data Package Validator, Data Hub LOD Validator)
• Media publishing platforms
• (Medium, Microsoft Sway, Tilda, GitBook, Wordpress, Blogger)
9© Maksimenkova O., Neznanov A., Radchenko I.

10. Implementation in Universities
© Maksimenkova O., Neznanov A., Radchenko I. 10
2016-2017 academic year
• Data Journalism, first-year master students,
National Research University Higher School of
Economics, Moscow
• Students and free-listeners European
University, Saint-Petersburg
Two different stories

11. Post-survey (HSE)
© Maksimenkova O., Neznanov A., Radchenko I. 11
Question 33% 67%
Was your participation to
the data expedition useful
for you?
Yes Probably, yes
How was your time
allocated during the data
expedition?
Little by little, but
regularly
I worked not regularly.
Sometimes for a long,
sometimes there were no
time at all
Give a characteristic of your
interaction with the other
participants?
I prefer working alone I had enough interaction, I am
satisfied

12. Post-survey (HSE)
© Maksimenkova O., Neznanov A., Radchenko I. 12
•“I have created universal code which I am going to use in the other projects”
•“Data expedition is a completely new for me type of a project. So, to accomplish the task the
time had to be distributed in a separate way. But, each project goes to “Experience” storage
and this is good”
•“Practical experience in writing data-driven article”
What were useful for you in data expedition?
•“On the one hand, we were free in the selection of topic but on the other the result would
have been limited stricter. I mean clear requirements to our work. Moreover, it would be
better to work with new not previously publicized data (not only links to open data portals
which are we familiar with and have no interest). But the freedom in topic selection should
be kept”
•“I did not like that we immediately got down to independent work. At the lessons before the
data expeditions we only clean data. It would be better to pass through all the stages with
teachers’ control”
•“I wish more practice before a data expedition”
What were not satisfactory for you in data expedition?

13. Post-Survey (EU)
13© Maksimenkova O., Neznanov A., Radchenko I.
Questions Yes, % No, %
Have you previously worked with open data? 28,6 71,7
Have you previously worked with online services
for open data visualization?
7,1 92,9
Have you previously written scientific papers in
your native language?
92,9 7,1
Have you previously written scientific papers in
English?
35,7 64,3
Is it interesting for you to learn about Open Science
and its implementation for research?
92,2 7,1

14. Recommendations
• Evaluation should take into consideration two aspects of a DE:
• orientation to the predefined goal
• multi-stage nature of main Data Expedition’s processes
• Predefined goal allows one to check correspondence between task description and final digital
media artefacts
• Multi-stage nature of DE processes allows one to assess involvement and impact of
participants based on DE diary and logs
• The features of digital media artefact which is created during a DE
should be reflected in grading rules
• Proposed rubrics should be discussed with Data Expedition’s
participants
• Grading of data citation, reproducibility and provenance should
follow available guidelines
14© Maksimenkova O., Neznanov A., Radchenko I.

15. Future Work
Assessment in form of Data Expedition should be
implemented after a short supportive block which aims
to refresh relevant knowledge and skills
Working groups should be completed according to
rubrics
© Maksimenkova O., Neznanov A., Radchenko I. 15
This picture, license: CC BY

16. Contacts
• Contacts:
• Irina Radchenko
• E-mail: iradche@gmail.com
• Web-site: http://iradche.ru/english/
• Alexey Neznanov
• E-mail: aneznanov@hse.ru
• Web-site:
https://www.hse.ru/en/staff/aneznanov
• Olga Maksimenkova
• E-mail: omaksimenkova@hse.ru
• Web-site:
https://www.hse.ru/en/staff/maksimenkova
© Maksimenkova O., Neznanov A., Radchenko I. 16