Facilitator-based Large-scale Online Discussion Support System: COLLAGREE

1. Aug 31-Sep 4, 2014@ACIS AAI 2014,
Kitakyushu International Conference Center
Facilitating
Large-scale Collective Discussion
Takayuki Ito
Nagoya Institute of Technology,
JAPAN
ito@nitech.ac.jp

2. Takayuki Ito
Professor,
Nagoya Institute of Technology
E-mail : ito@nitech.ac.jp
http://www.itolab.nitech.ac.jp/~ito/
2000 Dr. of Engineering, Nagoya Institute of Technology
1999 Research fellow, Japan Society for the Promotion of Science (JSPS).
2000 Visiting Researcher, USC/ISI.
2001 Assoc. Professor, Japan Advanced Insti. of Sci. & Tech. (JAIST).
2003 Assoc. Professor, Dept. of CSE, Nagoya Institute of Technology.
2005 Visiting Scholar, Computer Science, Harvard University.
2005 Visiting Researcher, MIT Sloan School of Management.
2006 Assoc. Prof., School of Techno-Business Admin., Nagoya Institute of Technology.
2008 Visiting Scientist, Center for Collective Intelligence, MIT Sloan School of
Management.
2010 JST PREST researcher (super challenging type)
2011 Japanese Cabinet Office’s NEXT fund Principal Investigator
2013 AAMAS 2013 Program Chair
2014 Professor, School of Techno-Business Admin., Nagoya Institute of Technology.

3. ito@nitech.ac.jp
Area :
Computer Science, Artificial Intelligence, Multi-Agent Systems,
Collective Intelligence, Automated Negotiations, Auction Theory,
Mechanism Design, Smart City, Smart Grid, etc.
Main achievements:
• 2014 JSSST Fundamental Research Award
• 2013 JSPS Award (日本学術振興会賞）
• Prizes for Science and Technology, The Commendation for Science and
Technology by the Minister of Education, Culture, Sports, Science and Technology,
2013.
• （文部科学大臣表彰科学技術賞）
• The NEXT Funding Program from the Japanese Cabinet Office
• The Young Scientists' Prize, The Commendation for Science and Technology by
the Minister of Education, Culture, Sports, Science and Technology, 2007.
• （文部科学大臣表彰若手科学者賞）
• Nagao Special Researcher Award, IPSJ 2007
• Best Paper Award, The Fifth International Joint Conference on Autonomous
Agents and Multi-Agent Systems (AAMAS2006, 1/553), 2006.
• 2005 Best Paper Award, Japan Society for Softoware Science and Technoglogy.
• 2004 IPA Exploratory Software Creation Project, Super Creator Award.

4. Background and Goal
Need for a large-scale discussion and gathering opinions
City planning Design
Direct democratic discussion has
ito@nitech.ac.jp
been focused very much.
New products was developed using
much user voice.
Some fields need for a large-scale consensus
We develop an open web-based forum system
called COLLAGREE (COLLective, COLLaborateive, AGREEment)

5. Background and Goal
Some management-level people are checking the newly
created contents in open-source projects.
From viewpoint of collective intelligence, some level of
management or control are required.
→ Some management-level people avoid flaming
and check some contents.
Facilitator: who adequately coordinate, lead, integrate,
classify, and summarize discussion.
ito@nitech.ac.jp
Wikipedia
Real-world workshops
Wikipedia and Linux projects
Real workshop in collaborative group
City planning division｜Workshop with about 15 persons
Design division｜Brain storming with 4 persons
→ The real-world workshops are coordinated by a facilitator.
We introduce facilitators and support functions
into an large-scale online discussion.

6. Supporting Facilitators
In our system, some human facilitators coordinate
the large-scale discussion.
Facilitator
• lead discussion process.
• maintain human relations
Current Web-based discussion Large-scale discussion
Flaming
• Conflict between users
Groupthink
• The group results in an irrational.
Monitoring cost
• They must check enormous opinions.
Scale out problem
• Previous methods is for small-scale.
ito@nitech.ac.jp
Real-world discussion
• City planning
• Design
We have to support facilitators so that they can manage the “large-scale”
discussion.
We develop the support functions
to support facilitators for coordinating discussion.

7. Related Work
Climate CoLab is one of the most
famous web-based collective
intelligence projects. The goal of
Climate CoLab is to harness the
collective intelligence of thousands of
people from all around the world to
address global climate change.
This is a crowdsourcing platform
where citizens work with experts and
each other to create, analyze, and
select detailed proposals for what to
do about climate change.
They do not have “facilitators”.
Joshua Introne, Robert Laubacher, Gary Olson, and Thomas Malone. 2011. “The Climate CoLab: Large scale
model-based collaborative planning.” In Proceedings of International Conference on Collaboration
Technologies and Systems (CTS 2011)
Thomas W. Malone, Robert Laubacher, Josh Introne, Mark Klein, Hal Abelson, John Sterman, and Gary Olson.
2009. “The Climate Collaboratorium: Project Overview.” CCI Working Paper 2009-003 (Sept. 2009).
ito@nitech.ac.jp

8. Related Work
Deliveratorium is another project
where people submit ideas by
following the argumentation map.
The argumentation map is a kind of
the discussion structure.
Deliveratoriummakes it possible to
show the entire relations among
ideas and opinions very clearly using
the structure.
Ali Gurkan, Luca Iandoli, Mark Klein, and Giuseppe Zollo. 2010. ” Mediating debate
through on-line large-scale argumentation: Evidence from the field.” Information
Sciences 180, 19 (2010), 3686 – 3702.
ito@nitech.ac.jp

9. COLLAGREE User Interface｜Top
People can submit their opinions as a chunk of texts as like the
ordinary forums and electronic bulletin boards.
ito@nitech.ac.jp
Discussion categories
We can discuss some theme at the same time.

10. User Interface｜Argumentation
We develop several supporting functions for facilitators.
Support functions
① Agreement or disagreement analysis
for a comment is shown.
② Keywords highlighted. Facilitators can
understand what kind of keywords are
focused and important.
③ Issue tags by which participants can
add a tag each opinion and comment
so that they can search it afterwards.
④ Facilitation tab from which a facilitator
can input his/her instructions tor the
participants.
⑤ Reminders for participants. It sends
emails as well as reminders when some
related events happen.
ito@nitech.ac.jp
1
2
3
4
5
We developed several supporting functions for facilitators.

11. Demo Movie
User-Interface
1. Top / Theme
2. Discussion
Support function
1. Agreement or disagreement
2. Facilitation tab
3. Keywords highlighted
4. Issue tag
5. Reminders
ito@nitech.ac.jp

12. System Architecture
Light weight architecture
CoffeeScript
JavaScript
Spine.js Ruby on Rails
Comment
ito@nitech.ac.jp
Keyword analytics
Judgement of
agree / disagree
Web server(Nginx)
Opinion and Comment
data
Keyword data
Database(mongoDB)
Brawser
Data management
Opinion data
User data
Keyword data
Keyword
Agree / Disagree
Comment
Post
Ubuntu
Computer environments
OS Ubuntu12.04
Computer
Language
Ruby, JavaScript
CoffeeScript
Database mongoDB
Web server Ngnix
Framework Ruby on Rails3, Spine.js

13. A Large-scale Experiment in Nagoya City
We deployed our system for internet-based town meeting in
Nagoya city, as a city project led by Nagoya city mayor
An internet-based town meeting about “Town” based on the
Nagoya Next Generation Total City Planning 2014-2018
(4 discussion categories)
Press conference Newspapers
ito@nitech.ac.jp
Nov 19, 2013 ~
Dec 3, 2013 [15 days]
Nov18, Press conference with the mayor of Nagoya Nov 13, 30 Chunichi Shinbun (Japanese newspaper)
The civilian participants 264 people
The professional facilitators 9 people
Questionnaires online
• Operability and effect
• The importance of facilitators
We evaluated COLLAGREE by the large-scale experiment.

14. ito@nitech.ac.jp

15. ito@nitech.ac.jp

16. A Large-scale Experiment in Nagoya City
We announced in our experiment on Nagoya city website.
Explanation of COLLAGREE and Introduction
of an internet-based town meeting
We evaluated COLLAGREE by the large-scale experiment.
ito@nitech.ac.jp
■ Nagoya city website
A link to COLLAGREE

17. ito@nitech.ac.jp

18. Awarding Users A Prize from Mayor
Nagoya city mayor awarded a prize to the good users
3,072 visits
1,151 opinions 18,446 view
Awarding 3 good participants
10s, male
COLLAGREE succeeded gathering good opinions.
ito@nitech.ac.jp
Jan 29 Chunichi Shinbun
(Japanese newspaper)
30s, female
40s, female
City planning which matched feature
of Nagoya
Art and barrier-free
Training for region disaster prevention
Mayor “There are predominantly many young
people in the site, and opinions different from
the town meeting are useful.”

19. ito@nitech.ac.jp

20. We compared our system with real town meeting.
Real town meeting(Nagoya) COLLAGREE
COLLAGREE
Ages of Participants
10代
20代
30代
4010-19 20-29 30-39 40代-4 9
5500代-59
6O0代ver以 60上
無No回 a答nsw
er
Argument by the participant of the wide generation unlike real-world
town meeting
ito@nitech.ac.jp
20-29
26%
30-39
15%
50-59
17%
40-49
27%
Over 60
57%
No Answer
10-19
1%
Over 60
7%
12areas
678 people
Participants
264 people

21. Questionnaires about Facilitation [User]
We asked users about the importance of facilitation
非常にそう思う
そう思う
どちらでもない
そう思わない
全くそう思わない
非常にそう思う
そう思う
どちらでもない
そう思わない
全くそう思わない
83%
97%
Q.1 Was facilitation necessary?
Agree Strongly Agree
Q.2 Could facilitator harness discussion?
Agree Strongly Agree
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Comments
非常にそう思う
そう思う
どちらでもないAgree strongly Agree Indifferent
そDうisa思grわeeない
全Dくisそagうre思e sわtroなnいgly
• Facilitators and Facilitations are essential. In specially, there is no direction in the discussion
Participants understood the importance of facilitators
to coordinate discussion and gather many opinions.
ito@nitech.ac.jp
with many and unspecified persons.
• It is easy to post opinions because facilitator comments have much humanity.
• It is better that facilitator summarize discussion some times.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

22. Questionnaires about Facilitation [Facilitator]
We asked faciliators about the importance of facilitation
非常にそう思う
そう思う
どちらでもない
そう思わない
全くそう思わない
非常にそう思う
そう思う
どちらでもない
そう思わない
全くそう思わない
Agree Strongly Agree
Agree Strongly
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
非常にそう思う
そう思う
どちらでもないAgree strongly Agree Indifferent
そDうisa思grわeeない
全Dくisそagうre思e sわtroなnいgly
Facilitators understood the importance of facilitators
to coordinate discussion and gather many opinions.
ito@nitech.ac.jp
75%
88%
Q.1 Was facilitation necessary?
Q.2 Could facilitator harness discussion?
Agree
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Comments
• I thought that facilitators was necessary not to keep on giving an opinion.
• Facilitator is needed to keep the discussion safely.
• It is difficult to understand opinions of Silent majority.
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

23. Questionnaires about Facilitator support functions
We ask facilitators about the support functions
Q. Was each support functions useful for discussion?
非常にそう思う
そう思う
どちらでもない
そう思わない
全くそう思わない
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
ファシリテーション投稿
Agree / Disagree
賛成・反対表示
論点タグ
注目キーワード
行動履歴
Activity
リマインドメール
非常にそう思う
そう思う
どちらでもないAgree strongly Agree Indifferent
そDうisa思grわeeない
全Dくisそagうre思e sわtroなnいgly
We confirm usefulness of the support about the discussion
analysis, some difficulty to manage the large-scale discussion.
ito@nitech.ac.jp
Facilitation tab
Issue tag
Keyword highlight
Reminder

24. Summary
ito@nitech.ac.jp
Background
Goal
Need for large-scale discussion and gathering opinions.
We develop the functions to support the facilitator
coordinating discussion.
Analysis real-world workshop
Development of system
Large-scale experiment
An internet-based town meeting in Nagoya city
about “Town” based on the Nagoya Next
Generation Total City Planning 2014-2018.
Participants (80%) and facilitators (75%)
realized the importance of facilitator.
We confirmed usefulness of
facilitator
COLLAGREE succeeded coordinating a large-scale
discussion(266 people) and gathering
many opinions compared with the real-world
town meeting.
Usefulness of facilitator
Future Work
Many directions that include creating more effective
facilitation supports.

25. Ongoing work & Future direction
1. Online discussion for the design of new products in
a company
• Furniture design with Toyo-set inc.
2. Large scale online & offline simultaneous discussion
• Project with Aichi prefecture
3. Automated facilitation & Automated agreement
ito@nitech.ac.jp

26. Future direction
ito@nitech.ac.jp

27. Automated facilitation & Automated agreement
Done Done
ito@nitech.ac.jp
Human facilitator
Agent
ユーザ
Discussion
①Agent supports human facilitator
Human facilitator
②Agents & human participate
③Agents discuss on behalf of users
Agent facilitator
Discussion
④Automated facilitation by agent
Agent
ユーザ
Discussion
Automated agreement
Agent
Next
User User
Next

28. Special Thanks
Prof. Eizo Hideshima (NIT) :
For arranging research collaboration
Prof. Takanori Ito (NIT) :
For designing and planning
Students : For implementing and actual work
Mr. Kawamura (Mayor of Nagoya) : For announcement and awards
Nagoya city officers : For experiment planning
Japan Facilitation Association : For facilitation
Participants : For their valuable opinions and participation itself
ito@nitech.ac.jp
My family : For patient supports
Dr. Mark Klein (MIT) :
For wonderful comments