Final Report of Session Analysis Adhoc Committee APSIG2018

1. APSIG 2018- Session Analysis Final Report
1.1 Ad-Hoc Committee Members
Kilnam Chon - Program Committee Liaison
Shreedeep Rayamajhi - Co-Chair
GJ - Gnanajeyaraman Rajaram - Co-Chair
Maheeshwara Kirindigoda - Co-Chair
Zolzaya Shagdar
Unggul Sagena
Martin Burrows Jr.
Sunyoung Yang
1.2. Background of Ad Hoc Committee on Session Analysis
We have been working on 45-60 minutes sessions organized by APSIG participants, and had one
session in 2017 and seven sessions in 2018. But, their evaluations were not too good compared
with 90-minutes class by invited lecturers. We plan to analyze on these sessions and come up
with recommendations to Program Committee. We also need to consider reduction of number of
sessions organized by the participants.
1.3. Activities
First Remote Meeting on 2018.7.23
Discussion on the issues and the final report
Final Report by 2018.8.8
Second and Final Remote Meeting on 2018.8.8
1.4. Deliverable
1-3 page memo to APSIG Program Committee.
2.1. First Meeting - Minutes of Meeting, Suggestions and Follow-ups
First Ad Hoc Committee on Session Analysis Meeting
Date: Monday, 23 July 2018
Time: 07:00 - 09:00 UTC

2. 2.2. Meeting Attendees:
Shreedeep Rayamajhi - Co-Chair
GJ - Gnanajeyaraman Rajaram - Co-Chair (Editor)
Zolzaya Shagdar
Martin Burrows Jr.
Sunyoung Yang
Glazial Kae Tagamolila- Online Moderator
2.3. Agenda for First Ad Hoc Committee on Session Analysis Meeting
1. Survey Evaluation outcome
2. Challenges and opportunities of Case studies
3. How to improve the engagement in both classes?
4. Evaluation challenges
5. Strategies
2.4. Final report structure (chapters) --- gj
Schedule to develop the final report --- Shreedeep and Mahee
Coordinator ------Shreedeep
3.1. APSIG 2018 Outcomes / Discussions Preliminary discussions
3.1.1. Prof.Kilnam Chon
We had 7 case study sessions; many more than the last years. Our goal may be similar coverage
by invited lecturers (classes) and participants (case study sessions), and we reached similar
coverage in term of quantity, substantial increase over the last year. when we looked at the
evaluation scores, the case study sessions are well behind of the lectured class.

3. Firstly the committee collected feedback/ Survey- Google forms from APSIG 2018 Participants,
based on the feedback/ Survey form the Ad-hoc analysis committee received comments and
suggestions from APSIG 2018 Participants about session evaluation.
The Committee and the Program Committee Liaison Prof.Kilnam Chon prepared and discussed
the following tasks to among the members of the committee.
1. Shall we reduce the number of case study sessions since their performances are
not good? only 1 or 2 sessions had similar evaluation to the average classes.
2. What are the reasons of poor performance?
3. What do you suggest to have quality case study sessions in future?
The committee received comment and suggestions from APSIG participants/ members. Each and
every comments, Suggestions are noted and discussed through meeting and mail. All the
discussions are given below.
3.1.2 Martin:
I think the audience wanted more in-depth case studies with analysis and relevancy.
3.1.3. Zolzaya:
I do not think APSIG should reduce the number but if it had to, then 4-5 would be a good
number considering the time allocated.
Firstly, I felt that the participants did not have enough time to talk about their presentations.
Many of them were rushing and it made people disconnected from the topic. Secondly, in my
opinion, the contents of case study sessions were not interesting or challenging enough. I believe
they were merely just information than academic class or discussions. So the content should be
tailored to make people interested or updated. Finally, The speakers were not engaging with the
audience. Maybe it might have something to do with their experience in public speaking or so.
Maybe APSIG should reduce the number of case study session and give each speaker more time
to talk. Secondly, maybe revise and advice the case-study session speakers' presentations and
help them to try to make it interesting to the audience and challenging to the listeners. Finally,
may be choose or train the speakers on public speaking skills.
3.1.4.Mahee:

4. No, we need to improve the case study sessions by quality it is a tangible output of Apsig. Most
of them were not aware of the case study session structure. They mostly had a topic but not a
case which has been deeply studied.
Some case study sessions were fully loaded with many presentations. Some case study sessions
linked without the content similarity, ex my case study on youth digital addiction was on youth
engagement so that I had to narrow down the study to limit to suit the session topic. Due to that
haven't used the ppt to present.
Less coordination between session coordinators with the presentators. It should have
discussed much on the of the session content, content order,
Better coordination between session coordinators and presenters. Limit number of case studies at
one session. Relation of topics in session should be managed by its content. Otherwise keep
every case study as a separate one. Ex. "Internet shutdown" Time of the presentation should be
fixed and convey beforehand. Creating a guideline in case study presentation will too help to
uplift the quality. I still believe APSIG 2018 case study sessions went well.
Suggestion
1. It’s better to openly call for case study sessions and evaluate from their proposal
2. Pre evaluation will help to short list and group
3. It’s better to have one professional case study session done by a lecturer to let others
know how the case study session should be handled.
4. Create a mechanism to call for proposals
5. Evaluation through the proposal (Short listing and grouping)
6. Schedule group meetings to plan the case study sessions
7. If possible have a online training session or guiding session on how to run a case study
session or connect lecturers with case study proposers.
8. To present ne case study it is required minimum of 10 Minutes and 5 Minutes to discuss.
3.1.5.Unggul Sagena:
I personally think if we could have a guidelines among case study presenters it would be more
efficient. It should be before they send their presentations. For example, case study in topic 1 ,
there are 3 presenters. They are from National A, National B, National C and they just come up
with their specific case but along the line. Answers of same issues. The guideline could consists
of 3 parts, what are the issues should be the same.

5. What are the background, how their country (based on the presenters experiences) managed/or
not to overcome it, what is lesson learned from that. So in every case study topic 1 session, We
have table on the issue and know the difference in how the issue goes. How every country have
different approach etc. It would be good to be example for participants with almost the same
problems in accordance to internet governance in their country.
Management of time, and every presenters have their own style in presenting. Should be a
guideline to answer specific questions and how different approach to the issues by each country
(at least the presenter's organization did).
Presentation guideline:
Also for proposed workshop for IGF, should be more country specific or region specific. For
example, National Security and AI. Experiences such as "Geo based Pokemon Go" games in my
country last year had issues of what we call "national vital objects" (for example military
buildings) but people trespassing there just to "catch pokemon" using their smartphone. What is
the consequences, what if they do not know about "military building" as well as other vital
objects such as power plant, administrative office, etc is prohibited for taking picture or live
gram (like pokemon go, or Augmented Reality and in the future, AI based games).
On Fake News, Safenet (my organization) has released report on South East Asia leaders is using
fake news issues to silence critics.
http://www.thejakartapost.com/seasia/2018/06/22/se-asiancountries-use-fight-against-hoaxesto-
silence-critics-report.html
Also we could also have for example "election issues" such as in India, Pakistan, Indonesia
(democratic countries) and discuss in one case study session about hoax/ fake news and its
relation to the election.
http://www.thejakartapost.com/news/2018/03/15/indonesia-battles-fake-news-as-elections-
loom.html
3.1.6. Prof.Sunyoung:
Recommendations to the 2019 Program Committee.
1. Applicants' preparation
The application process can encourage applicants' self-learning by taking lectures from previous
years. For example, the application process can require applicants to offer brief summaries of
chosen lectures with their further questions for those lectures.

6. 2. Participants' preparation
Once the participants are selected from the pool of applicants, participants can create several
subgroups and prepare for the actual meeting through online discussions among themselves. This
preparation process can help participants to be ready for better discussions and questions for
different sessions. It will be helpful to have a leader for each subgroup who will facilitate online
discussions.
3. How to make APSIG beneficial for the lecturers
As we discussed during the closing session in the 2018 APSIG, we need to improve the quality
of discussions during the lectures. The key is coming up with good questions which the lecturers
will find relevant as well as hard to answer (meaning intellectually stimulating).
Applicants and participants preparation for the actual meeting (1 and 2 above) can improve the
quality of questions for the lecture discussions. The quality discussions during the lectures can be
the most beneficial for the lecturers. Another way to make APSIG beneficial for the lecturers can
be the quality interactions among the lecturers. We can ask lectures to become a discussant for
others' lectures and start the 45-minute discussions with the discussant's questions.
This can be a good learning opportunity for participants in terms of how to ask good questions.
Questions from lecturer-discussants can set the level of questions and discussions (quality) from
the beginning. This way will also enable lecturers to get involved in other lectures more in depth
and to get to know other lecturers better for future collaborations. This can be beneficial for the
lecturers.
4. How to improve Case study sessions
There were three deadlines for the lectures:
1. Lecture abstract,
2. Lecture outline, and
3. The final lecture PPTs.
If the Case study sessions have several deadlines in a similar way, each deadline can function as
an opportunity to gather feedback from other case study presenters as well as from the program
committee. Receiving feedback and revising case study presentations based on the feedback can
be effective to improve the quality of case study sessions if case study presenters use the peer-
review process actively.
5. Professional development program
The APSIG can offer the participants professional development program as well. This program
can include, for example, how to make effective presentation materials (most of presentations at

7. 2018 APSIG offered a lot of information densely within one PowerPoint page, which was far
from delivering information effectively), how to speak in public (slow pace, clear pitch, and
getting to the point succinctly), and how to write a good proposal (e.g., conference session,
funding, etc).
The lecturers can be good resources to develop this program since they probably have received
many professional development training to become experts in their areas. Qualified lecturers can
lead a professional development session in addition to their lectures at APSIG.
3.1.7. Shreedeep:
1. Pre online course of internet governance to be a taken by participants of APSIG or in the past
mandatory.
2. Coaching and mentorship program can be Introduced in APSIG 2018 so that there can be
proper guidance and cooperation
3. Lectures and interactive session needs to be managed.
For e.g. young lecturer should be placed in the afternoon and good lectures should be place in the
morning, keeps the participants going.
4. Due to language barriers Interactive session needs to be given priority which would help the
participants to open up.
5. All National SIG should have a Day for their activities and show their activities to the
participants this can be motivation and inspiration.
6. Real time exercise or workshop needs to be focused on issues of AI and IoT governance.
7. More data research and case studies needs to be focused so that the participants can relate to
the issues easily.
8. Public policy process or commenting can be integrated in APSIG as it helps in creating better
document and in open public policy process.
9. For IGF and other regional events, fixed proposal writing or experience sharing session needs
to be formalized or even an open forum.
10. Internet core values session should be there as it is the base of governance and the coming
technology of IoT and AI.
11. We should have open sessions like sharing their engagement and collaborative ideas so that
they can come up with better ideas and concept. It can be a warm up session.

8. 3.1.8. Based on Overall Evaluation Report by Aicha Chebbi and Session Evaluation Report
by Gnanajeyaraman Rajara [GJ]:
The committee received the preferences, comments and suggestions based on from overall
evaluation and session evaluation report mentioned below.
3.1.8.A. Based on Overall Evaluation Report by Aicha Chebbi
What lecture classes do you recommend for 2019?
Al, Block chain, Fake News, Privacy and Data Protection, Cyber Surveillance. We want to hear
more about digital governance and ai governance further, Social Implication of Digital
Technology, lot AKANN, Expert System, BI, Smart City, Network Security, Information
Security and Audit, Industries 4.0, Next generation technology governance, digital ecosystem
and governance, social sciences governance, civil society experience, cyber governance
Technology used in Financial sector, Its risks and compensating controls. There should be more
time for cybercrime law and policy. Fake news and upcoming recent issues.
Courses related to E-commerce could be added IC and sustainable development Goals
Privacy and Freedom of Expression
Role play session may be address the multi stakeholder role in digital society.
One-day hands-on tutorial sessions on Al & Blockchain technologies sound great.
Since APSIG is for leaders in the field it becomes necessary to have roundtable discussions for
sharing experiences. Also group work and presentation of their findings brings the participation
to a higher level. That would work better option than just presenting case studies. Learning from
each other through discussions become important for APSIG participants because they are
leaders in the field and not armatures.
3.1.8. B. Session Evaluation Report by Gnanajeyaraman Rajaram[GJ]:
Participants will have lesson learned and understanding more on relevant issues but in different
countries/regions.
 Case study is the session that we would like to hear this kind of subjects. Collaboration of
speakers/presenters with moderator will enhance more and elaborate more, APSIG will
summarize of cases and further draw collective actions toward regional campaigns.
 It's good though to have discussions after the sharing. The time was well managed and
divided among the speakers.

9.  Especially appreciated the presentation on engagement of persons with disabilities, and
this topic could be potentially expanded and explored further in future APSIG events.
 All the reports must come after approval of the local SIG Teams.
 This is another session that I think will be valuable to have every year, so that national
SIGs can learn from each other and those countries who are still planning to have their
own SIGs can learn as well.
 Real time exercise or workshop needs to be focused on issues of AI and IoT governance.
Public policy process or commenting can be integrated in APSIG creates better document
and in open public policy process.

10. Ad-Hoc Session Analysis and suggestions
Case Study SessionChallenges:
1. Language barriers
2. Case study communication approach
3. Lack of Preparation ( infographics| Visual | Interview)
4. Time
5. Sense of Responsibility among fellows
6. How to use the network of Fellows (collaboration)
Engagement:
1. Participants interest need to be targeted with the current evolving and interesting topic
2. Participants needs to be given the time for self-preparation with PPT, lectures and other
materials
3. The Pre calls of the APSIG should be prioritized for participants with respect to the case
study so that it helps them create better understanding and preparation.
4. Evaluation and reports have to be made mandatory
5. Mentoring and group can be created with returning fellows for better coordination and
networking to further give the participants a better scope
6. During the Case study session the organizer or moderator should reach out to the participants
for engagement asking questions and various other exercises that helps in the facilitation
process
Recommendation and suggestion
1. Evaluation to be mandatory for all participants
2. Report to be mandatory for all fellows and can be compiled into a recommendation
document
3. Creating groups among the participants for future collaboration and competition
4. Mentorship with returning fellows or alumina for guidance and cooperation
5. Choosing topic of interest for case studies during application form
6. During the pre-calls materials and discussion about the session to be shared for
preparation

11. Lecture vs Case studies vs BoF
1. Approach
2. Presentation
3. Responsibility
1. Approach: The session organizers approach towards the topic is very important. He/She
should be more facilitating, informative and welcoming so that the participants are comfortable
and can ask questions. Friendly approach is recommended for instants asking question to the
participants
2. Presentation: Presentation matters the most, the session should include info-
graphic, videos, data etc. New friendly techniques should be welcomed.
3. Responsibility: Responsibility is yet another aspect where the participants have to be aware
and should be more focused towards the engagement of the program in every possible way. They
have to be made aware that they need to prepare for the program as the information and materials
are easily available are listed in the websites. As the pre meetings are there, they can ask
question and make themselves comfortable they need to bring their part of the knowledge from
their side.
This side is lacking hugely where a lot of them didn't evenfill the evaluation and
participants do not attend the Pre meetings. We talked about how the other internet events
make it compulsory to send the report and write blogs for the completion of the program
Challenges: The major challenge seen
1. Lack of interaction
2. Hesitation
3. Lack of Responsibility
4. Preparation among participants

12. Sessionare not the problem we just need to change the pattern
Types Scope
Lecture Core topics More than 10
Case Studies More researchbased
and informational
5
BoF Aggressive topicand
challenges of the society
10